Adaptive unsupervised learning of human actions

Automatic detection of suspicious activities in CCTV camera feeds is crucial to the success of video surveillance systems. Such a capability can help transform the dumb CCTV cameras into smart surveillance tools for fighting crime and terror. Learning and classification of basic human actions is a precursor to detecting suspicious activities. Most of the current approaches rely on a non-realistic assumption that a complete dataset of normal human actions is available. This paper presents a different approach to deal with the problem of understanding human actions in video when no prior information is available. This is achieved by working with an incomplete dataset of basic actions which are continuously updated. Initially, all video segments are represented by Bags-Of-Words (BOW) method using only Term Frequency-Inverse Document Frequency (TF-IDF) features. Then, a data-stream clustering algorithm is applied for updating the system's knowledge from the incoming video feeds. Finally, all the actions are classified into different sets. Experiments and comparisons are conducted on the well known Weizmann and KTH datasets to show the efficacy of the proposed approach.

3D Trajectory Recovery for Tracking Multiple Objects and Trajectory Guided Recognition of Actions

A mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. The 3D trajectory, occlusion, and segmentation information are utilized in extracting stabilized views of the moving object. Trajectory-guided recognition (TGR) is proposed as a new and efficient method for adaptive classification of action. The TGR approach is demonstrated using "motion history images" that are then recognized via a mixture of Gaussian classifier. The system was tested in recognizing various dynamic human outdoor activities; e.g., running, walking, roller blading, and cycling. Experiments with synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.

Trajectory Guided Tracking and Recognition of Actions

A combined 2D, 3D approach is presented that allows for robust tracking of moving people and recognition of actions. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. Low-level features are often insufficient for detection, segmentation, and tracking of non-rigid moving objects. Therefore, an improved mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. The 3D trajectory, occlusion, and segmentation information are utilized in extracting stabilized views of the moving object that are then used as input to action recognition modules. Trajectory-guided recognition (TGR) is proposed as a new and efficient method for adaptive classification of action. The TGR approach is demonstrated using "motion history images" that are then recognized via a mixture-of-Gaussians classifier. The system was tested in recognizing various dynamic human outdoor activities: running, walking, roller blading, and cycling. Experiments with real and synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.

Visual perception of movement kinematics and the acquisition of "action prototypes"

Recognizing a class of movements as belonging to a "nominal" action category, such as walking, running, or throwing, is a fundamental human ability. Three experiments were undertaken to test the hypothesis that common ("prototypical") features of moving displays could be learned by observation. Participants viewed moving stick-figure displays resembling forearm flexion movements in the saggital plane. Four displays (presentation displays) were first presented in which one or more movement dimensions were combined with 2 respective cues: direction (up, down), speed (fast, slow), and extent (long, short). Eight test displays were then shown, and the observer indicated whether each test display was like or unlike those previously seen. The results showed that without corrective feedback, a single cue (e.g., up or down) could be correctly recognized, on average, with the proportion correct between .66 and .87. When two cues were manipulated (e.g., up and slow), recognition accuracy remained high, ranging between .72 and .89. Three-cue displays were also easily identified. These results provide the first empirical demonstration of action-prototype learning for categories of human action and show how apparently complex kinematic patterns can be categorized in terms of common features or cues. It was also shown that probability of correct recognition of kinematic properties was reduced when the set of 4 presentation displays were more variable with respect to their shared kinematic property, such as speed or amplitude. Finally, while not conclusive, the results (from 2 of the 3 experiments) did suggest that similarity (or "likeness") with respect to a common kinematic property (or properties) is more easily recognized than dissimilarity.

Human MutSalpha recognizes damaged DNA base pairs containing O6-methylguanine, O4-methylthymine, or the cisplatin-d(GpG) adduct.

Bacterial and mammalian mismatch repair systems have been implicated in the cellular response to certain types of DNA damage, and genetic defects in this pathway are known to confer resistance to the cytotoxic effects of DNA-methylating agents. Such observations suggest that in addition to their ability to recognize DNA base-pairing errors, members of the MutS family may also respond to genetic lesions produced by DNA damage. We show that the human mismatch recognition activity MutSalpha recognizes several types of DNA lesion including the 1,2-intrastrand d(GpG) crosslink produced by cis-diamminedichloroplatinum(II), as well as base pairs between O6-methylguanine and thymine or cytosine, or between O4-methylthymine and adenine. However, the protein fails to recognize 1,3-intrastrand adduct produced by trans-diamminedichloroplatinum(II) at a d(GpTpG) sequence. These observations imply direct involvement of the mismatch repair system in the cytotoxic effects of DNA-methylating agents and suggest that recognition of 1,2-intrastrand cis-diamminedichloroplatinum(II) adducts by MutSalpha may be involved in the cytotoxic action of this chemotherapeutic agent.

A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences

Human behaviour recognition has been, and still remains, a challenging problem that involves different areas of computational intelligence. The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts. In this paper, the problem is studied from a prediction point of view. We propose a novel method able to early detect behaviour using a small portion of the input, in addition to the capabilities of it to predict behaviour from new inputs. Specifically, we propose a predictive method based on a simple representation of trajectories of a person in the scene which allows a high level understanding of the global human behaviour. The representation of the trajectory is used as a descriptor of the activity of the individual. The descriptors are used as a cue of a classification stage for pattern recognition purposes. Classifiers are trained using the trajectory representation of the complete sequence. However, partial sequences are processed to evaluate the early prediction capabilities having a specific observation time of the scene. The experiments have been carried out using the three different dataset of the CAVIAR database taken into account the behaviour of an individual. Additionally, different classic classifiers have been used for experimentation in order to evaluate the robustness of the proposal. Results confirm the high accuracy of the proposal on the early recognition of people behaviours.

Modelling human behaviour and event detection from videos

Automatic analysis of human behaviour in large collections of videos is gaining interest, even more so with the advent of file sharing sites such as YouTube. However, challenges still exist owing to several factors such as inter- and intra-class variations, cluttered backgrounds, occlusion, camera motion, scale, view and illumination changes. This research focuses on modelling human behaviour for action recognition in videos. The developed techniques are validated on large scale benchmark datasets and applied on real-world scenarios such as soccer videos. Three major contributions are made. The first contribution is in the area of proper choice of a feature representation for videos. This involved a study of state-of-the-art techniques for action recognition, feature extraction processing and dimensional reduction techniques so as to yield the best performance with optimal computational requirements. Secondly, temporal modelling of human behaviour is performed. This involved frequency analysis and temporal integration of local information in the video frames to yield a temporal feature vector. Current practices mostly average the frame information over an entire video and neglect the temporal order. Lastly, the proposed framework is applied and further adapted to real-world scenario such as soccer videos. A dataset consisting of video sequences depicting events of players falling is created from actual match data to this end and used to experimentally evaluate the proposed framework.

Dynamics of contour, object and face processing in the human visual cortex

The neural basis of visual perception can be understood only when the sequence of cortical activity underlying successful recognition is known. The early steps in this processing chain, from retina to the primary visual cortex, are highly local, and the perception of more complex shapes requires integration of the local information. In Study I of this thesis, the progression from local to global visual analysis was assessed by recording cortical magnetoencephalographic (MEG) responses to arrays of elements that either did or did not form global contours. The results demonstrated two spatially and temporally distinct stages of processing: The first, emerging 70 ms after stimulus onset around the calcarine sulcus, was sensitive to local features only, whereas the second, starting at 130 ms across the occipital and posterior parietal cortices, reflected the global configuration. To explore the links between cortical activity and visual recognition, Studies II III presented subjects with recognition tasks of varying levels of difficulty. The occipito-temporal responses from 150 ms onwards were closely linked to recognition performance, in contrast to the 100-ms mid-occipital responses. The averaged responses increased gradually as a function of recognition performance, and further analysis (Study III) showed the single response strengths to be graded as well. Study IV addressed the attention dependence of the different processing stages: Occipito-temporal responses peaking around 150 ms depended on the content of the visual field (faces vs. houses), whereas the later and more sustained activity was strongly modulated by the observers attention. Hemodynamic responses paralleled the pattern of the more sustained electrophysiological responses. Study V assessed the temporal processing capacity of the human object recognition system. Above sufficient luminance, contrast and size of the object, the processing speed was not limited by such low-level factors. Taken together, these studies demonstrate several distinct stages in the cortical activation sequence underlying the object recognition chain, reflecting the level of feature integration, difficulty of recognition, and direction of attention.

Multidimensionality of Actors in Business Networks: The Influence of Social Action in Pharmacy Networks in Finland

In the markets-as-networks approach business networks are conceived as dynamic actor structures, giving focus to exchange relationships and actors’ capabilities to control and co-ordinate activities and resources. Researchers have shared an understanding that actors’ actions are crucial for the development of business networks and for network dynamics. However, researchers have mainly studied firms as business actors and excluded individuals, although both firms and individuals can be seen as business actors. This focus on firms as business actors has resulted in a paucity of research on human action and the exchange of intangible resources in business networks, e.g. social exchange between individuals in social networks. Consequently, the current conception of business networks fails to appreciate the richness of business actors, the human character of business action and the import of social action in business networks. The central assumption in this study is that business actors are multidimensional and that their specific constitution in any given situation is determined by human interaction in social networks. Multidimensionality is presented as a concept for exploring how business actors act in different situations and how actors simultaneously manage multiple identities: individual, organisational, professional, business and network identities. The study presents a model that describes the multidimensionality of actors in business networks and conceptualises the connection between social exchange and human action in business networks. Empirically the study explores the change that has taken place in pharmaceutical retailing in Finland during recent years. The phenomenon of emerging pharmacy networks is highly contemporary in the Nordic countries, where the traditional license-based pharmacy business is changing. The study analyses the development of two Finnish pharmacy chains, one integrated and one voluntary chain, and the network structures and dynamics in them. Social Network Analysis is applied to explore the social structures within the pharmacy networks. The study shows that emerging pharmacy networks are multifaceted phenomena where political, economic, social, cultural, and historical elements together contribute to the observed changes. Individuals have always been strongly present in the pharmacy business and the development of pharmacy networks provides an interesting example of human actors’ influence in the development of business networks. The dynamics or forces driving the network development can be linked to actors’ own economic and social motives for developing the business. The study highlights the central role of individuals and social networks in the development of the two studied pharmacy networks. The relation between individuals and social networks is reciprocal. The social context of every individual enables multidimensional business actors. The mix of various identities, both individual and collective identities, is an important part of network dynamics. Social networks in pharmacy networks create a platform for exchange and social action, and social networks enable and support business network development.

学习不良少年的自我意识情绪研究

Self-conscious emotions (guilt, shame, embarrassment, pride, etc) are social emotions, and involve complex appraisals of how one’s behavior has been evaluated by the self and other people according to some value standards. Self-conscious emotions play an important role in human life by arousing and regulating human action tendencies, feeling and thoughts, which can promote people to work hard in achievement and task fields, maintain good interpersonal relationship according with social morality and expectation. The present study aimed to examine complex self-conscious emotional understanding capabilities in junior middle school students with and without learning disabilities, how the self-conscious emotions generate, and relationship between self-conscious emotions and self-representation in academic and interpersonal fields. Situational experimental methods were used in this research, and the results would give further supports for learning disabilities intervention. The main results of present research are as follows. 1. The study included 4 parts and 6 experiments. The aim of study 1 was to explore whether juveniles with learning disabilities understood complex self-conscious emotions differently from juveniles without learning disabilities. We surveyed the self-conscious emotions understanding of 37 learning disabilities and 45 non-learning disabilities with the emotional situation stories. The results indicated that the self-conscious emotional recognition in others for learning disabilities was lower than that of non-learning disabilities in different emotional recognition tasks. Moreover, children with learning disabilities were more inclined to recognize emotions in themselves as elemental emotions, however, children without learning disabilities were more inclined to recognize emotions in themselves as self-conscious emotions. 2. The aim of study 2 was to explore the generative mechanism of self-conscious emotions in academic and interpersonal fields with the method of situational experiments, namely to examine whether the self-discrepancy could cause self-conscious emotions for learning disabilities. 84 learning disabilities (in experiment 1) and 80 learning disabilities (in experiment 2) participated in the research, and the results were as follows. (1) Self discrepancy caused participants’ self-conscious emotions effectively in academic and interpersonal fields. One’s own and parents’ perspercive on the actual-ideal self-discrepancy both produced dejection-related emotions (shame、embarrassment) and agitation-related emotions (guilt). （2）In academic fields, children with learning disabilities caused higher level negative self-conscious emotions (embarrassment, shame, and guilt) and lower level positive self-conscious emotion (pride). However, there were no differences of self-conscious emotions for children with and without learning disabilities in non-academic fields. 3. The aim of study 3 was to explore what influence had self-conscious emotions on self-representation for learning disabilities with the method of situational experiments. 57 learning disabilities (in experiment 1) and 67 learning disabilities (in experiment 2) participated in the research, and the results were as follows. （1）The negative self-conscious for learning disabilities could influence their positive or negative academic and positive interpersonal self-representation stability, the ways in which self-evaluation of ability mediate these effects. However, there was no significant effect for the negative self-conscious and self-evaluation of ability predicting negative interpersonal self-representation stability. （2）The stability level of positive academic and interpersonal self-representation for learning disabilities was lower than that of non-learning disabilities. There was no significant difference of the negative interpersonal self-representation stability for children with and without learning disabilities in the positive self-conscious valence condition. However, the stability level of negative interpersonal self-representation for learning disabilities was lower than that of non-learning disabilities in the negative self-conscious valence condition. 4. The aim of study 4 was to explore the intervention effects for self-conscious emotions training course on emotional comprehension cability. 65 learning disabilities (34 in experimental group, and 31 in control group) participated in the research. The results showed that self-conscious emotions course boosted the self-conscious emotions apprehensive level for children with learning disabilities.

An efficient feature selection method for mobile devices with application to activity recognition

This paper presents a feature selection method for data classification, which combines a model-based variable selection technique and a fast two-stage subset selection algorithm. The relationship between a specified (and complete) set of candidate features and the class label is modelled using a non-linear full regression model which is linear-in-the-parameters. The performance of a sub-model measured by the sum of the squared-errors (SSE) is used to score the informativeness of the subset of features involved in the sub-model. The two-stage subset selection algorithm approaches a solution sub-model with the SSE being locally minimized. The features involved in the solution sub-model are selected as inputs to support vector machines (SVMs) for classification. The memory requirement of this algorithm is independent of the number of training patterns. This property makes this method suitable for applications executed in mobile devices where physical RAM memory is very limited. An application was developed for activity recognition, which implements the proposed feature selection algorithm and an SVM training procedure. Experiments are carried out with the application running on a PDA for human activity recognition using accelerometer data. A comparison with an information gain based feature selection method demonstrates the effectiveness and efficiency of the proposed algorithm.

Mediated action, narratives of risk-taking, and identity formation in adolescents with a visual impairment

The purpose of this project was to discern the inherent tension present in narratives told by adolescents with a visual impairment as they attempted to make sense of their experiences, specifically those surrounding risk. Mediated action, based on the foundational work of Vygotsky and Bakhtin, was used as both a theoretical and methodological approach; it is the theory that there are two components that constitute any human action: the "agent," or the person who is doing the acting, and the "mediational means" that he or she is using to accomplish the action in question. Tension ensues as neither is able to fully explain human behaviour. Ten adolescents with a visual impairment participated in a narrative interview, revealing numerous counter-narratives surrounding risk-taking, including "experimentation undertaken using good judgment." Participants offered examples of how they engaged, appropriated, resisted and transformed the dominant narratives of disability and adolescence in their identity formation.

La capacité d'action collective des populations marginalisées dans le cadre des stratégies de lutte pour la reconnaissance : les cas de Cité de l'Éternel à Port-au-Prince (Haïti) et de la Sierra Santa Catarina (Mexico)

Cette thèse analyse la capacité d’action collective des populations marginalisées situées respectivement dans un bidonville appelé Cité de l’Éternel à Port-au-Prince (Haïti) et dans des campements à la Sierra Santa Catarina, Iztapalapa (Mexico). À Port-au-Prince, avant la chute de la dictature des Duvalier, des «tontons macoutes» envahirent un terrain situé en bordure du quai de la capitale, près du boulevard Harry Truman. Après s’y être installés, ils ont procédé à la vente de parcelles destinées à la construction de logement à des particuliers. Mais après la chute de Jean-Claude Duvalier, en 1986, des gens de la populace en ont profité pour envahir ce qui restait de ces terrains marécageux. Après l’occupation, ils se sont organisés pour défendre collectivement leur propriété avant d’entreprendre, par la suite, des démarches pour y amener des services et obtenir la régularisation de leur situation sur ces territoires. À la Sierra Santa Catarina, des populations conduites par des militants d’un Front populaire ont occupé des terrains situés au pied d’une montagne de sable afin d’accéder à la propriété et de construire leur demeure. À l’instar des populations de la Cité, ces gens se sont organisés pour aménager des espaces, y ériger des logements provisoires, monter la garde afin de ne pas être déguerpis par les forces de police. Tout en travaillant pour accéder à leur manière aux services de base, elles entreprennent des actions auprès des institutions publiques afin d’obtenir la régularisation de leur situation. Par rapport à la capacité d’action collective de ces populations, les théories sociologiques sont divisées. D’un côté, certains auteurs soutiennent la thèse de l’incapacité de ces populations d’avoir des intérêts collectifs et d’agir en conséquence. Selon eux, sans une médiation sociale ou à défaut d’une agrégation et d’une représentation politiques, ces populations sont incapables d’avoir une subjectivité collective. De l’autre, des auteurs pensent qu’à partir des liens d’amitié de parenté et de voisinage, indépendamment de leurs situations socioéconomiques, ces populations peuvent créer des stratégies de subsistance et de luttes qui leur permettent de trouver des solutions à des problèmes tant individuels que collectifs. S’agissant des populations qui envahissent des terrains en milieu urbain pour habiter, les actions de ces gens là sont définis déjà comme une forme d’action collective inscrite dans des rapports sociaux qui se caractérisent par la différenciation entre les groupes sociaux dans l’accès à la propriété. Ainsi, leurs revendications de reconnaissance et de régularisation auprès des instances étatiques sont déterminées par leur mode d’accès aux biens et aux richesses inégalement réparties en Haïti et au Mexique. Les populations des deux territoires ont entrepris diverses démarches auprès de certaines institutions et réalisé des actions collectives soit pour amener des services de base tels que l’eau et l’électricité, soit pour obtenir de l’État la reconnaissance des territoires envahis, c’est-à-dire leur jonction à la cartographie de la ville. Cette reconnaissance implique non seulement l’installation des services réguliers au bénéfice de la population mais aussi l’octroi à chaque propriétaire de son titre de propriété. Si dans le cas de la Sierra Santa Catarina les démarches sont entreprises auprès des institutions publiques, dans celui de Port-au-Prince, ce sont les ONG ou les agence de coopération qui sont touchées et qui fournissent certains services à la population conformément à la priorité de leurs bailleurs de fonds. Les interventions auprès de l’État se font plutôt dans le but d’obtenir une autorisation de fonctionnement d’une association locale. Il ressort des approches théoriques et des actions collectives réalisées par ces populations qu’on ne peut pas dire qu’elles sont incapables d’avoir une subjectivité collective et des intérêts communs sans une agrégation et une représentation politique. À partir de différents liens entre les individus, des associations sont créées lesquelles permettent d’établir une médiation entre les populations et d’autres organismes. Dans le cas des campements, les actions collectives sont certainement mises à contribution par quelques leaders. Cela participe de toute une tradition politique au Mexique. Néanmoins, dans certains campements, des populations parviennent à tenir tête jusqu’à révoquer certains leaders. Au-delà de leur situation socioéconomique, de l’emprise de certains dirigeants de campement, de l’indifférence de l’État (dans le cas de Port-au-Prince, notamment), ces populations font preuve d’une étonnante capacité critique de leur situation tant dans leurs relations avec les dirigeants des associations et des campements que par rapport à l’État. Ceci pourrait soulever des doutes quant à la possibilité qu’elles soient réellement ou inconsciemment manipulées. Cela suggère la possibilité d’actions collectives autonomes de portée critique là où les circonstances le permettent Mots clés : Marginalisation, action collective, reconnaissance, Débrouille, capacité critique, bidonvilles, instrumentalisation politique, reconnaissance fragmentée.

Reconnaissance des actions humaines à partir d'une séquence vidéo

The work done in this master's thesis, presents a new system for the recognition of human actions from a video sequence. The system uses, as input, a video sequence taken by a static camera. A binary segmentation method of the the video sequence is first achieved, by a learning algorithm, in order to detect and extract the different people from the background. To recognize an action, the system then exploits a set of prototypes generated from an MDS-based dimensionality reduction technique, from two different points of view in the video sequence. This dimensionality reduction technique, according to two different viewpoints, allows us to model each human action of the training base with a set of prototypes (supposed to be similar for each class) represented in a low dimensional non-linear space. The prototypes, extracted according to the two viewpoints, are fed to a $K$-NN classifier which allows us to identify the human action that takes place in the video sequence. The experiments of our model conducted on the Weizmann dataset of human actions provide interesting results compared to the other state-of-the art (and often more complicated) methods. These experiments show first the sensitivity of our model for each viewpoint and its effectiveness to recognize the different actions, with a variable but satisfactory recognition rate and also the results obtained by the fusion of these two points of view, which allows us to achieve a high performance recognition rate.

Translation Invariance in Object Recognition, and Its Relation to Other Visual Transformations

Human object recognition is generally considered to tolerate changes of the stimulus position in the visual field. A number of recent studies, however, have cast doubt on the completeness of translation invariance. In a new series of experiments we tried to investigate whether positional specificity of short-term memory is a general property of visual perception. We tested same/different discrimination of computer graphics models that were displayed at the same or at different locations of the visual field, and found complete translation invariance, regardless of the similarity of the animals and irrespective of direction and size of the displacement (Exp. 1 and 2). Decisions were strongly biased towards same decisions if stimuli appeared at a constant location, while after translation subjects displayed a tendency towards different decisions. Even if the spatial order of animal limbs was randomized ("scrambled animals"), no deteriorating effect of shifts in the field of view could be detected (Exp. 3). However, if the influence of single features was reduced (Exp. 4 and 5) small but significant effects of translation could be obtained. Under conditions that do not reveal an influence of translation, rotation in depth strongly interferes with recognition (Exp. 6). Changes of stimulus size did not reduce performance (Exp. 7). Tolerance to these object transformations seems to rely on different brain mechanisms, with translation and scale invariance being achieved in principle, while rotation invariance is not.