Human action segmentation via controlled use of missing data in HMMs

Segmentation of individual actions from a stream of human motion is an open problem in computer vision. This paper approaches the problem of segmenting higher-level activities into their component sub-actions using Hidden Markov Models modified to handle missing data in the observation vector. By controlling the use of missing data, action labels can be inferred from the observation vector during inferencing, thus performing segmentation and classification simultaneously. The approach is able to segment both prominent and subtle actions, even when subtle actions are grouped together. The advantage of this method over sliding windows and Viterbi state sequence interrogation is that segmentation is performed as a trainable task, and the temporal relationship between actions is encoded in the model and used as evidence for action labelling.

Robust recognition and segmentation of human actions using HMMs with missing observations

This paper describes the integration of missing observation data with hidden Markov models to create a framework that is able to segment and classify individual actions from a stream of human motion using an incomplete 3D human pose estimation. Based on this framework, a model is trained to automatically segment and classify an activity sequence into its constituent subactions during inferencing. This is achieved by introducing action labels into the observation vector and setting these labels as missing data during inferencing, thus forcing the system to infer the probability of each action label. Additionally, missing data provides recognition-level support for occlusions and imperfect silhouette segmentation, permitting the use of a fast (real-time) pose estimation that delegates the burden of handling undetected limbs onto the action recognition system. Findings show that the use of missing data to segment activities is an accurate and elegant approach. Furthermore, action recognition can be accurate even when almost half of the pose feature data is missing due to occlusions, since not all of the pose data is important all of the time.

Anticipatory Planning Reveals Segmentation of Cortical Motor Output During Action Observation

It has been argued that the variation in brain activity that occurs when observing another person reflects a representation of actions that is indivisible, and which plays out in full once the intent of the actor can be discerned. We used transcranial magnetic stimulation to probe the excitability of corticospinal projections to 2 intrinsic hand muscles while motions to reach and grasp an object were observed. A symbolic cue either faithfully indicated the required final orientation of the object and thus the nature of the grasp that was required, or was in conflict with the movement subsequently displayed. When the cue was veridical, modulation of excitability was in accordance with the functional role of the muscles in the action observed. If however the cue had indicated that the alternative grasp would be required, modulation of output to first dorsal interosseus was consistent with the action specified, rather than the action observed-until the terminal phase of the motion sequence during which the object was seen lifted. Modulation of corticospinal output during observation is thus segmented-it progresses initially in accordance with the action anticipated, and if discrepancies are revealed by visual input, coincides thereafter with that of the action seen.

Multi-action recognition via stochastic modelling of optical flow and gradients

In this paper we propose a novel approach to multi-action recognition that performs joint segmentation and classification. This approach models each action using a Gaussian mixture using robust low-dimensional action features. Segmentation is achieved by performing classification on overlapping temporal windows, which are then merged to produce the final result. This approach is considerably less complicated than previous methods which use dynamic programming or computationally expensive hidden Markov models (HMMs). Initial experiments on a stitched version of the KTH dataset show that the proposed approach achieves an accuracy of 78.3%, outperforming a recent HMM-based approach which obtained 71.2%.

Learning Feature Trajectories Using Gabor Filter Bank for Human Activity Segmentation and Recognition

We describe a novel method for human activity segmentation and interpretation in surveillance applications based on Gabor filter-bank features. A complex human activity is modeled as a sequence of elementary human actions like walking, running, jogging, boxing, hand-waving etc. Since human silhouette can be modeled by a set of rectangles, the elementary human actions can be modeled as a sequence of a set of rectangles with different orientations and scales. The activity segmentation is based on Gabor filter-bank features and normalized spectral clustering. The feature trajectories of an action category are learnt from training example videos using dynamic time warping. The combined segmentation and the recognition processes are very efficient as both the algorithms share the same framework and Gabor features computed for the former can be used for the later. We have also proposed a simple shadow detection technique to extract good silhouette which is necessary for good accuracy of an action recognition technique.

Unconstrained monocular 3D human pose estimation by action detection and cross-modality regression forest

This work addresses the challenging problem of unconstrained 3D human pose estimation (HPE) from a novel perspective. Existing approaches struggle to operate in realistic applications, mainly due to their scene-dependent priors, such as background segmentation and multi-camera network, which restrict their use in unconstrained environments. We therfore present a framework which applies action detection and 2D pose estimation techniques to infer 3D poses in an unconstrained video. Action detection offers spatiotemporal priors to 3D human pose estimation by both recognising and localising actions in space-time. Instead of holistic features, e.g. silhouettes, we leverage the flexibility of deformable part model to detect 2D body parts as a feature to estimate 3D poses. A new unconstrained pose dataset has been collected to justify the feasibility of our method, which demonstrated promising results, significantly outperforming the relevant state-of-the-arts. © 2013 IEEE.

Segmentation et exclusion des Gitans de Perpignan : émergence d'une élite politique?

Dans la dernière décennie, une abondante littérature a documenté la situation des Rroms d'Europe Centrale et de l'Est, où a émergé une nouvelle élite politiquement activiste. Mais chez les Tsiganes d'Europe de l'Ouest, l’activisme politique d’une élite semblait absent. Cette étude de terrain a été réalisée chez Gitans de Perpignan, à la recherche d’une action et d’une élite politique chez ce groupe, dans le contexte culturel d’une société segmentaire à pouvoir diffus, frappée d’exclusion par la société majoritaire. En effet, je propose que le concept de société segmentaire puisse s’appliquer aux Gitans, et que l’exclusion des Gitans par les païos (non Gitans) constitue un déni de la réalité relationnelle des Gitans avec la majorité païa. Enfin, l’enquête a révélé la position de «médiateurs culturels» des différents agents qui interviennent entre le monde des Gitans et celui des païos. C’est à travers le rôle de «médiateurs culturels» qu’émerge peut-être une élite politique.

Choreographer-mathematician collaboration: developing machine segmentation techniques for motion capture analysis of dance

This paper/presentation focuses on the science of human movement, motion capture and cognition in action, and examines the necessity of choreographer-mathematician collaboration in developing appropriate analysis techniques.

Learning feature trajectories using Gabor Filter Bank for human activity segmentation and recognition

We describe a novel method for human activity segmentation and interpretation in surveillance applications based on Gabor filter-bank features. A complex human activity is modeled as a sequence of elementary human actions like walking, running, jogging, boxing, hand-waving etc. Since human silhouette can be modeled by a set of rectangles, the elementary human actions can be modeled as a sequence of a set of rectangles with different orientations and scales. The activity segmentation is based on Gabor filter-bank features and normalized spectral clustering. The feature trajectories of an action category are learnt from training example videos using Dynamic Time Warping. The combined segmentation and the recognition processes are very efficient as both the algorithms share the same framework and Gabor features computed for the former can be used for the later. We have also proposed a simple shadow detection technique to extract good silhouette which is necessary for good accuracy of an action recognition technique. © 2008 IEEE.

Automatic segmentation of relevant textures in agricultural images

One important issue emerging strongly in agriculture is related with the automatization of tasks, where the optical sensors play an important role. They provide images that must be conveniently processed. The most relevantimage processing procedures require the identification of green plants, in our experiments they come from barley and corn crops including weeds, so that some types of action can be carried out, including site-specific treatments with chemical products or mechanical manipulations. Also the identification of textures belonging to the soil could be useful to know some variables, such as humidity, smoothness or any others. Finally, from the point of view of the autonomous robot navigation, where the robot is equipped with the imaging system, some times it is convenient to know not only the soil information and the plants growing in the soil but also additional information supplied by global references based on specific areas. This implies that the images to be processed contain textures of three main types to be identified: green plants, soil and sky if any. This paper proposes a new automatic approach for segmenting these main textures and also to refine the identification of sub-textures inside the main ones. Concerning the green identification, we propose a new approach that exploits the performance of existing strategies by combining them. The combination takes into account the relevance of the information provided by each strategy based on the intensity variability. This makes an important contribution. The combination of thresholding approaches, for segmenting the soil and the sky, makes the second contribution; finally the adjusting of the supervised fuzzy clustering approach for identifying sub-textures automatically, makes the third finding. The performance of the method allows to verify its viability for automatic tasks in agriculture based on image processing