Human action segmentation via controlled use of missing data in HMMs

Segmentation of individual actions from a stream of human motion is an open problem in computer vision. This paper approaches the problem of segmenting higher-level activities into their component sub-actions using Hidden Markov Models modified to handle missing data in the observation vector. By controlling the use of missing data, action labels can be inferred from the observation vector during inferencing, thus performing segmentation and classification simultaneously. The approach is able to segment both prominent and subtle actions, even when subtle actions are grouped together. The advantage of this method over sliding windows and Viterbi state sequence interrogation is that segmentation is performed as a trainable task, and the temporal relationship between actions is encoded in the model and used as evidence for action labelling.

Local energy, the pre-envelope, and filter resolution

We examine the construction of new filters for computing local energy, and compare these filters with the Gabor filters and the three-point-filter of Venkatesh [l]. Further, we demonstrate that the effect of convolution with complex Gabor filters is to band-pass (with some differentiating effect) and compute the local energy of the result. The magnitude of the resulting local energy is then used to detect features [2], [3] (step features, texture etc.), and the phase is used to classify the detected features [l], [4] or provide disparity information for stereo [5] and motion work [6], [7]. Each of these types of information can be obtained at multiple resolutions, enabling the use of course to fine strategies for computing disparity, and allowing the discrimination of image textures on the basis of which parts of the Fourier domain they dominate [8], [9].

Video genre categorization using audio wavelet coefficients

In this paper, we investigate the use of a wavelet transform-based analysis of audio tracks accompanying videos for the problem of automatic program genre detection. We compare the classification performance based on wavelet-based audio features to that using conventional features derived from Fourier and time analysis for the task of discriminating TV programs such as news, commercials, music shows, concerts, motor racing games, and animated cartoons. Three different classifiers namely the Decision Trees, SVMs, and k-Nearest Neighbours are studied to analyse the reliability of the performance of our wavelet features based approach. Further, we investigate the issue of an appropriate duration of an audio clip to be analyzed for this automatic genre determination. Our experimental results show that features derived from the wavelet transform of the audio signal can very well separate the six video genres studied. It is also found that there is no significant difference in performance with varying audio clip durations across the classifiers.

An efficient least common subgraph algorithm for video indexing

Many tasks in computer vision can be expressed as graph problems. This allows the task to be solved using a well studied algorithm, however many of these algorithms are of exponential complexity. This is a disadvantage when considered in the context of searching a database of images or videos for similarity. Work by Mesaner and Bunke (1995) has suggested a new class of graph matching algorithms which uses a priori knowledge about a database of models to reduce the time taken during online classification. This paper presents a new algorithm which extends the earlier work to detection of the largest common subgraph.

Classifying complex human motion using point distribution models

The Point Distribution Model (PDM) has been successfully used in modelling shape variations in groups of static images. It has also been effectively adapted to temporal image sets and used to track moving bodies such as hands and walking persons. However standard models do not consider the temporal characteristics of the data and are purely models of shape. This research proposes an extension to the PDM which explicitly considers the temporal sequencing of the images in the motion. The modified model can then be built from temporal quantities such as linear velocity and acceleration which are derived from the images. The new model formulation also enables movements to be tracked and classified according to their distinguishing temporal characteristics. This has been tested against distinct sets of arm movements under varying sets of experimental conditions.

Coordination of multiple cameras to track multiple people

In this paper, we present a distributed surveillance system that uses multiple cheap static cameras to track multiple people in indoor environments. The system has a set of Camera Processing Modules and a Central Module to coordinate the tracking tasks among the cameras. Since each object in the scene can be tracked by a number of cameras, the problem is how to choose the most appropriate camera for each object. We propose a novel algorithm to allocate objects to cameras using the object-to-camera distance while taking into account occlusion. The algorithm attempts to assign objects in the overlapping fields of view to the nearest camera which can see the object without occlusion. Experimental results show that the system can coordinate cameras to track people properly and can deal well with occlusion.

Detecting deviations from known paths and speeds in a surveillance situation

This paper presents techniques for analysing human behaviour via video surveillance. In known scenes under surveillance, common paths of movement between entry and exit points are obtained and classified. These are used, together with a priori velocity data, to serve as a model of normal traffic flow in the scene. Surveillance sequences are then processed to extract and track the movement of people in the scene, which is compared with the models to enable detection of abnormal movement