Learning discriminative sequence models from partially labelled data for activity recognition

Recognising daily activity patterns of people from low-level sensory data is an important problem. Traditional approaches typically rely on generative models such as the hidden Markov models and training on fully labelled data. While activity data can be readily acquired from pervasive sensors, e.g. in smart environments, providing manual labels to support fully supervised learning is often expensive. In this paper, we propose a new approach based on partially-supervised training of discriminative sequence models such as the conditional random field (CRF) and the maximum entropy Markov model (MEMM). We show that the approach can reduce labelling effort, and at the same time, provides us with the flexibility and accuracy of the discriminative framework. Our experimental results in the video surveillance domain illustrate that these models can perform better than their generative counterpart (i.e. the partially hidden Markov model), even when a substantial amount of labels are unavailable.

Generative models for automatic recognition of human daily activities from a single triaxial accelerometer

In this work, we compare two generative models including Gaussian Mixture Model (GMM) and Hidden Markov Model (HMM) with Support Vector Machine (SVM) classifier for the recognition of six human daily activity (i.e., standing, walking, running, jumping, falling, sitting-down) from a single waist-worn tri-axial accelerometer signals through 4-fold cross-validation and testing on a total of thirteen subjects, achieving an average recognition accuracy of 96.43% and 98.21% in the first experiment and 95.51% and 98.72% in the second, respectively. The results demonstrate that both HMM and GMM are not only able to learn but also capable of generalization while the former outperformed the latter in the recognition of daily activities from a single waist worn tri-axial accelerometer. In addition, these two generative models enable the assessment of human activities based on acceleration signals with varying lengths.

Link between vertical and horizontal movement patterns of cod in the North Sea

We used a geolocation method based on tidal amplitude and water depth to assess the horizontal movements of 14 cod Gadus morhua equipped with time-depth recorders (TDR) in the North Sea and English Channel. Tracks ranged from 40 to 468 d and showed horizontal movements of up to 455 km and periods of continuous localised residence of up to 360 d. Cod spent time both in midwater (43% of total time) and near the seabed (57% of total time). A variety of common vertical movement patterns were seen within periods of both residence and directed horizontal movement. Hence particular patterns of vertical movement could not unequivocally define periods of migration or localised residence. After long horizontal movements, cod tended to adopt resident behaviour for several months and then return to broadly the same location where they were tagged, indicating a geospatial instinct. The results suggest that residence and homing behaviour are important features of Atlantic cod behaviour.

Fully automatic localization and segmentation of 3D vertebral bodies from CT/MR images via a learning-based method

In this paper, we address the problems of fully automatic localization and segmentation of 3D vertebral bodies from CT/MR images. We propose a learning-based, unified random forest regression and classification framework to tackle these two problems. More specifically, in the first stage, the localization of 3D vertebral bodies is solved with random forest regression where we aggregate the votes from a set of randomly sampled image patches to get a probability map of the center of a target vertebral body in a given image. The resultant probability map is then further regularized by Hidden Markov Model (HMM) to eliminate potential ambiguity caused by the neighboring vertebral bodies. The output from the first stage allows us to define a region of interest (ROI) for the segmentation step, where we use random forest classification to estimate the likelihood of a voxel in the ROI being foreground or background. The estimated likelihood is combined with the prior probability, which is learned from a set of training data, to get the posterior probability of the voxel. The segmentation of the target vertebral body is then done by a binary thresholding of the estimated probability. We evaluated the present approach on two openly available datasets: 1) 3D T2-weighted spine MR images from 23 patients and 2) 3D spine CT images from 10 patients. Taking manual segmentation as the ground truth (each MR image contains at least 7 vertebral bodies from T11 to L5 and each CT image contains 5 vertebral bodies from L1 to L5), we evaluated the present approach with leave-one-out experiments. Specifically, for the T2-weighted MR images, we achieved for localization a mean error of 1.6 mm, and for segmentation a mean Dice metric of 88.7% and a mean surface distance of 1.5 mm, respectively. For the CT images we achieved for localization a mean error of 1.9 mm, and for segmentation a mean Dice metric of 91.0% and a mean surface distance of 0.9 mm, respectively.

Bayesian nonparametric approaches to abnormality detection in video surveillance

In data science, anomaly detection is the process of identifying the items, events or observations which do not conform to expected patterns in a dataset. As widely acknowledged in the computer vision community and security management, discovering suspicious events is the key issue for abnormal detection in video surveil-lance. The important steps in identifying such events include stream data segmentation and hidden patterns discovery. However, the crucial challenge in stream data segmenta-tion and hidden patterns discovery are the number of coherent segments in surveillance stream and the number of traffic patterns are unknown and hard to specify. Therefore, in this paper we revisit the abnormality detection problem through the lens of Bayesian nonparametric (BNP) and develop a novel usage of BNP methods for this problem. In particular, we employ the Infinite Hidden Markov Model and Bayesian Nonparamet-ric Factor Analysis for stream data segmentation and pattern discovery. In addition, we introduce an interactive system allowing users to inspect and browse suspicious events.

Prosthetic motor imaginary task classification using single channel of electroencephalography

Brain Computer Interface (BCI) is playing a very important role in human machine communications. Recent communication systems depend on the brain signals for communication. In these systems, users clearly manipulate their brain activity rather than using motor movements in order to generate signals that could be used to give commands and control any communication devices, robots or computers. In this paper, the aim was to estimate the performance of a brain computer interface (BCI) system by detecting the prosthetic motor imaginary tasks by using only a single channel of electroencephalography (EEG). The participant is asked to imagine moving his arm up or down and our system detects the movement based on the participant brain signal. Some features are extracted from the brain signal using Mel-Frequency Cepstrum Coefficient and based on these feature a Hidden Markov model is used to help in knowing if the participant imagined moving up or down. The major advantage in our method is that only one channel is needed to take the decision. Moreover, the method is online which means that it can give the decision as soon as the signal is given to the system. Hundred signals were used for testing, on average 89 % of the up down prosthetic motor imaginary tasks were detected correctly. This method can be used in many different applications such as: moving artificial prosthetic limbs and wheelchairs due to it's high speed and accuracy.

First comprehensive multi-tissue transcriptome of Cherax quadricarinatus (Decapoda: Parastacidae) reveals unexpected diversity of endogenous cellulase

The Australian freshwater crayfish species, Cherax quadricarinatus Von Martens, 1868, is an important commercial and invasive species that is also being increasingly used as a model organism to address important and interesting questions in crustacean biology. Through deep sequencing of the transcriptome of C. quadricarinatus from the hepatopancreas and four other tissues, we examine the evolution of endogenously transcribed cellulase genes and provide new insights into controversial issues regarding the nutritional biology of crayfishes. A cluster assembly approach yielded one of the highest quality transcriptome assemblies for a decapod crustacean to date. A total of 206,341,872 reads with an average read length of 80 bp were generated from sequencing the transcriptomes from the heart, kidney, hepatopancreas, nerve, and testis tissues. The assembled transcriptome contains a total of 44,525 transcripts. A total of 65 transcripts coding for carbohydrate-active enzymes (CAZy) were identified based on hidden Markov model (HMM), and a majority of them display high relative transcript abundance in the hepatopancreas tissue, supporting their role in nutrient digestion. Comprehensive phylogenetic analyses of proteins belonging to two main glycosyl hydrolase families (GH9 and GH5) suggest shared ancestry of C. quadricarinatus cellulases with other characterized crustacean cellulases. Our study significantly expands the number of known crustacean-derived CAZy-coding transcripts. More importantly, the surprising level of evolutionary diversification of these proteins in C. quadricarinatus suggests that these enzymes may have been of critical importance in the adaptation of freshwater crayfishes to new plant-based food sources as part of their successful invasion of freshwater systems from marine ancestors.

Learning people movement model from multiple cameras for behaviour recognition

In surveillance systems for monitoring people behaviours, it is important to build systems that can adapt to the signatures of people's tasks and movements in the environment. At the same time, it is important to cope with noisy observations produced by a set of cameras with possibly different characteristics. In previous work, we have implemented a distributed surveillance system designed for complex indoor environments [1]. The system uses the Abstract Hidden Markov mEmory Model (AHMEM) for modelling and specifying complex human behaviours that can take place in the environment. Given a sequence of observations from a set of cameras, the system employs approximate probabilistic inference to compute the likelihood of different possible behaviours in real-time. This paper describes the techniques that can be used to learn the different camera noise models and the human movement models to be used in this system. The system is able to monitor and classify people behaviours as data is being gathered, and we provide classification results showing the system is able to identify behaviours of people from their movement signatures.

Hierarchical semi-markov conditional random fields for recursive sequential data

Inspired by the hierarchical hidden Markov models (HHMM), we present the hierarchical semi-Markov conditional random field (HSCRF), a generalisation of embedded undirected Markov chains to model complex hierarchical, nested Markov processes. It is parameterised in a discriminative framework and has polynomial time algorithms for learning and inference. Importantly, we develop efficient algorithms for learning and constrained inference in a partially-supervised setting, which is important issue in practice where labels can only be obtained sparsely. We demonstrate the HSCRF in two applications: (i) recognising human activities of daily living (ADLs) from indoor surveillance cameras, and (ii) noun-phrase chunking. We show that the HSCRF is capable of learning rich hierarchical models with reasonable accuracy in both fully and partially observed data cases.

Robust recognition and segmentation of human actions using HMMs with missing observations

This paper describes the integration of missing observation data with hidden Markov models to create a framework that is able to segment and classify individual actions from a stream of human motion using an incomplete 3D human pose estimation. Based on this framework, a model is trained to automatically segment and classify an activity sequence into its constituent subactions during inferencing. This is achieved by introducing action labels into the observation vector and setting these labels as missing data during inferencing, thus forcing the system to infer the probability of each action label. Additionally, missing data provides recognition-level support for occlusions and imperfect silhouette segmentation, permitting the use of a fast (real-time) pose estimation that delegates the burden of handling undetected limbs onto the action recognition system. Findings show that the use of missing data to segment activities is an accurate and elegant approach. Furthermore, action recognition can be accurate even when almost half of the pose feature data is missing due to occlusions, since not all of the pose data is important all of the time.

AdaBoost.MRF: boosted Markov random forests and application to multilevel activity recognition

Activity recognition is an important issue in building intelligent monitoring systems. We address the recognition of multilevel activities in this paper via a conditional Markov random field (MRF), known as the dynamic conditional random field (DCRF). Parameter estimation in general MRFs using maximum likelihood is known to be computationally challenging (except for extreme cases), and thus we propose an efficient boosting-based algorithm AdaBoost.MRF for this task. Distinct from most existing work, our algorithm can handle hidden variables (missing labels) and is particularly attractive for smarthouse domains where reliable labels are often sparsely observed. Furthermore, our method works exclusively on trees and thus is guaranteed to converge. We apply the AdaBoost.MRF algorithm to a home video surveillance application and demonstrate its efficacy.

Human action segmentation via controlled use of missing data in HMMs

Segmentation of individual actions from a stream of human motion is an open problem in computer vision. This paper approaches the problem of segmenting higher-level activities into their component sub-actions using Hidden Markov Models modified to handle missing data in the observation vector. By controlling the use of missing data, action labels can be inferred from the observation vector during inferencing, thus performing segmentation and classification simultaneously. The approach is able to segment both prominent and subtle actions, even when subtle actions are grouped together. The advantage of this method over sliding windows and Viterbi state sequence interrogation is that segmentation is performed as a trainable task, and the temporal relationship between actions is encoded in the model and used as evidence for action labelling.