Kinect and Episodic Reasoning for Human Action Recognition

This paper presents a method for rational behaviour recognition that combines vision-based pose estimation with knowledge modeling and reasoning. The proposed method consists of two stages. First, RGB-D images are used in the estimation of the body postures. Then, estimated actions are evaluated to verify that they make sense. This method requires rational behaviour to be exhibited. To comply with this requirement, this work proposes a rational RGB-D dataset with two types of sequences, some for training and some for testing. Preliminary results show the addition of knowledge modeling and reasoning leads to a significant increase of recognition accuracy when compared to a system based only on computer vision.

ReadingAct RGB-D action dataset and human action recognition from local features

For general home monitoring, a system should automatically interpret people’s actions. The system should be non-intrusive, and able to deal with a cluttered background, and loose clothes. An approach based on spatio-temporal local features and a Bag-of-Words (BoW) model is proposed for single-person action recognition from combined intensity and depth images. To restore the temporal structure lost in the traditional BoW method, a dynamic time alignment technique with temporal binning is applied in this work, which has not been previously implemented in the literature for human action recognition on depth imagery. A novel human action dataset with depth data has been created using two Microsoft Kinect sensors. The ReadingAct dataset contains 20 subjects and 19 actions for a total of 2340 videos. To investigate the effect of using depth images and the proposed method, testing was conducted on three depth datasets, and the proposed method was compared to traditional Bag-of-Words methods. Results showed that the proposed method improves recognition accuracy when adding depth to the conventional intensity data, and has advantages when dealing with long actions.

Human action categorization using conditional random field

Automatic human action recognition has been a challenging issue in the field of machine vision. Some high-level features such as SIFT, although with promising performance for action recognition, are computationally complex to some extent. To deal with this problem, we construct the features based on the Distance Transform of body contours, which is relatively simple and computationally efficient, to represent human action in the video. After extracting the features from videos, we adopt the Conditional Random Field for modeling the temporal action sequences. The proposed method is tested with an available standard dataset. We also testify the robustness of our method on various realistic conditions, such as body occlusion or intersection.

Human action recognition based on 3D SIFT and LDA model

How to recognize human action from videos captured by modern cameras efficiently and effectively is a challenge in real applications. Traditional methods which need professional analysts are facing a bottleneck because of their shortcomings. To cope with the disadvantage, methods based on computer vision techniques, without or with only a few human interventions, have been proposed to analyse human actions in videos automatically. This paper provides a method combining the three dimensional Scale Invariant Feature Transform (SIFT) detector and the Latent Dirichlet Allocation (LDA) model for human motion analysis. To represent videos effectively and robustly, we extract the 3D SIFT descriptor around each interest point, which is sampled densely from 3D Space-time video volumes. After obtaining the representation of each video frame, the LDA model is adopted to discover the underlying structure-the categorization of human actions in the collection of videos. Public available standard datasets are used to test our method. The concluding part discusses the research challenges and future directions.

Human action recognition based on pyramid histogram of oriented gradients

Human action recognition has been attracted lots of interest from computer vision researchers due to its various promising applications. In this paper, we employ Pyramid Histogram of Orientation Gradient (PHOG) to characterize human figures for action recognition. Comparing to silhouette-based features, the PHOG descriptor does not require extraction of human silhouettes or contours. Two state-space models, i.e.; Hidden Markov Model (HMM) and Conditional Random Field (CRF), are adopted to model the dynamic human movement. The proposed PHOG descriptor and the state-space models with respect to different parameters are tested using a standard dataset. We also testify the robustness of the method with respect to various unconstrained conditions and viewpoints. Promising experimental result demonstrates the effectiveness and robustness of our proposed method.

Human action segmentation via controlled use of missing data in HMMs

Segmentation of individual actions from a stream of human motion is an open problem in computer vision. This paper approaches the problem of segmenting higher-level activities into their component sub-actions using Hidden Markov Models modified to handle missing data in the observation vector. By controlling the use of missing data, action labels can be inferred from the observation vector during inferencing, thus performing segmentation and classification simultaneously. The approach is able to segment both prominent and subtle actions, even when subtle actions are grouped together. The advantage of this method over sliding windows and Viterbi state sequence interrogation is that segmentation is performed as a trainable task, and the temporal relationship between actions is encoded in the model and used as evidence for action labelling.

A framework to understand human action

O trabalho consiste numa análise de dois modelos explicativos do comportamento humano considerados fundamentais na literatura contemporânea sobre teoria da ação. O primeiro modelo, o causalista, tenta explicar as ações em termos de causas e leis gerais. O segundo explica a ação em termos de intenções e silogismos práticos. As dificuldades e problemas de ambos modelos são apresentadas e na última parte do ensaio propomos um modelo alternativo baseado na noção de retrodução.

Image summarisation: human action description from static images

Dissertação de Mestrado, Processamento de Linguagem Natural e Indústrias da Língua, Faculdade de Ciências Humanas e Sociais, Universidade do Algarve, 2014

Adaptive unsupervised learning of human actions

Automatic detection of suspicious activities in CCTV camera feeds is crucial to the success of video surveillance systems. Such a capability can help transform the dumb CCTV cameras into smart surveillance tools for fighting crime and terror. Learning and classification of basic human actions is a precursor to detecting suspicious activities. Most of the current approaches rely on a non-realistic assumption that a complete dataset of normal human actions is available. This paper presents a different approach to deal with the problem of understanding human actions in video when no prior information is available. This is achieved by working with an incomplete dataset of basic actions which are continuously updated. Initially, all video segments are represented by Bags-Of-Words (BOW) method using only Term Frequency-Inverse Document Frequency (TF-IDF) features. Then, a data-stream clustering algorithm is applied for updating the system's knowledge from the incoming video feeds. Finally, all the actions are classified into different sets. Experiments and comparisons are conducted on the well known Weizmann and KTH datasets to show the efficacy of the proposed approach.

Multi-action recognition via stochastic modelling of optical flow and gradients

In this paper we propose a novel approach to multi-action recognition that performs joint segmentation and classification. This approach models each action using a Gaussian mixture using robust low-dimensional action features. Segmentation is achieved by performing classification on overlapping temporal windows, which are then merged to produce the final result. This approach is considerably less complicated than previous methods which use dynamic programming or computationally expensive hidden Markov models (HMMs). Initial experiments on a stitched version of the KTH dataset show that the proposed approach achieves an accuracy of 78.3%, outperforming a recent HMM-based approach which obtained 71.2%.

Multidimensionality of Actors in Business Networks: The Influence of Social Action in Pharmacy Networks in Finland

In the markets-as-networks approach business networks are conceived as dynamic actor structures, giving focus to exchange relationships and actors’ capabilities to control and co-ordinate activities and resources. Researchers have shared an understanding that actors’ actions are crucial for the development of business networks and for network dynamics. However, researchers have mainly studied firms as business actors and excluded individuals, although both firms and individuals can be seen as business actors. This focus on firms as business actors has resulted in a paucity of research on human action and the exchange of intangible resources in business networks, e.g. social exchange between individuals in social networks. Consequently, the current conception of business networks fails to appreciate the richness of business actors, the human character of business action and the import of social action in business networks. The central assumption in this study is that business actors are multidimensional and that their specific constitution in any given situation is determined by human interaction in social networks. Multidimensionality is presented as a concept for exploring how business actors act in different situations and how actors simultaneously manage multiple identities: individual, organisational, professional, business and network identities. The study presents a model that describes the multidimensionality of actors in business networks and conceptualises the connection between social exchange and human action in business networks. Empirically the study explores the change that has taken place in pharmaceutical retailing in Finland during recent years. The phenomenon of emerging pharmacy networks is highly contemporary in the Nordic countries, where the traditional license-based pharmacy business is changing. The study analyses the development of two Finnish pharmacy chains, one integrated and one voluntary chain, and the network structures and dynamics in them. Social Network Analysis is applied to explore the social structures within the pharmacy networks. The study shows that emerging pharmacy networks are multifaceted phenomena where political, economic, social, cultural, and historical elements together contribute to the observed changes. Individuals have always been strongly present in the pharmacy business and the development of pharmacy networks provides an interesting example of human actors’ influence in the development of business networks. The dynamics or forces driving the network development can be linked to actors’ own economic and social motives for developing the business. The study highlights the central role of individuals and social networks in the development of the two studied pharmacy networks. The relation between individuals and social networks is reciprocal. The social context of every individual enables multidimensional business actors. The mix of various identities, both individual and collective identities, is an important part of network dynamics. Social networks in pharmacy networks create a platform for exchange and social action, and social networks enable and support business network development.