Identifying multi-view patterns with hierarchy and granularity based multimodal (HGM) cognitive model

Humans perceive entities such as objects, patterns, events, etc. as concepts, which are the basic units in human intelligence and communications. In addition, perceptions of these entities could be abstracted and generalised at multiple levels of granularity. In particular, such granulation allows the formation and usage of concepts in human intelligence. Such natural granularity in human intelligence could inspire and motivate the design and development of pattern identification approach in Data Mining. In our opinion, a pattern could be perceived at multiple levels of granularity and thus we advocate for the co-existence of hierarchy and granularity. In addition, granular patterns exist across different sources of data (multimodality). In this paper, we present a cognitive model that incorporates the characteristics of Hierarchy, Granularity and Multimodality for multi-view patterns identification in crime domain. Such framework is implemented with Growing Self Organising Maps (GSOM) and some experimental results are presented and discussed.

Activity recognition and abnormality detection with the switching hidden semi-Markov model

This paper addresses the problem of learning and recognizing human activities of daily living (ADL), which is an important research issue in building a pervasive and smart environment. In dealing with ADL, we argue that it is beneficial to exploit both the inherent hierarchical organization of the activities and their typical duration. To this end, we introduce the Switching Hidden Semi-Markov Model (S-HSMM), a two-layered extension of the hidden semi-Markov model (HSMM) for the modeling task. Activities are modeled in the S-HSMM in two ways: the bottom layer represents atomic activities and their duration using HSMMs; the top layer represents a sequence of high-level activities where each high-level activity is made of a sequence of atomic activities. We consider two methods for modeling duration: the classic explicit duration model using multinomial distribution, and the novel use of the discrete Coxian distribution. In addition, we propose an effective scheme to detect abnormality without the need for training on abnormal data. Experimental results show that the S-HSMM performs better than existing models including the flat HSMM and the hierarchical hidden Markov model in both classification and abnormality detection tasks, alleviating the need for presegmented training data. Furthermore, our discrete Coxian duration model yields better computation time and generalization error than the classic explicit duration model.

Hierarchical hidden Markov models with general state hierarchy

The hierarchical hidden Markov model (HHMM) is an extension of the hidden Markov model to include a hierarchy of the hidden states. This form of hierarchical modeling has been found useful in applications such as handwritten character recognition, behavior recognition, video indexing, and text retrieval. Nevertheless, the state hierarchy in the original HHMM is restricted to a tree structure. This prohibits two different states from having the same child, and thus does not allow for sharing of common substructures in the model. In this paper, we present a general HHMM in which the state hierarchy can be a lattice allowing arbitrary sharing of substructures. Furthermore, we provide a method for numerical scaling to avoid underflow, an important issue in dealing with long observation sequences. We demonstrate the working of our method in a simulated environment where a hierarchical behavioral model is automatically learned and later used for recognition.

Hierarchical monitoring of people's behaviors in complex environments using multiple cameras

We present a distributed, surveillance system that works in large and complex indoor environments. To track and recognize behaviors of people, we propose the use of the Abstract Hidden Markov Model (AHMM), which can be considered as an extension of the Hidden Markov Model (HMM), where the single Markov chain in the HMM is replaced by a hierarchy of Markov policies. In this policy hierarchy, each behavior can be represented as a policy at the corresponding level of abstraction. The noisy observations are handled in the same way as an HMM and an efficient Rao-Blackwellised particle filter method is used to compute the probabilities of the current policy at different levels of the hierarchy The novelty of the paper lies in the implementation of a scalable framework in the context of both the scale of behaviors and the size of the environment, making it ideal for distributed surveillance. The results of the system demonstrate the ability to answer queries about people's behaviors at different levels of details using multiple cameras in a large and complex indoor environment.

Hierarchical recognition of intentional human gestures for sports video annotation

We present a novel technique for the recognition of complex human gestures for video annotation using accelerometers and the hidden Markov model. Our extension to the standard hidden Markov model allows us to consider gestures at different levels of abstraction through a hierarchy of hidden states. Accelerometers in the form of wrist bands are attached to humans performing intentional gestures, such as umpires in sports. Video annotation is then performed by populating the video with time stamps indicating significant events, where a particular gesture occurs. The novelty of the technique lies in the development of a probabilistic hierarchical framework for complex gesture recognition and the use of accelerometers to extract gestures and significant events for video annotation.

Content structure discovery in educational videos using shared structures in the hierarchical hidden Markov models

In this paper, we present an application of the hierarchical HMM for structure discovery in educational videos. The HHMM has recently been extended to accommodate the concept of shared structure, ie: a state might multiply inherit from more than one parents. Utilising the expressiveness of this model, we concentrate on a specific class of video -educational videos - in which the hierarchy of semantic units is simpler and clearly defined in terms of topics and its subunits. We model the hierarchy of topical structures by an HHMM and demonstrate the usefulness of the model in detecting topic transitions.

Hierarchical representation for spatial knowledge

One of the fundamental issues in building autonomous agents is to be able to sense, represent and react to the world. Some of the earlier work [Mor83, Elf90, AyF89] has aimed towards a reconstructionist approach, where a number of sensors are used to obtain input that is used to construct a model of the world that mirrors the real world. Sensing and sensor fusion was thus an important aspect of such work. Such approaches have had limited success, and some of the main problems were the issues of uncertainty arising from sensor error and errors that accumulated in metric, quantitative models. Recent research has therefore looked at different ways of examining the problems. Instead of attempting to get the most accurate and correct model of the world, these approaches look at qualitative models to represent the world, which maintain relative and significant aspects of the environment rather than all aspects of the world. The relevant aspects of the world that are retained are determined by the task at hand which in turn determines how to sense. That is, task directed or purposive sensing is used to build a qualitative model of the world, which though inaccurate and incomplete is sufficient to solve the problem at hand. This paper examines the issues of building up a hierarchical knowledge representation of the environment with limited sensor input that can be actively acquired by an agent capable of interacting with the environment. Different tasks require different aspects of the environment to be abstracted out. For example, low level tasks such as navigation require aspects of the environment that are related to layout and obstacle placement. For the agent to be able to reposition itself in an environment, significant features of spatial situations and their relative placement need to be kept. For the agent to reason about objects in space, for example to determine the position of one object relative to another, the representation needs to retain information on relative locations of start and finish of the objects, that is endpoints of objects on a grid. For the agent to be able to do high level planning, the agent may need only the relative position of the starting point and destination, and not the low level details of endpoints, visual clues and so on. This indicates that a hierarchical approach would be suitable, such that each level in the hierarchy is at a different level of abstraction, and thus suitable for a different task. At the lowest level, the representation contains low level details of agent's motion and visual clues to allow the agent to navigate and reposition itself. At the next level of abstraction the aspects of the representation allow the agent to perform spatial reasoning, and finally the highest level of abstraction in the representation can be used by the agent for high level planning.

Topic transition detection using hierarchical hidden Markov and semi-Markov models

In this paper we introduce a probabilistic framework to exploit hierarchy, structure sharing and duration information for topic transition detection in videos. Our probabilistic detection framework is a combination of a shot classification step and a detection phase using hierarchical probabilistic models. We consider two models in this paper: the extended Hierarchical Hidden Markov Model (HHMM) and the Coxian Switching Hidden semi-Markov Model (S-HSMM) because they allow the natural decomposition of semantics in videos, including shared structures, to be modeled directly, and thus enabling efficient inference and reducing the sample complexity in learning. Additionally, the S-HSMM allows the duration information to be incorporated, consequently the modeling of long-term dependencies in videos is enriched through both hierarchical and duration modeling. Furthermore, the use of the Coxian distribution in the S-HSMM makes it tractable to deal with long sequences in video. Our experimentation of the proposed framework on twelve educational and training videos shows that both models outperform the baseline cases (flat HMM and HSMM) and performances reported in earlier work in topic detection. The superior performance of the S-HSMM over the HHMM verifies our belief that duration information is an important factor in video content modeling.

A bayesian nonparametric joint factor model for learning shared and individual subspaces from multiple data sources

Joint analysis of multiple data sources is becoming increasingly popular in transfer learning, multi-task learning and cross-domain data mining. One promising approach to model the data jointly is through learning the shared and individual factor subspaces. However, performance of this approach depends on the subspace dimensionalities and the level of sharing needs to be specified a priori. To this end, we propose a nonparametric joint factor analysis framework for modeling multiple related data sources. Our model utilizes the hierarchical beta process as a nonparametric prior to automatically infer the number of shared and individual factors. For posterior inference, we provide a Gibbs sampling scheme using auxiliary variables. The effectiveness of the proposed framework is validated through its application on two real world problems - transfer learning in text and image retrieval.

A slice sampler for restricted hierarchical beta process with applications to shared subspace learning

Hierarchical beta process has found interesting applications in recent years. In this paper we present a modified hierarchical beta process prior with applications to hierarchical modeling of multiple data sources. The novel use of the prior over a hierarchical factor model allows factors to be shared across different sources. We derive a slice sampler for this model, enabling tractable inference even when the likelihood and the prior over parameters are non-conjugate. This allows the application of the model in much wider contexts without restrictions. We present two different data generative models – a linear Gaussian-Gaussian model for real valued data and a linear Poisson-gamma model for count data. Encouraging transfer learning results are shown for two real world applications – text modeling and content based image retrieval.

Enhancing an evolving tree-based text document visualization model with fuzzy c-Means clustering

An improved evolving model, i.e., Evolving Tree (ETree) with Fuzzy c-Means (FCM), is proposed for undertaking text document visualization problems in this study. ETree forms a hierarchical tree structure in which nodes (i.e., trunks) are allowed to grow and split into child nodes (i.e., leaves), and each node represents a cluster of documents. However, ETree adopts a relatively simple approach to split its nodes. Thus, FCM is adopted as an alternative to perform node splitting in ETree. An experimental study using articles from a flagship conference of Universiti Malaysia Sarawak (UNIMAS), i.e., Engineering Conference (ENCON), is conducted. The experimental results are analyzed and discussed, and the outcome shows that the proposed ETree-FCM model is effective for undertaking text document clustering and visualization problems.

Extraction of latent patterns and contexts from social honest signals using hierarchical Dirichlet processes

A fundamental task in pervasive computing is reliable acquisition of contexts from sensor data. This is crucial to the operation of smart pervasive systems and services so that they might behave efficiently and appropriately upon a given context. Simple forms of context can often be extracted directly from raw data. Equally important, or more, is the hidden context and pattern buried inside the data, which is more challenging to discover. Most of existing approaches borrow methods and techniques from machine learning, dominantly employ parametric unsupervised learning and clustering techniques. Being parametric, a severe drawback of these methods is the requirement to specify the number of latent patterns in advance. In this paper, we explore the use of Bayesian nonparametric methods, a recent data modelling framework in machine learning, to infer latent patterns from sensor data acquired in a pervasive setting. Under this formalism, nonparametric prior distributions are used for data generative process, and thus, they allow the number of latent patterns to be learned automatically and grow with the data - as more data comes in, the model complexity can grow to explain new and unseen patterns. In particular, we make use of the hierarchical Dirichlet processes (HDP) to infer atomic activities and interaction patterns from honest signals collected from sociometric badges. We show how data from these sensors can be represented and learned with HDP. We illustrate insights into atomic patterns learned by the model and use them to achieve high-performance clustering. We also demonstrate the framework on the popular Reality Mining dataset, illustrating the ability of the model to automatically infer typical social groups in this dataset. Finally, our framework is generic and applicable to a much wider range of problems in pervasive computing where one needs to infer high-level, latent patterns and contexts from sensor data.

A hierarchical cloud pricing system

Cloud computing is experiencing phenomenal growth and there are now many vendors offering their cloud services. In cloud computing, cloud providers cooperate together to offer their computing resource as a utility and software as a service to customers. The demands and the price of cloud service should be negotiated between providers and users based on the Service Level Agreement (SLA). In order to help cloud providers achieving an agreeable price for their services and maximizing the benefits of both cloud providers and clients, this paper proposes a cloud pricing system consisting of hierarchical system, M/M/c queuing model and pricing model. Simulation results verify the efficiency of our proposed system.

Topic model kernel : an empirical study towards probabilistically reduced features for classification

Probabilistic topic models have become a standard in modern machine learning with wide applications in organizing and summarizing ‘documents’ in high-dimensional data such as images, videos, texts, gene expression data, and so on. Representing data by dimensional reduction of mixture proportion extracted from topic models is not only richer in semantics than bag-of-word interpretation, but also more informative for classification tasks. This paper describes the Topic Model Kernel (TMK), a high dimensional mapping for Support Vector Machine classification of data generated from probabilistic topic models. The applicability of our proposed kernel is demonstrated in several classification tasks from real world datasets. We outperform existing kernels on the distributional features and give the comparative results on non-probabilistic data types.