Class-specific sparse codes for representing activities

In this paper we investigate the effectiveness of class specific sparse codes in the context of discriminative action classification. The bag-of-words representation is widely used in activity recognition to encode features, and although it yields state-of-the art performance with several feature descriptors it still suffers from large quantization errors and reduces the overall performance. Recently proposed sparse representation methods have been shown to effectively represent features as a linear combination of an over complete dictionary by minimizing the reconstruction error. In contrast to most of the sparse representation methods which focus on Sparse-Reconstruction based Classification (SRC), this paper focuses on a discriminative classification using a SVM by constructing class-specific sparse codes for motion and appearance separately. Experimental results demonstrates that separate motion and appearance specific sparse coefficients provide the most effective and discriminative representation for each class compared to a single class-specific sparse coefficients.

Synchronization of user-generated videos through trajectory correspondence and a refinement procedure

Temporal synchronization of multiple video recordings of the same dynamic event is a critical task in many computer vision applications e.g. novel view synthesis and 3D reconstruction. Typically this information is implied through the time-stamp information embedded in the video streams. User-generated videos shot using consumer grade equipment do not contain this information; hence, there is a need to temporally synchronize signals using the visual information itself. Previous work in this area has either assumed good quality data with relatively simple dynamic content or the availability of precise camera geometry. Our first contribution is a synchronization technique which tries to establish correspondence between feature trajectories across views in a novel way, and specifically targets the kind of complex content found in consumer generated sports recordings, without assuming precise knowledge of fundamental matrices or homographies. We evaluate performance using a number of real video recordings and show that our method is able to synchronize to within 1 sec, which is significantly better than previous approaches. Our second contribution is a robust and unsupervised view-invariant activity recognition descriptor that exploits recurrence plot theory on spatial tiles. The descriptor is individually shown to better characterize the activities from different views under occlusions than state-of-the-art approaches. We combine this descriptor with our proposed synchronization method and show that it can further refine the synchronization index. © 2013 ACM.

Context-aware real-time assistant architecture for pervasive healthcare

The aging population in many countries brings into focus rising healthcare costs and pressure on conventional healthcare services. Pervasive healthcare has emerged as a viable solution capable of providing a technology-driven approach to alleviate such problems by allowing healthcare to move from the hospital-centred care to self-care, mobile care, and at-home care. The state-of-the-art studies in this field, however, lack a systematic approach for providing comprehensive pervasive healthcare solutions from data collection to data interpretation and from data analysis to data delivery. In this thesis we introduce a Context-aware Real-time Assistant (CARA) architecture that integrates novel approaches with state-of-the-art technology solutions to provide a full-scale pervasive healthcare solution with the emphasis on context awareness to help maintaining the well-being of elderly people. CARA collects information about and around the individual in a home environment, and enables accurately recognition and continuously monitoring activities of daily living. It employs an innovative reasoning engine to provide accurate real-time interpretation of the context and current situation assessment. Being mindful of the use of the system for sensitive personal applications, CARA includes several mechanisms to make the sophisticated intelligent components as transparent and accountable as possible, it also includes a novel cloud-based component for more effective data analysis. To deliver the automated real-time services, CARA supports interactive video and medical sensor based remote consultation. Our proposal has been validated in three application domains that are rich in pervasive contexts and real-time scenarios: (i) Mobile-based Activity Recognition, (ii) Intelligent Healthcare Decision Support Systems and (iii) Home-based Remote Monitoring Systems.

Towards Real-time Speech Emotion Recognition for Affective E-Learning

The original article is available as an open access file on the Springer website in the following link: http://link.springer.com/article/10.1007/s10639-015-9388-2

A sequential algorithm for sparse support vector classifiers

Support vector machines (SVMs), though accurate, are not preferred in applications requiring high classification speed or when deployed in systems of limited computational resources, due to the large number of support vectors involved in the model. To overcome this problem we have devised a primal SVM method with the following properties: (1) it solves for the SVM representation without the need to invoke the representer theorem, (2) forward and backward selections are combined to approach the final globally optimal solution, and (3) a criterion is introduced for identification of support vectors leading to a much reduced support vector set. In addition to introducing this method the paper analyzes the complexity of the algorithm and presents test results on three public benchmark problems and a human activity recognition application. These applications demonstrate the effectiveness and efficiency of the proposed algorithm.


--------------------------------------------------------------------------------

Deep learning architecture for gesture recognition

Pós-graduação em Engenharia Mecânica - FEG

Evaluation of Three State-of-the-Art Classifiers for Recognition of Activities of Daily Living from Smart Home Ambient Data

Smart homes for the aging population have recently started attracting the attention of the research community. The "health state" of smart homes is comprised of many different levels; starting with the physical health of citizens, it also includes longer-term health norms and outcomes, as well as the arena of positive behavior changes. One of the problems of interest is to monitor the activities of daily living (ADL) of the elderly, aiming at their protection and well-being. For this purpose, we installed passive infrared (PIR) sensors to detect motion in a specific area inside a smart apartment and used them to collect a set of ADL. In a novel approach, we describe a technology that allows the ground truth collected in one smart home to train activity recognition systems for other smart homes. We asked the users to label all instances of all ADL only once and subsequently applied data mining techniques to cluster in-home sensor firings. Each cluster would therefore represent the instances of the same activity. Once the clusters were associated to their corresponding activities, our system was able to recognize future activities. To improve the activity recognition accuracy, our system preprocessed raw sensor data by identifying overlapping activities. To evaluate the recognition performance from a 200-day dataset, we implemented three different active learning classification algorithms and compared their performance: naive Bayesian (NB), support vector machine (SVM) and random forest (RF). Based on our results, the RF classifier recognized activities with an average specificity of 96.53%, a sensitivity of 68.49%, a precision of 74.41% and an F-measure of 71.33%, outperforming both the NB and SVM classifiers. Further clustering markedly improved the results of the RF classifier. An activity recognition system based on PIR sensors in conjunction with a clustering classification approach was able to detect ADL from datasets collected from different homes. Thus, our PIR-based smart home technology could improve care and provide valuable information to better understand the functioning of our societies, as well as to inform both individual and collective action in a smart city scenario.

Noise-tolerance feasibility for restricted-domain Information Retrieval systems

Information Retrieval systems normally have to work with rather heterogeneous sources, such as Web sites or documents from Optical Character Recognition tools. The correct conversion of these sources into flat text files is not a trivial task since noise may easily be introduced as a result of spelling or typeset errors. Interestingly, this is not a great drawback when the size of the corpus is sufficiently large, since redundancy helps to overcome noise problems. However, noise becomes a serious problem in restricted-domain Information Retrieval specially when the corpus is small and has little or no redundancy. This paper devises an approach which adds noise-tolerance to Information Retrieval systems. A set of experiments carried out in the agricultural domain proves the effectiveness of the approach presented.

Human behaviour recognition in data-scarce domains

This paper presents the novel theory for performing multi-agent activity recognition without requiring large training corpora. The reduced need for data means that robust probabilistic recognition can be performed within domains where annotated datasets are traditionally unavailable. Complex human activities are composed from sequences of underlying primitive activities. We do not assume that the exact temporal ordering of primitives is necessary, so can represent complex activity using an unordered bag. Our three-tier architecture comprises low-level video tracking, event analysis and high-level inference. High-level inference is performed using a new, cascading extension of the Rao–Blackwellised Particle Filter. Simulated annealing is used to identify pairs of agents involved in multi-agent activity. We validate our framework using the benchmarked PETS 2006 video surveillance dataset and our own sequences, and achieve a mean recognition F-Score of 0.82. Our approach achieves a mean improvement of 17% over a Hidden Markov Model baseline.

Modelling human behaviour and event detection from videos

Automatic analysis of human behaviour in large collections of videos is gaining interest, even more so with the advent of file sharing sites such as YouTube. However, challenges still exist owing to several factors such as inter- and intra-class variations, cluttered backgrounds, occlusion, camera motion, scale, view and illumination changes. This research focuses on modelling human behaviour for action recognition in videos. The developed techniques are validated on large scale benchmark datasets and applied on real-world scenarios such as soccer videos. Three major contributions are made. The first contribution is in the area of proper choice of a feature representation for videos. This involved a study of state-of-the-art techniques for action recognition, feature extraction processing and dimensional reduction techniques so as to yield the best performance with optimal computational requirements. Secondly, temporal modelling of human behaviour is performed. This involved frequency analysis and temporal integration of local information in the video frames to yield a temporal feature vector. Current practices mostly average the frame information over an entire video and neglect the temporal order. Lastly, the proposed framework is applied and further adapted to real-world scenario such as soccer videos. A dataset consisting of video sequences depicting events of players falling is created from actual match data to this end and used to experimentally evaluate the proposed framework.

Football Video Annotation Based on Player Motion Recognition Using Enhanced Entropy

This paper presents a semi-parametric Algorithm for parsing football video structures. The approach works on a two interleaved based process that closely collaborate towards a common goal. The core part of the proposed method focus perform a fast automatic football video annotation by looking at the enhance entropy variance within a series of shot frames. The entropy is extracted on the Hue parameter from the HSV color system, not as a global feature but in spatial domain to identify regions within a shot that will characterize a certain activity within the shot period. The second part of the algorithm works towards the identification of dominant color regions that could represent players and playfield for further activity recognition. Experimental Results shows that the proposed football video segmentation algorithm performs with high accuracy.

Contribution towards intelligent service management in wearable and ubiquitous devices

Hoy en día asistimos a un creciente interés por parte de la sociedad hacia el cuidado de la salud. Esta afirmación viene apoyada por dos realidades. Por una parte, el aumento de las prácticas saludables (actividad deportiva, cuidado de la alimentación, etc.). De igual manera, el auge de los dispositivos inteligentes (relojes, móviles o pulseras) capaces de medir distintos parámetros físicos como el pulso cardíaco, el ritmo respiratorio, la distancia recorrida, las calorías consumidas, etc. Combinando ambos factores (interés por el estado de salud y disponibilidad comercial de dispositivos inteligentes) están surgiendo multitud de aplicaciones capaces no solo de controlar el estado actual de salud, también de recomendar al usuario cambios de hábitos que lleven hacia una mejora en su condición física. En este contexto, los llamados dispositivos llevables (weareables) unidos al paradigma de Internet de las cosas (IoT, del inglés Internet of Things) permiten la aparición de nuevos nichos de mercado para aplicaciones que no solo se centran en la mejora de la condición física, ya que van más allá proponiendo soluciones para el cuidado de pacientes enfermos, la vigilancia de niños o ancianos, la defensa y la seguridad, la monitorización de agentes de riesgo (como bomberos o policías) y un largo etcétera de aplicaciones por llegar. El paradigma de IoT se puede desarrollar basándose en las existentes redes de sensores inalámbricos (WSN, del inglés Wireless Sensor Network). La conexión de los ya mencionados dispositivos llevables a estas redes puede facilitar la transición de nuevos usuarios hacia aplicaciones IoT. Pero uno de los problemas intrínsecos a estas redes es su heterogeneidad. En efecto, existen multitud de sistemas operativos, protocolos de comunicación, plataformas de desarrollo, soluciones propietarias, etc. El principal objetivo de esta tesis es realizar aportaciones significativas para solucionar no solo el problema de la heterogeneidad, sino también de dotar de mecanismos de seguridad suficientes para salvaguardad la integridad de los datos intercambiados en este tipo de aplicaciones. Algo de suma importancia ya que los datos médicos y biométricos de los usuarios están protegidos por leyes nacionales y comunitarias. Para lograr dichos objetivos, se comenzó con la realización de un completo estudio del estado del arte en tecnologías relacionadas con el marco de investigación (plataformas y estándares para WSNs e IoT, plataformas de implementación distribuidas, dispositivos llevables y sistemas operativos y lenguajes de programación). Este estudio sirvió para tomar decisiones de diseño fundamentadas en las tres contribuciones principales de esta tesis: un bus de servicios para dispositivos llevables (WDSB, Wearable Device Service Bus) basado en tecnologías ya existentes tales como ESB, WWBAN, WSN e IoT); un protocolo de comunicaciones inter-dominio para dispositivos llevables (WIDP, Wearable Inter-Domain communication Protocol) que integra en una misma solución protocolos capaces de ser implementados en dispositivos de bajas capacidades (como lo son los dispositivos llevables y los que forman parte de WSNs); y finalmente, la tercera contribución relevante es una propuesta de seguridad para WSN basada en la aplicación de dominios de confianza. Aunque las contribuciones aquí recogidas son de aplicación genérica, para su validación se utilizó un escenario concreto de aplicación: una solución para control de parámetros físicos en entornos deportivos, desarrollada dentro del proyecto europeo de investigación “LifeWear”. En este escenario se desplegaron todos los elementos necesarios para validar las contribuciones principales de esta tesis y, además, se realizó una aplicación para dispositivos móviles por parte de uno de los socios del proyecto (lo que contribuyó con una validación externa de la solución). En este escenario se usaron dispositivos llevables tales como un reloj inteligente, un teléfono móvil con sistema operativo Android y un medidor del ritmo cardíaco inalámbrico capaz de obtener distintos parámetros fisiológicos del deportista. Sobre este escenario se realizaron diversas pruebas de validación mediante las cuales se obtuvieron resultados satisfactorios. ABSTRACT Nowadays, society is shifting towards a growing interest and concern on health care. This phenomenon can be acknowledged by two facts: first, the increasing number of people practising some kind of healthy activity (sports, balanced diet, etc.). Secondly, the growing number of commercial wearable smart devices (smartwatches or bands) able to measure physiological parameters such as heart rate, breathing rate, distance or consumed calories. A large number of applications combining both facts are appearing. These applications are not only able to monitor the health status of the user, but also to provide recommendations about routines in order to improve the mentioned health status. In this context, wearable devices merged with the Internet of Things (IoT) paradigm enable the proliferation of new market segments for these health wearablebased applications. Furthermore, these applications can provide solutions for the elderly or baby care, in-hospital or in-home patient monitoring, security and defence fields or an unforeseen number of future applications. The introduced IoT paradigm can be developed with the usage of existing Wireless Sensor Networks (WSNs) by connecting the novel wearable devices to them. In this way, the migration of new users and actors to the IoT environment will be eased. However, a major issue appears in this environment: heterogeneity. In fact, there is a large number of operating systems, hardware platforms, communication and application protocols or programming languages, each of them with unique features. The main objective of this thesis is defining and implementing a solution for the intelligent service management in wearable and ubiquitous devices so as to solve the heterogeneity issues that are presented when dealing with interoperability and interconnectivity of devices and software of different nature. Additionally, a security schema based on trust domains is proposed as a solution to the privacy problems arising when private data (e.g., biomedical parameters or user identification) is broadcasted in a wireless network. The proposal has been made after a comprehensive state-of-the-art analysis, and includes the design of a Wearable Device Service Bus (WDSB) including the technologies collected in the requirement analysis (ESB, WWBAN, WSN and IoT). Applications are able to access the WSN services regardless of the platform and operating system where they are running. Besides, this proposal also includes the design of a Wearable Inter-Domain communication Protocols set (WIDP) which integrates lightweight protocols suitable to be used in low-capacities devices (REST, JSON, AMQP, CoAP, etc...). Furthermore, a security solution for service management based on a trustworthy domains model to deploy security services in WSNs has been designed. Although the proposal is a generic framework for applications based on services provided by wearable devices, an application scenario for testing purposes has been included. In this validation scenario it has been presented an autonomous physical condition performance system, based on a WSN, bringing the possibility to include several elements in an IoT scenario: a smartwatch, a physiological monitoring device and a smartphone. In summary, the general objective of this thesis is solving the heterogeneity and security challenges arising when developing applications for WSNs and wearable devices. As it has been presented in the thesis, the solution proposed has been successfully validated in a real scenario and the obtained results were satisfactory.