New generation of human machine interfaces for controlling UAV through depth based gesture recognition

New forms of natural interactions between human operators and UAVs (Unmanned Aerial Vehicle) are demanded by the military industry to achieve a better balance of the UAV control and the burden of the human operator. In this work, a human machine interface (HMI) based on a novel gesture recognition system using depth imagery is proposed for the control of UAVs. Hand gesture recognition based on depth imagery is a promising approach for HMIs because it is more intuitive, natural, and non-intrusive than other alternatives using complex controllers. The proposed system is based on a Support Vector Machine (SVM) classifier that uses spatio-temporal depth descriptors as input features. The designed descriptor is based on a variation of the Local Binary Pattern (LBP) technique to efficiently work with depth video sequences. Other major consideration is the especial hand sign language used for the UAV control. A tradeoff between the use of natural hand signs and the minimization of the inter-sign interference has been established. Promising results have been achieved in a depth based database of hand gestures especially developed for the validation of the proposed system.

Development of Brain-Computer Interfaces using Evolvable Hardware

Brain-Computer Interfaces are usually tackled from a medical point of view, correlating observed phenomena to physical facts known about the brain. Existing methods of classification lie in the application of deterministic algorithms and depend on certain degree of knowledge about the underlying phenomena so as to process data. In this demo, different architectures for an evolvable hardware classifier implemented on an FPGA are proposed, in line with the objective of generalizing evolutionary algorithms regardless of the application.

Context modelling for natural Human Computer Interaction applications in e-health

The conception of IoT (Internet of Things) is accepted as the future tendency of Internet among academia and industry. It will enable people and things to be connected at anytime and anyplace, with anything and anyone. IoT has been proposed to be applied into many areas such as Healthcare, Transportation，Logistics, and Smart environment etc. However, this thesis emphasizes on the home healthcare area as it is the potential healthcare model to solve many problems such as the limited medical resources, the increasing demands for healthcare from elderly and chronic patients which the traditional model is not capable of. A remarkable change in IoT in semantic oriented vision is that vast sensors or devices are involved which could generate enormous data. Methods to manage the data including acquiring, interpreting, processing and storing data need to be implemented. Apart from this, other abilities that IoT is not capable of are concluded, namely, interoperation, context awareness and security & privacy. Context awareness is an emerging technology to manage and take advantage of context to enable any type of system to provide personalized services. The aim of this thesis is to explore ways to facilitate context awareness in IoT. In order to realize this objective, a preliminary research is carried out in this thesis. The most basic premise to realize context awareness is to collect, model, understand, reason and make use of context. A complete literature review for the existing context modelling and context reasoning techniques is conducted. The conclusion is that the ontology-based context modelling and ontology-based context reasoning are the most promising and efficient techniques to manage context. In order to fuse ontology into IoT, a specific ontology-based context awareness framework is proposed for IoT applications. In general, the framework is composed of eight components which are hardware, UI (User Interface), Context modelling, Context fusion, Context reasoning, Context repository, Security unit and Context dissemination. Moreover, on the basis of TOVE (Toronto Virtual Enterprise), a formal ontology developing methodology is proposed and illustrated which consists of four stages: Specification & Conceptualization, Competency Formulation, Implementation and Validation & Documentation. In addition, a home healthcare scenario is elaborated by listing its well-defined functionalities. Aiming at representing this specific scenario, the proposed ontology developing methodology is applied and the ontology-based model is developed in a free and open-source ontology editor called Protégé. Finally, the accuracy and completeness of the proposed ontology are validated to show that this proposed ontology is able to accurately represent the scenario of interest.

Human-computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns

A more natural, intuitive, user-friendly, and less intrusive Human–Computer interface for controlling an application by executing hand gestures is presented. For this purpose, a robust vision-based hand-gesture recognition system has been developed, and a new database has been created to test it. The system is divided into three stages: detection, tracking, and recognition. The detection stage searches in every frame of a video sequence potential hand poses using a binary Support Vector Machine classifier and Local Binary Patterns as feature vectors. These detections are employed as input of a tracker to generate a spatio-temporal trajectory of hand poses. Finally, the recognition stage segments a spatio-temporal volume of data using the obtained trajectories, and compute a video descriptor called Volumetric Spatiograms of Local Binary Patterns (VS-LBP), which is delivered to a bank of SVM classifiers to perform the gesture recognition. The VS-LBP is a novel video descriptor that constitutes one of the most important contributions of the paper, which is able to provide much richer spatio-temporal information than other existing approaches in the state of the art with a manageable computational cost. Excellent results have been obtained outperforming other approaches of the state of the art.

Development of a hand-gesture recognition system for human-computer interaction

The aim of this Master Thesis is the analysis, design and development of a robust and reliable Human-Computer Interaction interface, based on visual hand-gesture recognition. The implementation of the required functions is oriented to the simulation of a classical hardware interaction device: the mouse, by recognizing a specific hand-gesture vocabulary in color video sequences. For this purpose, a prototype of a hand-gesture recognition system has been designed and implemented, which is composed of three stages: detection, tracking and recognition. This system is based on machine learning methods and pattern recognition techniques, which have been integrated together with other image processing approaches to get a high recognition accuracy and a low computational cost. Regarding pattern recongition techniques, several algorithms and strategies have been designed and implemented, which are applicable to color images and video sequences. The design of these algorithms has the purpose of extracting spatial and spatio-temporal features from static and dynamic hand gestures, in order to identify them in a robust and reliable way. Finally, a visual database containing the necessary vocabulary of gestures for interacting with the computer has been created.

Human-computer interaction for optical music recognition tasks

The need to digitise music scores has led to the development of Optical Music Recognition (OMR) tools. Unfortunately, the performance of these systems is still far from providing acceptable results. This situation forces the user to be involved in the process due to the need of correcting the mistakes made during recognition. However, this correction is performed over the output of the system, so these interventions are not exploited to improve the performance of the recognition. This work sets the scenario in which human and machine interact to accurately complete the OMR task with the least possible effort for the user.

Non-invasive Detection of Cortical Control Signals for Brain-Computer Interfaces

Thesis (Ph.D.)--University of Washington, 2016-05

Closed-Loop Deep Brain Stimulation: Bidirectional Neuroprosthetics for Tremor and BCI

Thesis (Ph.D.)--University of Washington, 2016-06

Formal analysis of human-computer interaction using model checking

Experiments with simulators allow psychologists to better understand the causes of human errors and build models of cognitive processes to be used in human reliability assessment (HRA). This paper investigates an approach to task failure analysis based on patterns of behaviour, by contrast to more traditional event-based approaches. It considers, as a case study, a formal model of an air traffic control (ATC) system which incorporates controller behaviour. The cognitive model is formalised in the CSP process algebra. Patterns of behaviour are expressed as temporal logic properties. Then a model-checking technique is used to verify whether the decomposition of the operator's behaviour into patterns is sound and complete with respect to the cognitive model. The decomposition is shown to be incomplete and a new behavioural pattern is identified, which appears to have been overlooked in the analysis of the data provided by the experiments with the simulator. This illustrates how formal analysis of operator models can yield fresh insights into how failures may arise in interactive systems.

Comparing interaction techniques for serious games through brain-computer interfaces:a user perception evaluation study

This paper examines the application of commercial and non-invasive electroencephalography (EEG)-based brain-computer (BCIs) interfaces with serious games. Two different EEG-based BCI devices were used to fully control the same serious game. The first device (NeuroSky MindSet) uses only a single dry electrode and requires no calibration. The second device (Emotiv EPOC) uses 14 wet sensors requiring additional training of a classifier. User testing was performed on both devices with sixty-two participants measuring the player experience as well as key aspects of serious games, primarily learnability, satisfaction, performance and effort. Recorded feedback indicates that the current state of BCIs can be used in the future as alternative game interfaces after familiarisation and in some cases calibration. Comparative analysis showed significant differences between the two devices. The first device provides more satisfaction to the players whereas the second device is more effective in terms of adaptation and interaction with the serious game.

Munkahelyi tanulást befolyásoló tényezők – humán-számítógép együttműködés vizsgálata (Influencing factors of on-the-job learning – Analysis of human-computer collaboration)

A komplex, dinamikus, tudásalapú társadalomban nemcsak a tanulás formái, hanem a tanulás helyszínei is módosulnak, s a munkahely tanulásban betöltött szerepe felértékelődött. A munkahelyi környezet is számos átalakuláson esett át, az információs és kommunikációs technológiák (IKT) fejlődésével egyidejűleg lehetővé vált többek között a távmunka, jelentősen átformálva a munkavégzés és a munkahelyi interakciók módját. A kutatók arra keresték a választ kutatásukban, hogy a szervezeten belül milyen tényezők támogatják vagy gátolják a munkahelyi tanulást. A kutatás fő üzenete, hogy a tanulás keretrendszere, az egyéni képességek és az észlelt tanulási szituáció együttesen határozza meg a munkahelyi tanulást. A kutatók eredményüket kvalitatív kutatással feltárt három esettanulmányon keresztül ismertetik. ____ In a knowledge-based society not only the forms of learning have been changed but also the places of learning. The role of workplace in the learning process is becoming more important. Meantime, there has been a substantial change in the working conditions as the development in information and communication technologies (ICTs) makes it possible to telecommute transforming remarkably the way of working and the interactions at the workplace. The central question of the research is which intra-organizational factors support or hinder onthe- job learning. The main message of the research is that the learning framework, the individual cognitive competences and the perceived learning situation influence collectively on-the-job learning. Authors present the results of the qualitative research though three case studies.

CASIMIRO: a robot head for human-computer interaction

The physical appearance and behavior of a robot is an important asset in terms of Human-Computer Interaction. Multimodality is also fundamental, as we humans usually expect to interact in a natural way with voice, gestures, etc. People approach complex interaction devices with stances similar to those used in their interaction with other people. In this paper we describe a robot head, currently under development, that aims to be a multimodal (vision, voice, gestures,...) perceptual user interface.

INNOVATIVE HUMAN-COMPUTER INTERACTIONS TO SUPPORT COGNITIVE COLLABORATIVE GEOSPATIAL ENVIRONMENTS

In the current world geospatial information is being demanded in almost real time, which requires the speed at which this data is processed and made available to the user to be at an all-time high. In order to keep up with this ever increasing speed, analysts must find ways to increase their productivity. At the same time the demand for new analysts is high, and current methods of training are long and can be costly. Through the use of human computer interactions and basic networking systems, this paper explores new ways to increase efficiency in data processing and analyst training.

Overlapped Sequence Types (sts) And Serogroups Of Avian Pathogenic (apec) And Human Extra-intestinal Pathogenic (expec) Escherichia Coli Isolated In Brazil.

Avian pathogenic Escherichia coli (APEC) strains belong to a category that is associated with colibacillosis, a serious illness in the poultry industry worldwide. Additionally, some APEC groups have recently been described as potential zoonotic agents. In this work, we compared APEC strains with extraintestinal pathogenic E. coli (ExPEC) strains isolated from clinical cases of humans with extra-intestinal diseases such as urinary tract infections (UTI) and bacteremia. PCR results showed that genes usually found in the ColV plasmid (tsh, iucA, iss, and hlyF) were associated with APEC strains while fyuA, irp-2, fepC sitDchrom, fimH, crl, csgA, afa, iha, sat, hlyA, hra, cnf1, kpsMTII, clpVSakai and malX were associated with human ExPEC. Both categories shared nine serogroups (O2, O6, O7, O8, O11, O19, O25, O73 and O153) and seven sequence types (ST10, ST88, ST93, ST117, ST131, ST155, ST359, ST648 and ST1011). Interestingly, ST95, which is associated with the zoonotic potential of APEC and is spread in avian E. coli of North America and Europe, was not detected among 76 APEC strains. When the strains were clustered based on the presence of virulence genes, most ExPEC strains (71.7%) were contained in one cluster while most APEC strains (63.2%) segregated to another. In general, the strains showed distinct genetic and fingerprint patterns, but avian and human strains of ST359, or ST23 clonal complex (CC), presented more than 70% of similarity by PFGE. The results demonstrate that some zoonotic-related STs (ST117, ST131, ST10CC, ST23CC) are present in Brazil. Also, the presence of moderate fingerprint similarities between ST359 E. coli of avian and human origin indicates that strains of this ST are candidates for having zoonotic potential.