Human-computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns

A more natural, intuitive, user-friendly, and less intrusive Human–Computer interface for controlling an application by executing hand gestures is presented. For this purpose, a robust vision-based hand-gesture recognition system has been developed, and a new database has been created to test it. The system is divided into three stages: detection, tracking, and recognition. The detection stage searches in every frame of a video sequence potential hand poses using a binary Support Vector Machine classifier and Local Binary Patterns as feature vectors. These detections are employed as input of a tracker to generate a spatio-temporal trajectory of hand poses. Finally, the recognition stage segments a spatio-temporal volume of data using the obtained trajectories, and compute a video descriptor called Volumetric Spatiograms of Local Binary Patterns (VS-LBP), which is delivered to a bank of SVM classifiers to perform the gesture recognition. The VS-LBP is a novel video descriptor that constitutes one of the most important contributions of the paper, which is able to provide much richer spatio-temporal information than other existing approaches in the state of the art with a manageable computational cost. Excellent results have been obtained outperforming other approaches of the state of the art.

Development of a hand-gesture recognition system for human-computer interaction

The aim of this Master Thesis is the analysis, design and development of a robust and reliable Human-Computer Interaction interface, based on visual hand-gesture recognition. The implementation of the required functions is oriented to the simulation of a classical hardware interaction device: the mouse, by recognizing a specific hand-gesture vocabulary in color video sequences. For this purpose, a prototype of a hand-gesture recognition system has been designed and implemented, which is composed of three stages: detection, tracking and recognition. This system is based on machine learning methods and pattern recognition techniques, which have been integrated together with other image processing approaches to get a high recognition accuracy and a low computational cost. Regarding pattern recongition techniques, several algorithms and strategies have been designed and implemented, which are applicable to color images and video sequences. The design of these algorithms has the purpose of extracting spatial and spatio-temporal features from static and dynamic hand gestures, in order to identify them in a robust and reliable way. Finally, a visual database containing the necessary vocabulary of gestures for interacting with the computer has been created.

Human-computer interaction for optical music recognition tasks

The need to digitise music scores has led to the development of Optical Music Recognition (OMR) tools. Unfortunately, the performance of these systems is still far from providing acceptable results. This situation forces the user to be involved in the process due to the need of correcting the mistakes made during recognition. However, this correction is performed over the output of the system, so these interventions are not exploited to improve the performance of the recognition. This work sets the scenario in which human and machine interact to accurately complete the OMR task with the least possible effort for the user.

Quantifying epistemic actions in human-computer interaction

As digital systems move away from traditional desktop setups, new interaction paradigms are emerging that better integrate with users’ realworld surroundings, and better support users’ individual needs. While promising, these modern interaction paradigms also present new challenges, such as a lack of paradigm-specific tools to systematically evaluate and fully understand their use. This dissertation tackles this issue by framing empirical studies of three novel digital systems in embodied cognition – an exciting new perspective in cognitive science where the body and its interactions with the physical world take a central role in human cognition. This is achieved by first, focusing the design of all these systems on a contemporary interaction paradigm that emphasizes physical interaction on tangible interaction, a contemporary interaction paradigm; and second, by comprehensively studying user performance in these systems through a set of novel performance metrics grounded on epistemic actions, a relatively well established and studied construct in the literature on embodied cognition. The first system presented in this dissertation is an augmented Four-in-a-row board game. Three different versions of the game were developed, based on three different interaction paradigms (tangible, touch and mouse), and a repeated measures study involving 36 participants measured the occurrence of three simple epistemic actions across these three interfaces. The results highlight the relevance of epistemic actions in such a task and suggest that the different interaction paradigms afford instantiation of these actions in different ways. Additionally, the tangible version of the system supports the most rapid execution of these actions, providing novel quantitative insights into the real benefits of tangible systems. The second system presented in this dissertation is a tangible tabletop scheduling application. Two studies with single and paired users provide several insights into the impact of epistemic actions on the user experience when these are performed outside of a system’s sensing boundaries. These insights are clustered by the form, size and location of ideal interface areas for such offline epistemic actions to occur, as well as how can physical tokens be designed to better support them. Finally, and based on the results obtained to this point, the last study presented in this dissertation directly addresses the lack of empirical tools to formally evaluate tangible interaction. It presents a video-coding framework grounded on a systematic literature review of 78 papers, and evaluates its value as metric through a 60 participant study performed across three different research laboratories. The results highlight the usefulness and power of epistemic actions as a performance metric for tangible systems. In sum, through the use of such novel metrics in each of the three studies presented, this dissertation provides a better understanding of the real impact and benefits of designing and developing systems that feature tangible interaction.

Formal analysis of human-computer interaction using model checking

Experiments with simulators allow psychologists to better understand the causes of human errors and build models of cognitive processes to be used in human reliability assessment (HRA). This paper investigates an approach to task failure analysis based on patterns of behaviour, by contrast to more traditional event-based approaches. It considers, as a case study, a formal model of an air traffic control (ATC) system which incorporates controller behaviour. The cognitive model is formalised in the CSP process algebra. Patterns of behaviour are expressed as temporal logic properties. Then a model-checking technique is used to verify whether the decomposition of the operator's behaviour into patterns is sound and complete with respect to the cognitive model. The decomposition is shown to be incomplete and a new behavioural pattern is identified, which appears to have been overlooked in the analysis of the data provided by the experiments with the simulator. This illustrates how formal analysis of operator models can yield fresh insights into how failures may arise in interactive systems.

Individual differences in human-computer interaction

This thesis initially presents an 'assay' of the literature pertaining to individual differences in human-computer interaction. A series of experiments is then reported, designed to investigate the association between a variety of individual characteristics and various computer task and interface factors. Predictor variables included age, computer expertise, and psychometric tests of spatial visualisation, spatial memory, logical reasoning, associative memory, and verbal ability. These were studied in relation to a variety of computer-based tacks, including: (1) word processing and its component elements; (ii) the location of target words within passages of text; (iii) the navigation of networks and menus; (iv) command generation using menus and command line interfaces; (v) the search and selection of icons and text labels; (vi) information retrieval. A measure of self-report workload was also included in several of these experiments. The main experimental findings included: (i) an interaction between spatial ability and the manipulation of semantic but not spatial interface content; (ii) verbal ability being only predictive of certain task components of word processing; (iii) age differences in word processing and information retrieval speed but not accuracy; (iv) evidence of compensatory strategies being employed by older subjects; (v) evidence of performance strategy differences which disadvantaged high spatial subjects in conditions of low spatial information content; (vi) interactive effects of associative memory, expertise and command strategy; (vii) an association between logical reasoning and word processing but not information retrieval; (viii) an interaction between expertise and cognitive demand; and (ix) a stronger association between cognitive ability and novice performance than expert performance.

CASIMIRO: a robot head for human-computer interaction

The physical appearance and behavior of a robot is an important asset in terms of Human-Computer Interaction. Multimodality is also fundamental, as we humans usually expect to interact in a natural way with voice, gestures, etc. People approach complex interaction devices with stances similar to those used in their interaction with other people. In this paper we describe a robot head, currently under development, that aims to be a multimodal (vision, voice, gestures,...) perceptual user interface.

Using Research Methods in Human Computer Interaction to Design Technology for Resilience

Research in human computer interaction (HCI) covers both technological and human behavioural concerns. As a consequence, the contributions made in HCI research tend to be aware to either engineering or the social sciences. In HCI the purpose of practical research contributions is to reveal unknown insights about human behaviour and its relationship to technology. Practical research methods normally used in HCI include formal experiments, field experiments, field studies, interviews, focus groups, surveys, usability tests, case studies, diary studies, ethnography, contextual inquiry, experience sampling, and automated data collection. In this paper, we report on our experience using the evaluation methods focus groups, surveys and interviews and how we adopted these methods to develop artefacts: either interface’s design or information and technological systems. Four projects are examples of the different methods application to gather information about user’s wants, habits, practices, concerns and preferences. The goal was to build an understanding of the attitudes and satisfaction of the people who might interact with a technological artefact or information system. Conversely, we intended to design for information systems and technological applications, to promote resilience in organisations (a set of routines that allow to recover from obstacles) and user’s experiences. Organisations can here also be viewed within a system approach, which means that the system perturbations even failures could be characterized and improved. The term resilience has been applied to everything from the real estate, to the economy, sports, events, business, psychology, and more. In this study, we highlight that resilience is also made up of a number of different skills and abilities (self-awareness, creating meaning from other experiences, self-efficacy, optimism, and building strong relationships) that are a few foundational ingredients, which people should use along with the process of enhancing an organisation’s resilience. Resilience enhances knowledge of resources available to people confronting existing problems.

Human-Computer interaction methodologies applied in the evaluation of haptic digital musical instruments

Recent developments in interactive technologies have seen major changes in the manner in which artists, performers, and creative individuals interact with digital music technology; this is due to the increasing variety of interactive technologies that are readily available today. Digital Musical Instruments (DMIs) present musicians with performance challenges that are unique to this form of computer music. One of the most significant deviations from conventional acoustic musical instruments is the level of physical feedback conveyed by the instrument to the user. Currently, new interfaces for musical expression are not designed to be as physically communicative as acoustic instruments. Specifically, DMIs are often void of haptic feedback and therefore lack the ability to impart important performance information to the user. Moreover, there currently is no standardised way to measure the effect of this lack of physical feedback. Best practice would expect that there should be a set of methods to effectively, repeatedly, and quantifiably evaluate the functionality, usability, and user experience of DMIs. Earlier theoretical and technological applications of haptics have tried to address device performance issues associated with the lack of feedback in DMI designs and it has been argued that the level of haptic feedback presented to a user can significantly affect the user’s overall emotive feeling towards a musical device. The outcome of the investigations contained within this thesis are intended to inform new haptic interface.

Analysis of interaction of cloned human and/or sheep fetal hemoglobin gamma-chain and LPS in augmenting induction of inflammatory cytokine production in vivo and in vitro.

We have reported earlier that purified preparations of sheep fetal hemoglobin, but not adult hemoglobin, in concert with non-stimulatory doses of lipopolysaccharide (LPS) (lipid A), act cooperatively to regulate in vitro production of a number of cytokines, including TNFalpha, TGFbeta and IL-6 from murine and human leukocytes. Following in vivo treatment of mice with the same combination of hemoglobin and LPS, harvested spleen or peritoneal cells showed a similar augmented capacity to release these cytokines into culture supernatants. We report below that genetically cloned gamma-chain of human or sheep fetal hemoglobin, but not cloned alpha- or beta-chains, can produce this cooperative effect, as indeed can HPLC purified, heme-free, gamma-chains derived from cord blood fetal hemoglobin, and that purified haptoglobin completely abolishes the cooperative interaction.

From activity to active learning: drawing and dialogue interaction in virtual learning environments for computer-supported collaborative work in design.

Virtual learning environments (VLEs) would appear to be particular effective in computer-supported collaborative work (CSCW) for active learning. Most research studies looking at computer-supported collaborative design have focused on either synchronous or asynchronous modes of communication, but near-synchronous working has received relatively little attention. Yet it could be argued that near-synchronous communication encourages creative, rhetorical and critical exchanges of ideas, building on each other’s contributions. Furthermore, although many researchers have carried out studies on collaborative design protocol, argumentation and constructive interaction, little is known about the interaction between drawing and dialogue in near-synchronous collaborative design. The paper reports the first stage of an investigation into the requirements for the design and development of interactive systems to support the learning of collaborative design activities. The aim of the study is to understand the collaborative design processes while sketching in a shared white board and audio conferencing media. Empirical data on design processes have been obtained from observation of seven sessions with groups of design students solving an interior space-planning problem of a lounge-diner in a virtual learning environment, Lyceum, an in-house software developed by the Open University to support its students in collaborative learning.

Molecule modeling of human pentameric alpha(7) neuronal nicotinic acetylcholine receptor and its interaction with its agonist and competitive antagonist

The nicotinic Acetylcholine Receptor (nAChR) is the major class of neurotransmitter receptors that is involved in many neurodegenerative conditions such as schizophrenia, Alzheimer's and Parkinson's diseases. The N-terminal region or Ligand Binding Domain (LBD) of nAChR is located at pre- and post-synaptic nervous system, which mediates synaptic transmission. nAChR acts as the drug target for agonist and competitive antagonist molecules that modulate signal transmission at the nerve terminals. Based on Acetylcholine Binding Protein (AChBP) from Lymnea stagnalis as the structural template, the homology modeling approach was carried out to build three dimensional model of the N-terminal region of human alpha(7)nAChR. This theoretical model is an assembly of five alpha(7) subunits with 5 fold axis symmetry, constituting a channel, with the binding picket present at the interface region of the subunits. alpha-netlrotoxin is a potent nAChR competitive antagonist that readily blocks the channel resulting in paralysis. The molecular interaction of alpha-Bungarotoxin, a long chain alpha-neurotoxin from (Bungarus multicinctus) and human alpha(7)nAChR seas studied. Agonists such as acetylcholine, nicotine, which are used in it diverse array of biological activities, such as enhancements of cognitive performances, were also docked with the theoretical model of human alpha(7)nAChR. These docked complexes were analyzed further for identifying the crucial residues involved in interaction. These results provide the details of interaction of agonists and competitive antagonists with three dimensional model of the N-terminal region of human alpha(7)nAChR and thereby point to the design of novel lead compounds.

Structure, Dynamics, and RNA Interaction Analysis of the Human SBDS Protein

Shwachman-Bodian-Diamond syndrome is an autosomal recessive genetic syndrome with pleiotropic phenotypes, including pancreatic deficiencies, bone marrow dysfunctions with increased risk of myelodysplasia or leukemia, and skeletal abnormalities. This syndrome has been associated with mutations in the SBDS gene, which encodes a conserved protein showing orthologs in Archaea and eukaryotes. The Shwachman-Bodian-Diamond syndrome pleiotropic phenotypes may be an indication of different cell type requirements for a fully functional SBDS protein. RNA-binding activity has been predicted for archaeal and yeast SBDS orthologs, with the latter also being implicated in ribosome biogenesis. However, full-length SBDS orthologs function in a species-specific manner, indicating that the knowledge obtained from model systems may be of limited use in understanding major unresolved issues regarding SBDS function, namely, the effect of mutations in human SBDS on its biochemical function and the specificity of RNA interaction. We determined the solution structure and backbone dynamics of the human SBDS protein and describe its RNA binding site using NMR spectroscopy. Similarly to the crystal structures of Archaea, the overall structure of human SBDS comprises three well-folded domains. However, significant conformational exchange was observed in NMR dynamics experiments for the flexible linker between the N-terminal domain and the central domain, and these experiments also reflect the relative motions of the domains. RNA titrations monitored by heteronuclear correlation experiments and chemical shift mapping analysis identified a classic RNA binding site at the N-terminal FYSH (fungal, Yhr087wp, Shwachman) domain that concentrates most of the mutations described for the human SBDS. (C) 2010 Elsevier Ltd. All rights reserved.

Study of water and dimethylformamide interaction by computer simulation

Monte Carlo simulations of water-dimethylformamide (DMF) mixtures were performed in the isothermal and isobaric ensemble at 298.15 K and 1 atm. The intermolecular interaction energy was calculated using the classical 6-12 Lennard-Jones pairwise potential plus a Coulomb term. The TIP4P model was used for simulating water molecules, and a six-site model previously optimised by us was used to represent DMF. The potential energy for the water-DMF interaction was obtained via standard geometric combining rules using the original potential parameters for the pure liquids. The radial distribution functions calculated for water-DMF mixtures show well characterised hydrogen bonds between the oxygen site of DMF and hydrogen of water. A structureless correlation curve was observed for the interaction between the hydrogen site of the carbonyl group and the oxygen site of water. Hydration effects on the stabilisation of the DMF molecule in aqueous solution have been investigated using statistical perturbation theory. The results show that energetic changes involved in the hydration process are not strong enough to stabilise another configuration of DMF than the planar one.

Desenvolvimento de uma base de dados para fatores de transcrição de seres humanos e suas redes de interação: Human Transcriptional Regulation Interaction Database (HTRIDB 2.0)

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)