Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

Different visual exploration of tool-related gestures in left hemisphere brain damaged patients is associated with poor gestural imitation.

According to the direct matching hypothesis, perceived movements automatically activate existing motor components through matching of the perceived gesture and its execution. The aim of the present study was to test the direct matching hypothesis by assessing whether visual exploration behavior correlate with deficits in gestural imitation in left hemisphere damaged (LHD) patients. Eighteen LHD patients and twenty healthy control subjects took part in the study. Gesture imitation performance was measured by the test for upper limb apraxia (TULIA). Visual exploration behavior was measured by an infrared eye-tracking system. Short videos including forty gestures (20 meaningless and 20 communicative gestures) were presented. Cumulative fixation duration was measured in different regions of interest (ROIs), namely the face, the gesturing hand, the body, and the surrounding environment. Compared to healthy subjects, patients fixated significantly less the ROIs comprising the face and the gesturing hand during the exploration of emblematic and tool-related gestures. Moreover, visual exploration of tool-related gestures significantly correlated with tool-related imitation as measured by TULIA in LHD patients. Patients and controls did not differ in the visual exploration of meaningless gestures, and no significant relationships were found between visual exploration behavior and the imitation of emblematic and meaningless gestures in TULIA. The present study thus suggests that altered visual exploration may lead to disturbed imitation of tool related gestures, however not of emblematic and meaningless gestures. Consequently, our findings partially support the direct matching hypothesis.

Score optimization and template updating in a biometric technique for authentication in mobiles based on gestures

This article focuses on the evaluation of a biometric technique based on the performance of an identifying gesture by holding a telephone with an embedded accelerometer in his/her hand. The acceleration signals obtained when users perform gestures are analyzed following a mathematical method based on global sequence alignment. In this article, eight different scores are proposed and evaluated in order to quantify the differences between gestures, obtaining an optimal EER result of 3.42% when analyzing a random set of 40 users of a database made up of 80 users with real attempts of falsification. Moreover, a temporal study of the technique is presented leeding to the need to update the template to adapt the manner in which users modify how they perform their identifying gesture over time. Six updating schemes have been assessed within a database of 22 users repeating their identifying gesture in 20 sessions over 4 months, concluding that the more often the template is updated the better and more stable performance the technique presents.

Grasp Mapping Between a 3-Finger Haptic Device and a Robotic Hand

This paper presents the implementation of a robust grasp mapping between a 3-finger haptic device (master) and a robotic hand (slave). Mapping is based on a grasp equivalence defined considering the manipulation capabilities of the master and slave devices. The metrics that translate the human hand gesture to the robotic hand workspace are obtained through an analytical user study. This allows a natural control of the robotic hand. The grasp mapping is accomplished defining 4 control modes that encapsulate all the grasps gestures considered.

Neural basis of lexical-semantic processing and integration of symbolic gestures with words

Three studies investigated the relation between symbolic gestures and words, aiming at discover the neural basis and behavioural features of the lexical semantic processing and integration of the two communicative signals. The first study aimed at determining whether elaboration of communicative signals (symbolic gestures and words) is always accompanied by integration with each other and, if present, this integration can be considered in support of the existence of a same control mechanism. Experiment 1 aimed at determining whether and how gesture is integrated with word. Participants were administered with a semantic priming paradigm with a lexical decision task and pronounced a target word, which was preceded by a meaningful or meaningless prime gesture. When meaningful, the gesture could be either congruent or incongruent with word meaning. Duration of prime presentation (100, 250, 400 ms) randomly varied. Voice spectra, lip kinematics, and time to response were recorded and analyzed. Formant 1 of voice spectra, and mean velocity in lip kinematics increased when the prime was meaningful and congruent with the word, as compared to meaningless gesture. In other words, parameters of voice and movement were magnified by congruence, but this occurred only when prime duration was 250 ms. Time to response to meaningful gesture was shorter in the condition of congruence compared to incongruence. Experiment 2 aimed at determining whether the mechanism of integration of a prime word with a target word is similar to that of a prime gesture with a target word. Formant 1 of the target word increased when word prime was meaningful and congruent, as compared to meaningless congruent prime. Increase was, however, present for whatever prime word duration. In the second study, experiment 3 aimed at determining whether symbolic prime gesture comprehension makes use of motor simulation. Transcranial Magnetic Stimulation was delivered to left primary motor cortex 100, 250, 500 ms after prime gesture presentation. Motor Evoked Potential of First Dorsal Interosseus increased when stimulation occurred 100 ms post-stimulus. Thus, gesture was understood within 100ms and integrated with the target word within 250 ms. Experiment 4 excluded any hand motor simulation in order to comprehend prime word. The effect of the prior presentation of a symbolic gesture on congruent target word processing was investigated in study 3. In experiment 5, symbolic gestures were presented as primes, followed by semantically congruent target word or pseudowords. In this case, lexical-semantic decision was accompanied by a motor simulation at 100ms after the onset of the verbal stimuli. Summing up, the same type of integration with a word was present for both prime gesture and word. It was probably subsequent to understanding of the signal, which used motor simulation for gesture and direct access to semantics for words. However, gesture and words could be understood at the same motor level through simulation if words were preceded by an adequate gestural context. Results are discussed in the prospective of a continuum between transitive actions and emblems, in parallelism with language; the grounded/symbolic content of the different signals evidences relation between sensorimotor and linguistic systems, which could interact at different levels.

Hand gesture recognition based on morphologic features

Comunicación presentada en el IX Simposium Nacional de Reconocimiento de Formas y Análisis de Imágenes, Benicàssim, Mayo, 2001.

3D Hand Pose Estimation with Neural Networks

We propose the design of a real-time system to recognize and interprethand gestures. The acquisition devices are low cost 3D sensors. 3D hand pose will be segmented, characterized and track using growing neural gas (GNG) structure. The capacity of the system to obtain information with a high degree of freedom allows the encoding of many gestures and a very accurate motion capture. The use of hand pose models combined with motion information provide with GNG permits to deal with the problem of the hand motion representation. A natural interface applied to a virtual mirrorwriting system and to a system to estimate hand pose will be designed to demonstrate the validity of the system.

Work at hand: an exploration of gesture in the context of work and everyday life to inform the design of gestural input devices

This research explores gestures used in the context of activities in the workplace and in everyday life in order to understand requirements and devise concepts for the design of gestural information appliances. A collaborative method of video interaction analysis devised to suit design explorations, the Video Card Game, was used to capture and analyse how gesture is used in the context of six different domains: the dentist's office; PDA and mobile phone use; the experimental biologist's laboratory; a city ferry service; a video cassette player repair shop; and a factory flowmeter assembly station. Findings are presented in the form of gestural themes, derived from the tradition of qualitative analysis but bearing some similarity to Alexandrian patterns. Implications for the design of gestural devices are discussed.

Hand pose detection for vision-based gesture interfaces

[EN]Vision-based applications designed for humanmachine interaction require fast and accurate hand detection. However, previous works on this field assume different constraints, like a limitation in the number of detected gestures, because hands are highly complex objects to locate. This paper presents an approach which changes the detection target without limiting the number of detected gestures. Using a cascade classifier we detect hands based on their wrists. With this approach, we introduce two main contributions: (1) a reliable segmentation, independently of the gesture being made and (2) a training phase faster than previous cascade classifier based methods. The paper includes experimental evaluations with different video streams that illustrate the efficiency and suitability for perceptual interfaces.

Fast and accurate hand pose detection for human-robot interaction

[EN]Enabling natural human-robot interaction using computer vision based applications requires fast and accurate hand detection. However, previous works in this field assume different constraints, like a limitation in the number of detected gestures, because hands are highly complex objects difficult to locate. This paper presents an approach which integrates temporal coherence cues and hand detection based on wrists using a cascade classifier. With this approach, we introduce three main contributions: (1) a transparent initialization mechanism without user participation for segmenting hands independently of their gesture, (2) a larger number of detected gestures as well as a faster training phase than previous cascade classifier based methods and (3) near real-time performance for hand pose detection in video streams.

[factors That Influence Bone Mass Of Healthy Children And Adolescents Measured By Quantitative Ultrasound At The Hand Phalanges: A Systematic Review].

To analyze the main factors that influence bone mass in children and teenagers assessed by quantitative ultrasound (QUS) of the phalanges. A systematic literature review was performed according to the PRISMA method with searches in databases Pubmed/Medline, SciELO and Bireme for the period 2001-2012, in English and Portuguese languages, using the keywords: children, teenagers, adolescent, ultrasound finger phalanges, quantitative ultrasound of phalanges, phalangeal quantitative ultrasound. 21 articles were included. Girls had, in QUS, Amplitude Dependent Speed of Sound (AD-SoS) values higher than boys during pubertal development. The values of the parameters of QUS of the phalanges and dual-energy X-ray Absorptiometry (DXA) increased with the increase of the maturational stage. Anthropometric variables such as age, weight, height, body mass index (BMI), lean mass showed positive correlations with the values of QUS of the phalanges. Physical activity has also been shown to be positively associated with increased bone mass. Factors such as ethnicity, genetics, caloric intake and socioeconomic profile have not yet shown a conclusive relationship and need a larger number of studies. QUS of the phalanges is a method used to evaluate the progressive acquisition of bone mass during growth and maturation of individuals in school phase, by monitoring changes that occur with increasing age and pubertal stage. There were mainly positive influences in variables of sex, maturity, height, weight and BMI, with similar data when compared to the gold standard method, the DXA.

Corpo-movimento-deficiencia : as formas dos discursos da/na dança em cadeira de rodas e seus processos de significação

Universidade Estadual de Campinas. Faculdade de Educação Física

Hand and ultrasonic instrumentation for orthograde root canal treatment of permanent teeth

Root canal treatment is a frequently performed dental procedure and is carried out on teeth in which irreversible pulpitis has led to necrosis of the dental pulp. Removal of the necrotic tissue remnants and cleaning and shaping of the root canal are important phases of root canal treatment. Treatment options include the use of hand and rotary instruments and methods using ultrasonic or sonic equipment. OBJECTIVES: The objectives of this systematic review of randomized controlled trials were to determine the relative clinical effectiveness of hand instrumentation versus ultrasonic instrumentation alone or in conjunction with hand instrumentation for orthograde root canal treatment of permanent teeth. MATERIAL AND METHODS: The search strategy retrieved 226 references from the Cochrane Oral Health Group Trials Register (7), the Cochrane Central Register of Controlled Trials (CENTRAL) (12), MEDLINE (192), EMBASE (8) and LILACS (7). No language restriction was applied. The last electronic search was conducted on December 13th, 2007. Screening of eligible studies was conducted in duplicate and independently. RESULTS: Results were to be expressed as fixed-effect or random-effects models using mean differences for continuous outcomes and risk ratios for dichotomous outcomes with 95% confdence intervals. Heterogeneity was to be investigated including both clinical and methodological factors. No eligible randomized controlled trials were identifed. CONCLUSIONS: This review illustrates the current lack of published or ongoing randomized controlled trials and the unavailability of high-level evidence based on clinically relevant outcomes referring to the effectiveness of ultrasonic instrumentation used alone or as an adjunct to hand instrumentation for orthograde root canal treatment. In the absence of reliable research-based evidence, clinicians should base their decisions on clinical experience, individual circumstances and in conjunction with patients' preferences where appropriate. Future randomized controlled trials might focus more closely on evaluating the effectiveness of combinations of these interventions with an emphasis on not only clinically relevant, but also patient-centered outcomes.

Developmental Potential of Bovine Hand-Made Clone Embryos Reconstructed by Aggregation or Fusion with Distinct Cytoplasmic Volumes

Animal cloning has been associated with developmental abnormalities, with the level of heteroplasmy caused by the procedure being one of its potential limiting factors. The aim of this study was to determine the effect of the fusion of hemicytoplasts or aggregation of hemiembryos, varying the final cytoplasmic volume, on development and cell density of embryos produced by hand-made cloning (HMC), parthenogenesis or by in vitro fertilization (IVF). One or two enucleated hemicytoplasts were paired and fused with one skin somatic cell. Activated clone and zona-free parthenote embryos and hemiembryos were in vitro cultured in the well-of-the-well (WOW) system, being allocated to one of six experimental groups, on a per WOW basis: single clone or parthenote hemiembryos (1 x 50%); aggregation of two (2 x 50%), three (3 x 50%), or four (4 x 50%) clone or parthenote hemiembryos; single clone or parthenote embryos (1 x 100%); or aggregation of two clone or parthenote embryos (2 x 100%). Control zona-intact parthenote or IVF embryos were in vitro cultured in four-well dishes. Results indicated that the increase in the number of aggregated structures within each WOW was followed by a linear increase in cleavage, blastocyst rate, and cell density. The increase in cytoplasmic volume, either by fusion or by aggregation, had a positive effect on embryo development, supporting the establishment of pregnancies and the birth of a viable clone calf after transfer to recipients. However, embryo aggregation did not improve development on a hemicytoplast basis, except for the aggregation of two clone embryos.

Hand position on the bunch and source-sink ratio influence the banana fruit susceptibility to crown rot disease

The postharvest development of crown rot of bananas depends notably on the fruit susceptibility to this disease at harvest. It has been shown that fruit susceptibility to crown rot is variable and it was suggested that this depends on environmental preharvest factors. However, little is known about the preharvest factors influencing this susceptibility. The aim of this work was to evaluate the extent to which fruit filling characteristics during growth and the fruit development stage influence the banana susceptibility to crown rot. This involved evaluating the influence of (a) the fruit position at different levels of the banana bunch (hands) and (b) changing the source-sink ratio (So-Si ratio), on the fruit susceptibility to crown rot. The fruit susceptibility was determined by measuring the internal necrotic surface (INS) after artificial inoculation of Colletotrichum musae. A linear correlation (r = -0.95) was found between the hand position on the bunch and the INS. The So-Si ratio was found to influence the pomological characteristics of the fruits and their susceptibility to crown rot. Fruits of bunches from which six hands were removed (two hands remaining on the bunch) proved to be significantly less susceptible to crown rot (INS = 138.3 mm 2) than those from bunches with eight hands (INS = 237.9 mm 2). The banana susceptibility to crown rot is thus likely to be influenced by the fruit development stage and filling characteristics. The present results highlight the importance of standardising hand sampling on a bunch when testing fruit susceptibility to crown rot. They also show that hand removal in the field has advantages in the context of integrated pest management, making it possible to reduce fruit susceptibility to crown rot while increasing fruit size.