Hand Gesture Detection & Recognition System

The project introduces an application using computer vision for Hand gesture recognition. A camera records a live video stream, from which a snapshot is taken with the help of interface. The system is trained for each type of count hand gestures (one, two, three, four, and five) at least once. After that a test gesture is given to it and the system tries to recognize it.A research was carried out on a number of algorithms that could best differentiate a hand gesture. It was found that the diagonal sum algorithm gave the highest accuracy rate. In the preprocessing phase, a self-developed algorithm removes the background of each training gesture. After that the image is converted into a binary image and the sums of all diagonal elements of the picture are taken. This sum helps us in differentiating and classifying different hand gestures.Previous systems have used data gloves or markers for input in the system. I have no such constraints for using the system. The user can give hand gestures in view of the camera naturally. A completely robust hand gesture recognition system is still under heavy research and development; the implemented system serves as an extendible foundation for future work.

Combating shoulder-surfing: a hidden button gesture based scheme

This project describes an authentication technique that is shoulder-surfing resistant. Shoulder surfing is an attack in which an attacker can get access to private information by observing the user’s interaction with a terminal, or by using recording tools to record the user interaction and study the obtained data, with the objective of obtaining unauthorized access to a target user’s personal information. The technique described here relies on gestural analysis coupled with a secondary channel of authentication that uses button pressing. The thesis presents and evaluates multiple alternative algorithms for gesture analysis, and furthermore assesses the effectiveness of the technique.

A software development process model for gesture-based interface

Gesture-based applications have particularities, since users interact in a natural way, much as they interact in the non-digital world. Hence, new requirements are needed on the software design process. This paper shows a software development process model for these applications, including requirement specification, design, implementation, and testing procedures. The steps and activities of the proposed model were tested through a game case study, which is a puzzle game. The puzzle is completed when all pieces of a painting are correctly positioned by the drag and drop action of users hand gesture. It also shows the results obtained of applying a heuristic evaluation on this game. © 2012 IEEE.

Gesture-based teleoperation using a holonomic robot

This work presents the development and integration of an user interface (UI) framework based on various current input devices that take advantage of our ergonomics. The purpose is to teleoperate a holonomic robot using upper member gestures and postures for studying the suitable of such interfaces when programming and interacting with a mobile robot. As performance vary from UI to UI the framework is focused to be used as a complementary industrial or didactic tool thus, changing how inexperience users tackle their first impressions when working with mobile robots while performing simple gesture-based teleoperation tasks. © 2012 ICROS.

Deep learning architecture for gesture recognition

Pós-graduação em Engenharia Mecânica - FEG

Eye-trauma: Event driven control software for gesture recognition in medical simulation

Il progetto Eye-Trauma si colloca all'interno dello sviluppo di un simulatore chirurgico per traumi alla zona oculare, sviluppato in collaborazione con Simulation Group in Boston, Harvard Medical School e Massachusetts General Hospital. Il simulatore presenta un busto in silicone fornito di moduli intercambiabili della zona oculare, per simulare diversi tipi di trauma. L'utilizzatore è chiamato ad eseguire la procedura medica di saturazione tramite degli strumenti chirurgici su cui sono installati dei sensori di forza e di apertura. I dati collezionati vengono utilizzati all'interno del software per il riconoscimento dei gesti e il controllo real-time della performance. L'algoritmo di gesture recognition, da me sviluppato, si basa sul concetto di macchine a stati; la transizione tra gli stati avviene in base agli eventi rilevati dal simulatore.

Progettazione di un'applicazione di tipo "gesture-based" per il controllo di velivoli senza pilota a bordo

Progettazione e programmazione di un interfaccia per il controllo col movimento della mano di un simulatore di UAV.

Comprehensive assessment of gesture production: a new test of upper limb apraxia (TULIA)

BACKGROUND: Only few standardized apraxia scales are available and they do not cover all domains and semantic features of gesture production. Therefore, the objective of the present study was to evaluate the reliability and validity of a newly developed test of upper limb apraxia (TULIA), which is comprehensive and still short to administer. METHODS: The TULIA consists of 48 items including imitation and pantomime domain of non-symbolic (meaningless), intransitive (communicative) and transitive (tool related) gestures corresponding to 6 subtests. A 6-point scoring method (0-5) was used (score range 0-240). Performance was assessed by blinded raters based on videos in 133 stroke patients, 84 with left hemisphere damage (LHD) and 49 with right hemisphere damage (RHD), as well as 50 healthy subjects (HS). RESULTS: The clinimetric findings demonstrated mostly good to excellent internal consistency, inter- and intra-rater (test-retest) reliability, both at the level of the six subtests and at individual item level. Criterion validity was evaluated by confirming hypotheses based on the literature. Construct validity was demonstrated by a high correlation (r = 0.82) with the De Renzi-test. CONCLUSION: These results show that the TULIA is both a reliable and valid test to systematically assess gesture production. The test can be easily applied and is therefore useful for both research purposes and clinical practice.

Interference with gesture production by theta burst stimulation over left inferior frontal cortex

The traditional view of a predominant inferior parietal representation of gestures has been recently challenged by neuroimaging studies demonstrating that gesture production and discrimination may critically depend on inferior frontal lobe function. The aim of the present work was therefore to investigate the effect of transient disruption of these brain sites by continuous theta burst stimulation (cTBS) on gesture production and recognition.

Gesture-Based, Touch-Free Multi-User Gaming on Wall-Sized, High-Resolution Tiled Displays

Having to carry input devices can be inconvenient when interacting with wall-sized, high-resolution tiled displays. Such displays are typically driven by a cluster of computers. Running existing games on a cluster is non-trivial, and the performance attained using software solutions like Chromium is not good enough. This paper presents a touch-free, multi-user, humancomputer interface for wall-sized displays that enables completely device-free interaction. The interface is built using 16 cameras and a cluster of computers, and is integrated with the games Quake 3 Arena (Q3A) and Homeworld. The two games were parallelized using two different approaches in order to run on a 7x4 tile, 21 megapixel display wall with good performance. The touch-free interface enables interaction with a latency of 116 ms, where 81 ms are due to the camera hardware. The rendering performance of the games is compared to their sequential counterparts running on the display wall using Chromium. Parallel Q3A’s framerate is an order of magnitude higher compared to using Chromium. The parallel version of Homeworld performed on par with the sequential, which did not run at all using Chromium. Informal use of the touch-free interface indicates that it works better for controlling Q3A than Homeworld.

Estimating Gesture Accuracy in Motion-Based Health Games

This manuscript details a technique for estimating gesture accuracy within the context of motion-based health video games using the MICROSOFT KINECT. We created a physical therapy game that requires players to imitate clinically significant reference gestures. Player performance is represented by the degree of similarity between the performed and reference gestures and is quantified by collecting the Euler angles of the player's gestures, converting them to a three-dimensional vector, and comparing the magnitude between the vectors. Lower difference values represent greater gestural correspondence and therefore greater player performance. A group of thirty-one subjects was tested. Subjects achieved gestural correspondence sufficient to complete the game's objectives while also improving their ability to perform reference gestures accurately.

Impaired gesture performance in schizophrenia: particular vulnerability of meaningless pantomimes

Schizophrenia patients frequently present with subtle motor impairments, including higher order motor function such as hand gesture performance. Using cut off scores from a standardized gesture test, we previously reported gesture deficits in 40% of schizophrenia patients irrespective of the gesture content. However, these findings were based on normative data from an older control group. Hence, we now aimed at determining cut-off scores in an age and gender matched control group. Furthermore, we wanted to explore whether gesture categories are differentially affected in Schizophrenia. Gesture performance data of 30 schizophrenia patients and data from 30 matched controls were compared. Categories included meaningless, intransitive (communicative) and transitive (object related) hand gestures, which were either imitated or pantomimed, i.e. produced on verbal command. Cut-off scores of the age matched control group were higher than the previous cut-off scores in an older control group. An ANOVA tested effects of group, domain (imitation or pantomime), and semantic category (meaningless, transitive or intransitive), as well as their interaction. According to the new cut-off scores, 67% of the schizophrenia patients demonstrated gestural deficits. Patients performed worse in all gesture categories, however meaningless gestures on verbal command were particularly impaired (p = 0.008). This category correlated with poor frontal lobe function (p < 0.001). In conclusion, gestural deficits in schizophrenia are even more frequent than previously reported. Gesture categories that pose higher demands on planning and selection such as pantomime of meaningless gestures are predominantly affected and associated with the well-known frontal lobe dysfunction.