Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

Distance perception in immersive environments: the role of photorealism

Immersive environments (IE) are being increasingly used in order to perform psychophysical experiments. The versatility in terms of stimuli presentation and control and the less time-consuming procedures are their greatest strengths. However, to ensure that IE results can be generalized to real world scenarios we must first provide evidence that performance in IE is quantitatively indistinguishable from performance in real-world. Our goal was to perceptually validate distance perception for CAVE-like IEs. Participants performed a Frontal Matching Distance Task (Durgin & Li, 2011) in three different conditions: real-world scenario (RWS); photorealistic IE (IEPH) and non-photorealistic IE (IENPH). Underestimation of distance was found across all the conditions, with a significant difference between the three conditions (Wilks’ Lambda = .38, F(2,134)= 110.8, p<.01, significant pairwise differences with p<.01). We found a mean error of 2.3 meters for the RWS, 5 meters for the IEPH, and of 6 meters for the IENPH in a pooled data set of 5 participants. Results indicate that while having a photorealistic IE with perspective and stereoscopic depth cues might not be enough to elicit a real-world performance in distance judgment tasks, nevertheless this type of environment minimizes the discrepancy between simulation and real-world when compared with non-photorealistic IEs.

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores