Hand gesture recognition for human computer interaction : a comparative study of different image features

Hand gesture recognition for human computer interaction, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. In this study we try to identify hand features that, isolated, respond better in various situations in human-computer interaction. The extracted features are used to train a set of classifiers with the help of RapidMiner in order to find the best learner. A dataset with our own gesture vocabulary consisted of 10 gestures, recorded from 20 users was created for later processing. Experimental results show that the radial signature and the centroid distance are the features that when used separately obtain better results, with an accuracy of 91% and 90,1% respectively obtained with a Neural Network classifier. These to methods have also the advantage of being simple in terms of computational complexity, which make them good candidates for real-time hand gesture recognition.

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Generic system for human-computer gesture interaction

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for humancomputer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"

STAR: Speech Therapy with Augmented Reality for children with autism spectrum disorders

Graphics based systems of Augmented and Alternative Communication are widely used to promote communication in people with Autism Spectrum Disorders. This study discusses an integration of Augmented Reality in communication interventions, by relating elements of Augmented and Alternative Communication and Applied Behaviour Analysis strategies. An architecture for an Augmented Reality based interactive system to assist interventions is proposed. STAR provides an Augmented Reality tool to assist interventions performed by therapists and support for parents to join in and participate in the child’s intervention. Finally we report on the usage of the Augmented Reality tool in interventions with children with Autism Spectrum Disorders.

The human eye as human-machine interface

Eye tracking as an interface to operate a computer is under research for a while and new systems are still being developed nowadays that provide some encouragement to those bound to illnesses that incapacitates them to use any other form of interaction with a computer. Although using computer vision processing and a camera, these systems are usually based on head mount technology being considered a contact type system. This paper describes the implementation of a human-computer interface based on a fully non-contact eye tracking vision system in order to allow people with tetraplegia to interface with a computer. As an assistive technology, a graphical user interface with special features was developed including a virtual keyboard to allow user communication, fast access to pre-stored phrases and multimedia and even internet browsing. This system was developed with the focus on low cost, user friendly functionality and user independency and autonomy.

Study of the interaction between asphalt and plastic wastes in new polymer modified binders (PMB)

The aim of this study is evaluating the interaction between several base pen grade asphalt binders (35/50, 50/70, 70/100, 160/220) and two different plastic wastes (EVA and HDPE), for a set of new polymer modified binders produced with different amounts of both plastic wastes. After analysing the results obtained for the several polymer modified binders evaluated in this study, including a commercial modified binder, it can be concluded that the new PMBs produced with the base bitumen 70/100 and 5% of each plastic waste (HDPE or EVA) results in binders with very good performance, similar to that of the commercial modified binder.

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

Dynamic group formation in mobile computer supported collaborative learning environment

Forming suitable learning groups is one of the factors that determine the efficiency of collaborative learning activities. However, only a few studies were carried out to address this problem in the mobile learning environments. In this paper, we propose a new approach for an automatic, customized, and dynamic group formation in Mobile Computer Supported Collaborative Learning (MCSCL) contexts. The proposed solution is based on the combination of three types of grouping criteria: learner’s personal characteristics, learner’s behaviours, and context information. The instructors can freely select the type, the number, and the weight of grouping criteria, together with other settings such as the number, the size, and the type of learning groups (homogeneous or heterogeneous). Apart from a grouping mechanism, the proposed approach represents a flexible tool to control each learner, and to manage the learning processes from the beginning to the end of collaborative learning activities. In order to evaluate the quality of the implemented group formation algorithm, we compare its Average Intra-cluster Distance (AID) with the one of a random group formation method. The results show a higher effectiveness of the proposed algorithm in forming homogenous and heterogeneous groups compared to the random method.

Interaction paradigms versus age-related user profiles: an evaluation on content selection

Novel input modalities such as touch, tangibles or gestures try to exploit human's innate skills rather than imposing new learning processes. However, despite the recent boom of different natural interaction paradigms, it hasn't been systematically evaluated how these interfaces influence a user's performance or whether each interface could be more or less appropriate when it comes to: 1) different age groups; and 2) different basic operations, as data selection, insertion or manipulation. This work presents the first step of an exploratory evaluation about whether or not the users' performance is indeed influenced by the different interfaces. The key point is to understand how different interaction paradigms affect specific target-audiences (children, adults and older adults) when dealing with a selection task. 60 participants took part in this study to assess how different interfaces may influence the interaction of specific groups of users with regard to their age. Four input modalities were used to perform a selection task and the methodology was based on usability testing (speed, accuracy and user preference). The study suggests a statistically significant difference between mean selection times for each group of users, and also raises new issues regarding the “old” mouse input versus the “new” input modalities.

Ergonomic evaluation of office workplaces with Rapid Office Strain Assessment (ROSA)

The regular use of the computer in the office contributed to the appearance of many risk factors related with work-related musculoskeletal disorders (WRMSD) such as maintaining static sitting postures for long time and awkward postures of the head, neck and upper limbs, leading to increased muscle activity in the cervical spine and shoulders. The objective of this study was to evaluate the presence of risk factors for WRMSD in an office using the Rapid Assessment Office Strain method (ROSA). Based on the results of this ergonomic evaluation, an occupational gym program was designed and implemented. Thirty-eight workplaces were evaluated using the observation of the tasks and pictures records in order to characterize those tasks in more detail. The ROSA tool was applied by an observer, who selected the appropriate score based on the worker's posture as well as the time spent in each posture. Scores were recorded for the sections of the method, specifically Chair, Monitor and Mouse and Keyboard and Telephone. The scores were recorded in a sheet developed for the method. The mean ROSA final score was 3.61 ± 0.64, for Chair section was 3.45 ± 0.55, to Monitor and Telephone section was 3.11 ± 0.61, and to Mouse and Keyboard section was 2.11 ± 0.31. The results led to understand that the analyzed tasks represent situations of risk of discomfort and, according to the methods guidelines, further research and modifications of the workplace may be necessary. It should be emphasized that these scores may not be related to the poor available equipment but with the need to optimize their use by the workers. It was noticed also that the interaction of workers with the tasks and the adopted sitting posture at the computer throughout the day have effects at a muscular level, essentially for the cervical area and shoulders. ROSA tool is an useful and easy method to assess several risk factors associated with WRMSD, also allowing the design of specific occupational gym programs.

Renewable energy system for an isolated micro grid

This paper presents the development of the power electronics needed for the interaction between the electrical generator of a wind turbine and an isolated ac micro grid. In this system there are basically two types of receptors for the energy produced by the wind turbine, which are the loads connected to the isolated micro grid and the batteries used to store energy. There are basically two states in which the system will work. One of the states is when there is enough wind power to supply the loads and the extra energy is used to charge the batteries. The other state is when there is low wind power and the batteries have to compensate the lack of power, so that the isolated micro grid has enough power to supply at least the priority loads. In this paper are presented the hardware and the control algorithm for the developed system. The topology was previously tested in computer simulations, using the software PSIM 9.0, and then validated with the implementation of a laboratory prototype.

(Un)suitability of the use of pH buffers in biological, biochemical and environmental studies and their interaction with metal ions - a review

The use of buffers to maintain the pH within a desired range is a very common practice in chemical, biochemical and biological studies. Among them, zwitterionic N-substituted aminosulfonic acids, usually known as Good's buffers, although widely used, can complex metals and interact with biological systems. The present work reviews, discusses and updates the metal complexation characteristics of thirty one commercially available buffers. In addition, their impact on biological systems is also presented. The influences of these buffers on the results obtained in biological, biochemical and environmental studies, with special focus on their interaction with metal ions, are highlighted and critically reviewed. Using chemical speciation simulations, based on the current knowledge of the metal-buffer stability constants, a proposal of the most adequate buffer to employ for a given metal ion is presented.