Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

The human eye as human-machine interface

Eye tracking as an interface to operate a computer is under research for a while and new systems are still being developed nowadays that provide some encouragement to those bound to illnesses that incapacitates them to use any other form of interaction with a computer. Although using computer vision processing and a camera, these systems are usually based on head mount technology being considered a contact type system. This paper describes the implementation of a human-computer interface based on a fully non-contact eye tracking vision system in order to allow people with tetraplegia to interface with a computer. As an assistive technology, a graphical user interface with special features was developed including a virtual keyboard to allow user communication, fast access to pre-stored phrases and multimedia and even internet browsing. This system was developed with the focus on low cost, user friendly functionality and user independency and autonomy.

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

Global vs. local nonlinear optimization techniques for human-like movement of an anthropomorphic robot

In this paper a comparison between using global and local optimization techniques for solving the problem of generating human-like arm and hand movements for an anthropomorphic dual arm robot is made. Although the objective function involved in each optimization problem is convex, there is no evidence that the admissible regions of these problems are convex sets. For the sequence of movements for which the numerical tests were done there were no significant differences between the optimal solutions obtained using the global and the local techniques. This suggests that the optimal solution obtained using the local solver is indeed a global solution.

Nonlinear optimization for human-like synchronous movements of a dual arm-hand robotic system

In previous work we have presented a model capable of generating human-like movements for a dual arm-hand robot involved in human-robot cooperative tasks. However, the focus was on the generation of reach-to-grasp and reach-to-regrasp bimanual movements and no synchrony in timing was taken into account. In this paper we extend the previous model in order to accomplish bimanual manipulation tasks by synchronously moving both arms and hands of an anthropomorphic robotic system. Specifically, the new extended model has been designed for two different tasks with different degrees of difficulty. Numerical results were obtained by the implementation of the IPOPT solver embedded in our MATLAB simulator.

Towards human-like bimanual movements in anthropomorphic robots: a nonlinear optimization approach

Previously we have presented a model for generating human-like arm and hand movements on an unimanual anthropomorphic robot involved in human-robot collaboration tasks. The present paper aims to extend our model in order to address the generation of human-like bimanual movement sequences which are challenged by scenarios cluttered with obstacles. Movement planning involves large scale nonlinear constrained optimization problems which are solved using the IPOPT solver. Simulation studies show that the model generates feasible and realistic hand trajectories for action sequences involving the two hands. The computational costs involved in the planning allow for real-time human robot-interaction. A qualitative analysis reveals that the movements of the robot exhibit basic characteristics of human movements.

Biomimetic strategies to engineer mineralised human tissues

In the last few years, many reports have been describing promising biocompatible and biodegradable materials that can mimic in a certain extent the multidimensional hierarchical structure of bone, while are also capable of releasing bioactive agents or drugs in a controlled manner. Despite these great advances, new developments in the design and fabrication technologies are required to address the need to engineer suitable biomimetic materials in order tune cells functions, i.e. enhance cell-biomaterial interactions, and promote cell adhesion, proliferation, and differentiation ability. Scaffolds, hydrogels, fibres and composite materials are the most commonly used as biomimetics for bone tissue engineering. Dynamic systems such as bioreactors have also been attracting great deal of attention as it allows developing a wide range of novel in vitro strategies for the homogeneous coating of scaffolds and prosthesis with ceramics, and production of biomimetic constructs, prior its implantation in the body. Herein, it is overviewed the biomimetic strategies for bone tissue engineering, recent developments and future trends. Conventional and more recent processing methodologies are also described.

Human papillomavirus (HPV) screening and cervical cancer burden. A Brazilian perspective

This review tackles the issues related to disease burden caused by cervical cancer (CC) and its precursor (CIN) lesions in Brazil. A special focus is given to new technologies with potential to interfere with the development of CC by reducing the high-risk human papillomavirus (hr-HPV)-induced lesions that remain a major public health burden in all developing countries where organized screening programs do not exist. Globally, 85 % of all incident CC and 50 % of CC deaths occur in the developing countries. Unfortunately, most regions of Brazil still demonstrate high mortality rates, ranking CC as the second most common cancer among Brazilian women. Recently, CC screening programs have been tailored in the country to enable early detection of CC precursor lesions and thereby reduce cancer mortality. A combination of HPV testing with liquid-based cytology (LBC) seems to be a promising new approach in CC screening, with high expectation to offer an adequate control of CC burden in this country.

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores