Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Nonlinear optimization for human-like synchronous movements of a dual arm-hand robotic system

In previous work we have presented a model capable of generating human-like movements for a dual arm-hand robot involved in human-robot cooperative tasks. However, the focus was on the generation of reach-to-grasp and reach-to-regrasp bimanual movements and no synchrony in timing was taken into account. In this paper we extend the previous model in order to accomplish bimanual manipulation tasks by synchronously moving both arms and hands of an anthropomorphic robotic system. Specifically, the new extended model has been designed for two different tasks with different degrees of difficulty. Numerical results were obtained by the implementation of the IPOPT solver embedded in our MATLAB simulator.

Review of robotic technology for keyhole transcranial stereotactic neurosurgery

The research of stereotactic apparatus to guide surgical devices began in 1908, yet a major part of today's stereotactic neurosurgeries still rely on stereotactic frames developed almost half a century ago. Robots excel at handling spatial information, and are, thus, obvious candidates in the guidance of instrumentation along precisely planned trajectories. In this review, we introduce the concept of stereotaxy and describe a standard stereotactic neurosurgery. Neurosurgeons' expectations and demands regarding the role of robots as assistive tools are also addressed. We list the most successful robotic systems developed specifically for or capable of executing stereotactic neurosurgery. A critical review is presented for each robotic system, emphasizing the differences between them and detailing positive features and drawbacks. An analysis of the listed robotic system features is also undertaken, in the context of robotic application in stereotactic neurosurgery. Finally, we discuss the current perspective, and future directions of a robotic technology in this field. All robotic systems follow a very similar and structured workflow despite the technical differences that set them apart. No system unequivocally stands out as an absolute best. The trend of technological progress is pointing toward the development of miniaturized cost-effective solutions with more intuitive interfaces.

Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"

Generating timed trajectories foran autonomous robot

Tese de Doutoramento Programa Doutoral em Engenharia Electrónica e Computadores

Towards "green" smart materials for force and strain sensors: The case of polyaniline

Stress/strain sensors constitute a class of devices with a global ever-growing market thanks to their use in many fields of modern life. They are typically constituted by thin metal foils deposited on flexible supports. However, the low inherent resistivity and limited flexibility of their constituents make them inadequate for several applications, such as measuring large movements in robotic systems and biological tissues. As an alternative to the traditional compounds, in the present work we will show the advantages to employ a smart material, polyaniline (PANI), prepared by an innovative environmentally friendly route, for force/strain sensor applications wherein simple processing, environmental friendliness and sensitivity are particularly required.

Sistema de visão robótico baseado no sensor kinect para orientação no espaço e contorno de obstáculos

Dissertação de mestrado em Engenharia Eletrónica Industrial e Computadores (área de especialização em Robótica)

Caracterização química e física de conformal coatings envolvidos na produção de PCBA's

Dissertação de mestrado integrado em Engenharia de Materiais

Interação multimodal de dispositivos robóticos em ambiente simulado

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

A bio-inspired computational model for motion detection

Tese de Doutoramento (Programa Doutoral em Engenharia Biomédica)