Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

The human eye as human-machine interface

Eye tracking as an interface to operate a computer is under research for a while and new systems are still being developed nowadays that provide some encouragement to those bound to illnesses that incapacitates them to use any other form of interaction with a computer. Although using computer vision processing and a camera, these systems are usually based on head mount technology being considered a contact type system. This paper describes the implementation of a human-computer interface based on a fully non-contact eye tracking vision system in order to allow people with tetraplegia to interface with a computer. As an assistive technology, a graphical user interface with special features was developed including a virtual keyboard to allow user communication, fast access to pre-stored phrases and multimedia and even internet browsing. This system was developed with the focus on low cost, user friendly functionality and user independency and autonomy.

Improving quality of medical service with mobile health software

An increasing number of m-Health applications are being developed benefiting health service delivery. In this paper, a new methodology based on the principle of calm computing applied to diagnostic and therapeutic procedure reporting is proposed. A mobile application was designed for the physicians of one of the Portuguese major hospitals, which takes advantage of a multi-agent interoperability platform, the Agency for the Integration, Diffusion and Archive (AIDA). This application allows the visualization of inpatients and outpatients medical reports in a quicker and safer manner, in addition to offer a remote access to information. This project shows the advantages in the use of mobile software in a medical environment but the first step is always to build or use an interoperability platform, flexible, adaptable and pervasive. The platform offers a comprehensive set of services that restricts the development of mobile software almost exclusively to the mobile user interface design. The technology was tested and assessed in a real context by intensivists.

User expectations for media sharing practices in open display networks

Open Display Networks have the potential to allow many content creators to publish their media to an open-ended set of screen displays. However, this raises the issue of how to match that content to the right displays. In this study, we aim to understand how the perceived utility of particular media sharing scenarios is affected by three independent variables, more specifically: (a) the locativeness of the content being shared; (b) how personal that content is and (c) the scope in which it is being shared. To assess these effects, we composed a set of 24 media sharing scenarios embedded with different treatments of our three independent variables. We then asked 100 participants to express their perception of the relevance of those scenarios. The results suggest a clear preference for scenarios where content is both local and directly related to the person that is publishing it. This is in stark contrast to the types of content that are commonly found in public displays, and confirms the opportunity that open displays networks may represent a new media for self-expression. This novel understanding may inform the design of new publication paradigms that will enable people to share media across the display networks.

MixAR mobile prototype: visualizing virtually reconstructed ancient structures in situ

Archeology and related areas have a special interest on cultural heritage sites since they provide valuable information about past civilizations. However, the ancient buildings present in these sites are commonly found in an advanced state of degradation which difficult the professional/expert analysis. Virtual reconstructions of such buildings aim to provide a digital insight of how these historical places could have been in ancient times. Moreover, the visualization of such models has been explored by some Augmented Reality (AR) systems capable of providing support to experts. Their compelling and appealing environments have also been applied to promote the social and cultural participation of general public. The existing AR solutions regarding this thematic rarely explore the potential of realism, due to the following lacks: the exploration of mixed environments is usually only supported for indoors or outdoors, not both in the same system; the adaptation of the illumination conditions to the reconstructed structures is rarely addressed causing a decrease of credibility. MixAR [1] is a system concerned with those challenges, aiming to provide the visualization of virtual buildings augmented upon real ruins, allowing soft transitions among its interiors and exteriors and using relighting techniques for a faithful interior illumination, while the user freely moves in a given cultural heritage site, carrying a mobile unit. Regarding the focus of this paper, we intend to report the current state of MixAR mobile unit prototype, which allows visualizing virtual buildings – properly aligned with real-world structures – based on user's location, during outdoor navigation. In order to evaluate the prototype performance, a set of tests were made using virtual models with different complexities.

Reverse engineering of web applications

The MAP-i Doctoral Program of the Universities of Minho, Aveiro and Porto

Monitorização e controlo em ambientes industriais através de rede de sensores sem fios

Dissertação de mestrado integrado em Engenharia de Telecomunicações e Informática

Desenvolvimento de um filtro ativo paralelo monofásico compacto e didático utilizando MOSFETs

Dissertação de mestrado integrado em Engenharia Eletrónica Industrial e Computadores

Gestão e instalação de redes de sensores sem fios (domótica)

Dissertação de mestrado integrado em Engenharia Eletrónica Industrial e Computadores

Aplicação web para configuração e acesso a câmaras ONVIF

Dissertação de mestrado integrado em Engenharia Eletrónica Industrial e Computadores

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Generic system for human-computer gesture interaction

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for humancomputer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.