Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

Análise da estrutura de mercado de Resíduos de Equipamentos Elétricos e Eletrónicos: REEE em Portugal

Dissertação de mestrado em Economia Industrial e de Empresa

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Generic system for human-computer gesture interaction

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for humancomputer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.

Bidirectional battery charger with grid-to-vehicle, vehicle-to-grid and vehicle-to-home technologies

This paper presents the development of na on-board bidirectional battery charger for Electric Vehicles (EVs) targeting Grid-to-Vehicle (G2V), Vehicle-to-Grid (V2G), and Vehicle-to-Home (V2H) technologies. During the G2V operation mode the batteries are charged from the power grid with sinusoidal current and unitary power factor. During the V2G operation mode the energy stored in the batteries can be delivered back to the power grid contributing to the power system stability. In the V2H operation mode the energy stored in the batteries can be used to supply home loads during power outages, or to supply loads in places without connection to the power grid. Along the paper the hardware topology of the bidirectional battery charger is presented and the control algorithms are explained. Some considerations about the sizing of the AC side passive filter are taken into account in order to improve the performance in the three operation modes. The adopted topology and control algorithms are accessed through computer simulations and validated by experimental results achieved with a developed laboratory prototype operating in the different scenarios.

Monovarietal extra-virgin olive oil classification: a fusion of human sensory attributes and an electronic tongue

Olive oil quality grading is traditionally assessed by human sensory evaluation of positive and negative attributes (olfactory, gustatory, and final olfactorygustatory sensations). However, it is not guaranteed that trained panelist can correctly classify monovarietal extra-virgin olive oils according to olive cultivar. In this work, the potential application of human (sensory panelists) and artificial (electronic tongue) sensory evaluation of olive oils was studied aiming to discriminate eight single-cultivar extra-virgin olive oils. Linear discriminant, partial least square discriminant, and sparse partial least square discriminant analyses were evaluated. The best predictive classification was obtained using linear discriminant analysis with simulated annealing selection algorithm. A low-level data fusion approach (18 electronic tongue signals and nine sensory attributes) enabled 100 % leave-one-out cross-validation correct classification, improving the discrimination capability of the individual use of sensor profiles or sensory attributes (70 and 57 % leave-one-out correct classifications, respectively). So, human sensory evaluation and electronic tongue analysis may be used as complementary tools allowing successful monovarietal olive oil discrimination.

Mortars with incorporation of PCM based in different binders: mechanical and thermal behavior

Currently we are witnessing a huge concern of society with the parameters of comfort of the buildings and the energetic consumptions. It is known that there is a huge consumption of non-renewable sources of energy. Thus, it is urgent to develop and explore ways to take advantage of renewable sources of energy by improving the energy efficiency of buildings. The mortars with incorporation of phase change materials (PCM) have the ability to regulate the temperature inside buildings, contributing to the thermal comfort and reduction of the use of heating and cooling equipment, using only the energy supplied by the sun. However, the incorporation of phase change materials in mortars modifies its characteristics. The main purpose of this study was mechanical and thermal characterization of mortars with incorporation of PCM in mortars based in different binders. The binders studied were aerial lime, hydraulic lime, gypsum and cement. For each type of binder a reference composition (0% PCM) and a composition with incorporation of 40% of PCM were developed. It was possible to observe that the incorporation of PCM in mortars caused differences in properties such as workability, compressive strength, flexural strength and adhesion, however leads to an improvement of thermal behavior.

Hand gesture recognition for human computer interaction : a comparative study of different image features

Hand gesture recognition for human computer interaction, being a natural way of human computer interaction, is an area of active research in computer vision and machine learning. This is an area with many different possible applications, giving users a simpler and more natural way to communicate with robots/systems interfaces, without the need for extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them to convey information or for device control. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. In this study we try to identify hand features that, isolated, respond better in various situations in human-computer interaction. The extracted features are used to train a set of classifiers with the help of RapidMiner in order to find the best learner. A dataset with our own gesture vocabulary consisted of 10 gestures, recorded from 20 users was created for later processing. Experimental results show that the radial signature and the centroid distance are the features that when used separately obtain better results, with an accuracy of 91% and 90,1% respectively obtained with a Neural Network classifier. These to methods have also the advantage of being simple in terms of computational complexity, which make them good candidates for real-time hand gesture recognition.

Renewable energy system for an isolated micro grid

This paper presents the development of the power electronics needed for the interaction between the electrical generator of a wind turbine and an isolated ac micro grid. In this system there are basically two types of receptors for the energy produced by the wind turbine, which are the loads connected to the isolated micro grid and the batteries used to store energy. There are basically two states in which the system will work. One of the states is when there is enough wind power to supply the loads and the extra energy is used to charge the batteries. The other state is when there is low wind power and the batteries have to compensate the lack of power, so that the isolated micro grid has enough power to supply at least the priority loads. In this paper are presented the hardware and the control algorithm for the developed system. The topology was previously tested in computer simulations, using the software PSIM 9.0, and then validated with the implementation of a laboratory prototype.

Design of assemblies based on polymer-coated quantum dots and organic dye

During last years, photophysical properties of complexes of semiconductor quantum dots (QDs) with organic dyes have attracted increasing interest. The development of different assemblies based on QDs and organic dyes allows to increase the range of QDs applications, which include imaging, biological sensing and electronic devices.1 Some studies demonstrate energy transfer between QDs and organic dye in assemblies.2 However, for electronic devices purposes, a polymeric matrix is required to enhance QDs photostability. Thus, in order to attach the QDs to the polymer surface it is necessary to chemically modify the polymer to induce electronic charges and stabilize the QDs in the polymer. The present work aims to investigate the design of assemblies based on polymer-coated QDs and an integrated acceptor organic dye. Polymethylmethacrylate (PMMA) and polycarbonate (PC) were used as polymeric matrices, and nile red as acceptor. Additionally, a PMMA matrix modified with 2-mercaptoethylamine is used to improve the attachment between both the donor (QDs) and the acceptor (nile red), as well as to induce a covalent bond between the modified PMMA and the QDs. An enhancement of the energy transfer efficiency by using the modified PMMA is expected and the resulting assembly can be applied for energy harvesting.

Monitorização e controlo em ambientes industriais através de rede de sensores sem fios

Dissertação de mestrado integrado em Engenharia de Telecomunicações e Informática

Desenvolvimento de um BMS para aplicações de mobilidade elétrica

Dissertação de mestrado integrado em Engenharia Eletrónica Industrial e de Computadores

Implementação de sistemas de encriptação AES advanced encryption standard em hardware para segurança

Dissertação de mestrado integrado em Engenharia Electrónica Industrial e Computadores