Hand gesture recognition system based in computer vision and machine learning : applications on human-machine interaction

Sendo uma forma natural de interação homem-máquina, o reconhecimento de gestos implica uma forte componente de investigação em áreas como a visão por computador e a aprendizagem computacional. O reconhecimento gestual é uma área com aplicações muito diversas, fornecendo aos utilizadores uma forma mais natural e mais simples de comunicar com sistemas baseados em computador, sem a necessidade de utilização de dispositivos extras. Assim, o objectivo principal da investigação na área de reconhecimento de gestos aplicada à interacção homemmáquina é o da criação de sistemas, que possam identificar gestos específicos e usálos para transmitir informações ou para controlar dispositivos. Para isso as interfaces baseados em visão para o reconhecimento de gestos, necessitam de detectar a mão de forma rápida e robusta e de serem capazes de efetuar o reconhecimento de gestos em tempo real. Hoje em dia, os sistemas de reconhecimento de gestos baseados em visão são capazes de trabalhar com soluções específicas, construídos para resolver um determinado problema e configurados para trabalhar de uma forma particular. Este projeto de investigação estudou e implementou soluções, suficientemente genéricas, com o recurso a algoritmos de aprendizagem computacional, permitindo a sua aplicação num conjunto alargado de sistemas de interface homem-máquina, para reconhecimento de gestos em tempo real. A solução proposta, Gesture Learning Module Architecture (GeLMA), permite de forma simples definir um conjunto de comandos que pode ser baseado em gestos estáticos e dinâmicos e que pode ser facilmente integrado e configurado para ser utilizado numa série de aplicações. É um sistema de baixo custo e fácil de treinar e usar, e uma vez que é construído unicamente com bibliotecas de código. As experiências realizadas permitiram mostrar que o sistema atingiu uma precisão de 99,2% em termos de reconhecimento de gestos estáticos e uma precisão média de 93,7% em termos de reconhecimento de gestos dinâmicos. Para validar a solução proposta, foram implementados dois sistemas completos. O primeiro é um sistema em tempo real capaz de ajudar um árbitro a arbitrar um jogo de futebol robótico. A solução proposta combina um sistema de reconhecimento de gestos baseada em visão com a definição de uma linguagem formal, o CommLang Referee, à qual demos a designação de Referee Command Language Interface System (ReCLIS). O sistema identifica os comandos baseados num conjunto de gestos estáticos e dinâmicos executados pelo árbitro, sendo este posteriormente enviado para um interface de computador que transmite a respectiva informação para os robôs. O segundo é um sistema em tempo real capaz de interpretar um subconjunto da Linguagem Gestual Portuguesa. As experiências demonstraram que o sistema foi capaz de reconhecer as vogais em tempo real de forma fiável. Embora a solução implementada apenas tenha sido treinada para reconhecer as cinco vogais, o sistema é facilmente extensível para reconhecer o resto do alfabeto. As experiências também permitiram mostrar que a base dos sistemas de interação baseados em visão pode ser a mesma para todas as aplicações e, deste modo facilitar a sua implementação. A solução proposta tem ainda a vantagem de ser suficientemente genérica e uma base sólida para o desenvolvimento de sistemas baseados em reconhecimento gestual que podem ser facilmente integrados com qualquer aplicação de interface homem-máquina. A linguagem formal de definição da interface pode ser redefinida e o sistema pode ser facilmente configurado e treinado com um conjunto de gestos diferentes de forma a serem integrados na solução final.

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

EPS (Electronic Portfolio System)

In 2001, before the term “e-portfolio” was common parlance there was a perceived need to enable teachers in training to save, store, present and archive their electronically-based work so that it could be assessed by tutors. A Teacher Training Agency (TTA) Grant was used to design and implement an online system for trainee teachers to save evidence of their activities called Electronic Portfolio System (EPS). Over the years, its use and value has changed and, through the support of the Teacher Development Agency for schools (TDA), the system will continue to develop. The features of the system have grown organically because of technology changes and by tutors and mentors identifying affordances. This presentation will identify the principles and how they guide the development process. It will give an opportunity to elucidate the next stages in e-portfolio developments.

Seminar: Symbol Management systems to support democracy in development of an online Symbol Dictionary

Abstract This seminar will introduce an initial year of research exploring participation in the development of a bilingual symbol dictionary. Symbols can be a communication and literacy ‘lifeline’ for those unable to communicate through speech or writing. We will discuss how an online system has been built to overcome language, cultural and literacy skill issues for a country where 86% are expatriates but the target clients are Arabic born individuals with speech and language impairments. The symbols in use at present are inappropriate and yet there is no democratic way of providing a ‘user voice’ for making choices, let alone easy mechanisms for adapting and sharing newly developed symbols across the nation or extended Arabic world. This project aims to change this situation. Having sourced a series of symbols that could be adapted to suit user’s needs, the team needed to encourage those users, their carers and therapists to vote on whether the symbols would be appropriate and work with those already in use. The first prototype was developed and piloted during the WAISfest in 2013. The second phase needs further voting on the most suitably adapted symbols for use when communicating with others. There is a requirement to have mechanisms for evaluating the outcome of the votes, where symbols fail to represent accurate meanings, have inappropriate colours, representations and actions etc. There also remains the need to collect both quantitative and qualitative data. Not easy in a climate of acceptance of the expert view, a culture where to be critical can be a problem and time is not of the essence.

A system for classification of time-series data from industrial non-destructive device

This work proposes a system for classification of industrial steel pieces by means of magnetic nondestructive device. The proposed classification system presents two main stages, online system stage and off-line system stage. In online stage, the system classifies inputs and saves misclassification information in order to perform posterior analyses. In the off-line optimization stage, the topology of a Probabilistic Neural Network is optimized by a Feature Selection algorithm combined with the Probabilistic Neural Network to increase the classification rate. The proposed Feature Selection algorithm searches for the signal spectrogram by combining three basic elements: a Sequential Forward Selection algorithm, a Feature Cluster Grow algorithm with classification rate gradient analysis and a Sequential Backward Selection. Also, a trash-data recycling algorithm is proposed to obtain the optimal feedback samples selected from the misclassified ones.

Evaluación del funcionamiento de sistemas de análisis digital de imágenes - wipfrag (edición manual) y split online

En minería, la estimación de la curva granulométrica del escombro de voladura es importante para evaluar el diseño, ejecución y optimización de la misma. Para ello, actualmente se usan sistemas digitales de fotografías que obtienen dicha curva a partir de imágenes tomadas por una cámara. En este proyecto se ha analizado la fragmentación de seis voladuras realizadas en el año 2012 en la cantera “El Aljibe” situada en el término municipal de Almonacid de Toledo con un sistema automático en línea (Split Online) y con un software de otra compañía (WipFrag) que permite la edición manual de las imágenes. Han sido analizadas 120 imágenes de seis voladuras, elegidas aleatoriamente. Tras el estudio granulométrico, se observa que las curvas granulométricas obtenidas con ambos sistemas, estadísticamente, no son la misma en la mayor parte de la curva, por tanto, se analiza una posible relación entre los tamaños característicos X50 y X80, llegando a la conclusión de que ninguno de los sistemas es totalmente fiable, y es necesario calibrar los sistemas con datos de fragmentación reales obtenidos por medio de básculas. Abstract In mining, the estimate of the granulometric curve blasting debris is very important to evaluate the design, implementation and optimization of it. Currently, for the obtaining of this curves are used digital system of pictures taken by a camera. In this project, the fragmentation of six rock blasting were analyzed. The rock blastings are executed in 2012 in the quarry “El Aljibe” located in Almonacid de Toledo, with a automatic online system (Split Online) and a manual editing software (WipFrag). 120 randomly selected pictures have been analyzed. After the granulometric study, it appears that the size distribution curves obtained with both systems, statistically, are not the same, then, a possible relationship between the feature sizes X50 and X80 is analyzed, concluding that none of the systems is fully reliable, and systems must be calibrated with real data fragmentation obtained from data scales.

Application of an online-biomass sensor in an optical multisensory platform prototype for growth monitoring of biotechnical relevant microorganism and cell lines in single-use shake flasks

In the context of this work we evaluated a multisensory, noninvasive prototype platform for shake flask cultivations by monitoring three basic parameters (pH, pO2 and biomass). The focus lies on the evaluation of the biomass sensor based on backward light scattering. The application spectrum was expanded to four new organisms in addition to E. coli K12 and S. cerevisiae [1]. It could be shown that the sensor is appropriate for a wide range of standard microorganisms, e.g., L. zeae, K. pastoris, A. niger and CHO-K1. The biomass sensor signal could successfully be correlated and calibrated with well-known measurement methods like OD600, cell dry weight (CDW) and cell concentration. Logarithmic and Bleasdale-Nelder derived functions were adequate for data fitting. Measurements at low cell concentrations proved to be critical in terms of a high signal to noise ratio, but the integration of a custom made light shade in the shake flask improved these measurements significantly. This sensor based measurement method has a high potential to initiate a new generation of online bioprocess monitoring. Metabolic studies will particularly benefit from the multisensory data acquisition. The sensor is already used in labscale experiments for shake flask cultivations.

External quality assessment in the voluntary counseling and testing centers in the Brazilian Amazon using dried tube specimens: results of an effectiveness evaluation

INTRODUCTION : In 2011, the Brazilian Ministry of Health rolled out a program for the external quality assessment of rapid human immunodeficiency virus (HIV) tests using the dried tube specimen (DTS) method (EQA-RT/DTS-HIV). Our objective was to evaluate the implementation of this program at 71 voluntary counseling and testing centers (VCTCs) in the Brazilian Legal Amazonian area one year after its introduction. METHODS : Quantitative and qualitative study that analyzed secondary data and interviews with healthcare workers (HCWs) (n=39) and VCTC coordinators (n=32) were performed. The assessment used 18 key indicators to evaluate the three dimensions of the program's logical framework: structure, process, and result. Each indicator was scored from 1-4, and the aggregate results corresponding to the dimensions were expressed as proportions. The results were compared to the perceptions of the HCWs and coordinators regarding the EQA-RT/DTS-HIV program. RESULTS: The aggregate scores for the three dimensions of structure, process, and result were 91.7%, 78.6%, and 95%, respectively. The lowest score in each dimension corresponded to a different indicator: access to Quali-TR online system 39% (structure), registration in Quali-TR online system 38.7% (process), and VCTC completed the full process in the program's first round 63.4% (result). Approximately 36% of the HCWs and 52% of the coordinators reported enhanced trust in the program for its rapid HIV testing performance. CONCLUSIONS: All three program dimensions exhibited satisfactory results (>75%). Nevertheless, the study findings highlight the need to improve certain program components. Additionally, long-term follow-ups is needed to provide a more thorough picture of the process for external quality assessment.

Sistema de predição de avarias em máquinas de unidades fabris globalmente dispersas

Tese de Doutoramento Ramo Engenharia Industrial e de Sistemas

Eina de col·laboració en grup

Aquest projecte consisteix en el disseny i implementació de un sistema en línia que permet la creació i organització de grups de treball els usuaris dels quals podran comunicar-se i compartir informació.

Hitsauksen laadunhallinta offshore-teollisuuden alihankintakonepajassa

Diplomityön tavoitteena oli kuvata hitsauksen laadunhallintaa sekä hitsauksen laadunhallintajärjestelmän rakentamista offshore-teollisuuden konepajaan standardin ISO 3834-2 mukaisesti. Diplomityö koostuu teoriaosasta sekä käytännön osasta. Teoriaosa käsittelee laadunhallintaa yleisesti, minkä jälkeen se tarkentuu käsittelemään hitsauksen laadunhallintaa. Esimerkkinä on esitetty standardin NORSOK M-101 mukaisia laatuvaatimuksia. Laadunhallintaan liittyy oleellisesti myös hitsausliitoksen NDT-tarkastaminen, josta on käsitelty silmämääräinen tarkastus ja ultraäänitarkastus. Teoriaosan lopussa on käsitelty hitsauksen työturvallisuutta sekä hitsauksen laadunvalvontaa online-seurantajärjestelmällä. Käytännön osassa kerrotaan hitsauksen laatujärjestelmän rakentamisesta Meteco Oy:n Kuhmoisten tehtaalla. Yrityksen nykytilanne kartoitettiin, minkä jälkeen ryhdyttiin rakentamaan hitsauksen laadunhallintajärjestelmää. Tuloksena syntyi hitsauksen laatukäsikirja, joka pohjautuu standardiin ISO 3834-2. Käytännön osuudessa on kerrottu myös online-seurantajärjestelmän soveltuvuudesta alihankintakonepajan käyttöön. Hitsauksen laatujärjestelmän rakentamiseen kuului henkilöstön koulutustilaisuudet, joissa informoitiin edellä mainitun laatukäsikirjaan kuuluvista asioista. Tästä syntyi yritykseen myös koulutusohjelmasuunnitelma, joka antaa valmiudet yrityksen henkilökunnalle kehittyä työssään ja näin tehostaa toimintaa.

Search Engine Optimization Methods & Search Engine Indexing for CMS Applications

Search engine optimization & marketing is a set of processes widely used on websites to improve search engine rankings which generate quality web traffic and increase ROI. Content is the most important part of any website. CMS web development is now become very essential for most of organizations and online businesses to develop their online system and websites. Every online business using a CMS wants to get users (customers) to make profit and ROI. This thesis comprises a brief study of existing SEO methods, tools and techniques and how they can be implemented to optimize a content base website. In results, the study provides recommendations about how to use SEO methods; tools and techniques to optimize CMS based websites on major search engines. This study compares popular CMS systems like Drupal, WordPress and Joomla SEO features and how implementing SEO can be improved on these CMS systems. Having knowledge of search engine indexing and search engine working is essential for a successful SEO campaign. This work is a complete guideline for web developers or SEO experts who want to optimize a CMS based website on all major search engines.

Mining association rules in folksonomies

Social bookmark tools are rapidly emerging on the Web. In such systems users are setting up lightweight conceptual structures called folksonomies. These systems provide currently relatively few structure. We discuss in this paper, how association rule mining can be adopted to analyze and structure folksonomies, and how the results can be used for ontology learning and supporting emergent semantics. We demonstrate our approach on a large scale dataset stemming from an online system.

Subcutaneous phaeohyphomycosis caused by Phaeoacremonium parasiticum in a renal transplant patient

A 49-year-old renally transplanted man, under a five-year course of immunosuppressive therapy with prednisone and cyclosporine A, experienced a subcutaneous phaeohyphomycosis caused by Phaeoacremonium parasiticum. The clinical presentation consisted of impressive, large, inflammatory and draining cystic tumors on the left foot that had been present for one year. A significant improvement was obtained with itraconazole plus intralesional injection with amphotericin B. Drug interaction was observed between itraconazole and cyclosporine A causing a severe hypertensive crisis and requiring a temporary sharp reduction in cyclosporine administration. Subcutaneous phaeohyphomycosis caused by P. parasiticum is uncommon among major organ transplant patients but several cases have previously been published and some patterns are emerging, e. g., limbs are generally involved but no known traumatic event has preceded lesion development. The identification of the case isolate was confirmed using a recently published online system based in part on beta-tubulin sequence comparison.