Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Generic system for human-computer gesture interaction

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for humancomputer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of vision-based interaction systems can be the same for all applications and thus facilitate the implementation. In order to test the proposed solutions, three prototypes were implemented. For hand posture recognition, a SVM model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications.

Using students’ learning style to create effective learning groups in MCSCL environments

Students have different ways for learning and processing information. Some students prefer learning through seeing while others prefer learning through listening; some students prefer doing activities while other prefer reflecting.Some students reason logically, while others reason intuitively, etc. Identifying the learning style of each student, and providing learning content based on these styles represents a good method to enhance the learning quality. However, there are no efforts onhow to detect the students’ learning styles in mobile computer supported collaborative learning (MCSCL) environments. We present in this paper new ways for automatically detecting the learning styles of students in MCSCL environments based on the learning style model of Felder-Silverman. The identified learning styles of students could be then stored and used at anytime toassign each one of them to his/her appropriate learning group.

Sistema de apoio à aprendizagem na área da discalculia em menores

Dissertação de mestrado integrado em Engenharia Biomédica (área de especialização em Informática Médica)

VET: a strategic approach for economic, organisational and personal development in EU countries

Vocational Education and Training (VET) is a continuous long-term process of economic, organisational and personal development. It envisions the construction of dynamic skills to improve performance, productivity and organisational, personal and social development. This article focuses on generating skills. It frames training as a process of work-linked training and as a primary source for generating skills whilst seeking to boost creativity. It sheds light upon the discussion pertaining to learning transfer as a necessary condition to structure performance and competitiveness. It highlights the Learning Transfer System Inventory (LTSI), because it allows to measure the effectiveness of training and it identifies the organisations' weaknesses. The data used were collected from the Eurostat Database.

Hand gesture recognition system based in computer vision and machine learning

"Lecture notes in computational vision and biomechanics series, ISSN 2212-9391, vol. 19"

Hand gesture recognition system based in computer vision and machine learning: Applications on human-machine interaction

Tese de Doutoramento em Engenharia de Eletrónica e de Computadores

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

A proactive intelligent decision support system for predicting the popularity of online news

"Lecture notes in computer science series, ISSN 0302-9743, vol. 9273"

Detection of dangerous situations using a smart internet of things system

The Internet of Things (IoT) is a concept that can foster the emergence of innovative applications. In order to minimize parents’s concerns about their children’s safety, this paper presents the design of a smart Internet of Things system for identifying dangerous situations. The system will be based on real time collection and analysis of physiological signals monitored by non-invasive and non-intrusive sensors, Frequency IDentification (RFID) tags and a Global Positioning System (GPS) to determine when a child is in danger. The assumption of a state of danger is made taking into account the validation of a certain number of biometric reactions to some specific situations and according to a self-learning algorithm developed for this architecture. The results of the analysis of data collected and the location of the child will be able in real time to child’s care holders in a web application.

Interaction paradigms versus age-related user profiles: an evaluation on content selection

Novel input modalities such as touch, tangibles or gestures try to exploit human's innate skills rather than imposing new learning processes. However, despite the recent boom of different natural interaction paradigms, it hasn't been systematically evaluated how these interfaces influence a user's performance or whether each interface could be more or less appropriate when it comes to: 1) different age groups; and 2) different basic operations, as data selection, insertion or manipulation. This work presents the first step of an exploratory evaluation about whether or not the users' performance is indeed influenced by the different interfaces. The key point is to understand how different interaction paradigms affect specific target-audiences (children, adults and older adults) when dealing with a selection task. 60 participants took part in this study to assess how different interfaces may influence the interaction of specific groups of users with regard to their age. Four input modalities were used to perform a selection task and the methodology was based on usability testing (speed, accuracy and user preference). The study suggests a statistically significant difference between mean selection times for each group of users, and also raises new issues regarding the “old” mouse input versus the “new” input modalities.

Methanogens, sulphate and heavy metals: a complex system

Anaerobic digestion (AD) is a well-established technology used for the treatment of wastes and wastewaters with high organic content. During AD organic matter is converted stepwise to methane-containing biogasa renewable energy carrier. Methane production occurs in the last AD step and relies on methanogens, which are rather sensitive to some contaminants commonly found in wastewaters (e.g. heavy metals), or easily outcompeted by other groups of microorganisms (e.g. sulphate reducing bacteria, SRB). This review gives an overview of previous research and pilot-scale studies that shed some light on the effects of sulphate and heavy metals on methanogenesis. Despite the numerous studies on this subject, comparison is not always possible due to differences in the experimental conditions used and parameters explained. An overview of the possible benefits of methanogens and SRB co-habitation is also covered. Small amounts of sulphide produced by SRB can precipitate with metals, neutralising the negative effects of sulphide accumulation and free heavy metals on methanogenesis. Knowledge on how to untangle and balance sulphate reduction and methanogenesis is crucial to take advantage of the potential for the utilisation of biogenic sulphide as a metal detoxification agent with minimal loss in methane production in anaerobic digesters.

Um modelo para a gestão do conhecimento organizacional no contexto dos serviços partilhados com recurso à utilização do e-Learning

Tese de Doutoramento em Tecnologias e Sistemas de Informação

Learning joint representations for order and timing of perceptual-motor sequences: a dynamic neural field approach

Many of our everyday tasks require the control of the serial order and the timing of component actions. Using the dynamic neural field (DNF) framework, we address the learning of representations that support the performance of precisely time action sequences. In continuation of previous modeling work and robotics implementations, we ask specifically the question how feedback about executed actions might be used by the learning system to fine tune a joint memory representation of the ordinal and the temporal structure which has been initially acquired by observation. The perceptual memory is represented by a self-stabilized, multi-bump activity pattern of neurons encoding instances of a sensory event (e.g., color, position or pitch) which guides sequence learning. The strength of the population representation of each event is a function of elapsed time since sequence onset. We propose and test in simulations a simple learning rule that detects a mismatch between the expected and realized timing of events and adapts the activation strengths in order to compensate for the movement time needed to achieve the desired effect. The simulation results show that the effector-specific memory representation can be robustly recalled. We discuss the impact of the fast, activation-based learning that the DNF framework provides for robotics applications.

Off-line simulation inspires insight: a neurodynamics approach to efficient robot task learning

There is currently an increasing demand for robots able to acquire the sequential organization of tasks from social learning interactions with ordinary people. Interactive learning-by-demonstration and communication is a promising research topic in current robotics research. However, the efficient acquisition of generalized task representations that allow the robot to adapt to different users and contexts is a major challenge. In this paper, we present a dynamic neural field (DNF) model that is inspired by the hypothesis that the nervous system uses the off-line re-activation of initial memory traces to incrementally incorporate new information into structured knowledge. To achieve this, the model combines fast activation-based learning to robustly represent sequential information from single task demonstrations with slower, weight-based learning during internal simulations to establish longer-term associations between neural populations representing individual subtasks. The efficiency of the learning process is tested in an assembly paradigm in which the humanoid robot ARoS learns to construct a toy vehicle from its parts. User demonstrations with different serial orders together with the correction of initial prediction errors allow the robot to acquire generalized task knowledge about possible serial orders and the longer term dependencies between subgoals in very few social learning interactions. This success is shown in a joint action scenario in which ARoS uses the newly acquired assembly plan to construct the toy together with a human partner.