Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

A domain-specific language for ETL patterns specification in Data Warehousing Systems

During the last few years many research efforts have been done to improve the design of ETL (Extract-Transform-Load) systems. ETL systems are considered very time-consuming, error-prone and complex involving several participants from different knowledge domains. ETL processes are one of the most important components of a data warehousing system that are strongly influenced by the complexity of business requirements, their changing and evolution. These aspects influence not only the structure of a data warehouse but also the structures of the data sources involved with. To minimize the negative impact of such variables, we propose the use of ETL patterns to build specific ETL packages. In this paper, we formalize this approach using BPMN (Business Process Modelling Language) for modelling more conceptual ETL workflows, mapping them to real execution primitives through the use of a domain-specific language that allows for the generation of specific instances that can be executed in an ETL commercial tool.

Using a domain-specific language to enrich ETL schemas

Today it is easy to find a lot of tools to define data migration schemas among different types of information systems. Data migration processes use to be implemented on a very diverse range of applications, ranging from conventional operational systems to data warehousing platforms. The implementation of a data migration process often involves a serious planning, considering the development of conceptual migration schemas at early stages. Such schemas help architects and engineers to plan and discuss the most adequate way to migrate data between two different systems. In this paper we present and discuss a way for enriching data migration conceptual schemas in BPMN using a domain-specific language, demonstrating how to convert such enriched schemas to a first correspondent physical representation (a skeleton) in a conventional ETL implementation tool like Kettle.

Cross-language influences on new second language vocabulary acquisition: the effect of learning method

Dissertação de mestrado integrado em Psicologia

Digital manipulatives as scaffolds for preschoolers’ language development

The study reported here aims at contributing to a deeper understanding of the educational possibilities offered by digital manipulatives in preschool contexts. It presents a study carried with a digital manipulative to enhance the development of lexical knowledge and language awareness, which are relevant language abilities for formal literacy learning. The study took place in a Portuguese preschool, with a class of 20 five-year-olds in collaboration with the teacher. The digital manipulative supported the construction of multiple fictional worlds, motivating children's verbal interactions, and the playing of words and sound games, thus contextualizing the learning of an extensive collection of vocabulary and language awareness abilities. The degree of engagement and involvement that the manipulative provided in supporting children’s imaginative play as well as the imitation, in their own play, of the playful pedagogical interventions that the teacher had designed, shows the importance of well- designed materials that support a child-centered learning model. As such, it sustains a discussion on the potential of digital manipulatives to enhance fundamental language development in the preschool years. Further, the study highlights the importance of multidisciplinary teams in the creation of innovative pedagogical materials.

The music of language: an ERP investigation of the effects of musical training on emotional prosody processing

Recent studies have demonstrated the positive effects of musical training on the perception of vocally expressed emotion. This study investigated the effects of musical training on event-related potential (ERP) correlates of emotional prosody processing. Fourteen musicians and fourteen control subjects listened to 228 sentences with neutral semantic content, differing in prosody (one third with neutral, one third with happy and one third with angry intonation), with intelligible semantic content (semantic content condition--SCC) and unintelligible semantic content (pure prosody condition--PPC). Reduced P50 amplitude was found in musicians. A difference between SCC and PPC conditions was found in P50 and N100 amplitude in non-musicians only, and in P200 amplitude in musicians only. Furthermore, musicians were more accurate in recognizing angry prosody in PPC sentences. These findings suggest that auditory expertise characterizing extensive musical training may impact different stages of vocal emotional processing.

Modelling and verifying smell-free architectures with the Archery language

Architectural (bad) smells are design decisions found in software architectures that degrade the ability of systems to evolve. This paper presents an approach to verify that a software architecture is smellfree using the Archery architectural description language. The language provides a core for modelling software architectures and an extension for specifying constraints. The approach consists in precisely specifying architectural smells as constraints, and then verifying that software architectures do not satisfy any of them. The constraint language is based on a propositional modal logic with recursion that includes: a converse operator for relations among architectural concepts, graded modalities for describing the cardinality in such relations, and nominals referencing architectural elements. Four architectural smells illustrate the approach.

Mother's stress, mood and emotional involvement with the infant: 3 months before and 3 months after childbirth

Adverse effects of maternal anxiety and depression are well documented, namely on the foetus/child behaviour and development, but not as much attention has been given to the mother's emotional involvement with the offspring. To study mother's prenatal and postpartum stress, mood and emotional involvement with the infant, the State-Trait Anxiety Inventory, the Edinburgh Postnatal Depression Scale and the Mother-to-Infant Bonding Scale were filled in and cortisol levels were measured, 3 months before and 3 months after childbirth, in a sample of 91 Portuguese women. From pregnancy to the postpartum period, mother's cortisol levels, anxiety and emotional involvement toward the child decrease. No significant change was observed regarding mother's depression. Mother's depression predicted a worse emotional involvement before childbirth, while mother's anxiety predicted a worse emotional involvement with the infant after childbirth. Additionally, pregnant women with a worse emotional involvement with the offspring are at risk of poorer emotional involvement with the infant and higher anxiety and depression at 3 months postpartum. It should be given more attention to mother's poor emotional involvement with the offspring during pregnancy, as it interferes with her emotional involvement with the infant and her psychological adjustment 3 months after childbirth.

Mother-to-infant emotional involvement at birth

Objectives. To study mother-to-infant emotional involvement at birth, namely factors (socio-demographics, previous life events, type of delivery, pain at childbirth, support from partner, infant characteristics, early experiences with the newborn, and mother’s mood) that interfere with the mother’s positive, negative and not clear emotions toward the newborn. Methods. The Bonding Scale (an extended Portuguese version of the ‘New Mother-to-Infant Bonding Scale’) and the Edinburgh Postnatal Depression Scale were administrated during the first after delivery days to 315 mothers recruited at Ju´lio Dinis Maternity Hospital (MJD, Porto, Portugal). Results. A worse emotional involvement with the newborn was observed when the mother was unemployed, unmarried, had less than grade 9, previous obstetrical/psychological problems or was depressed, as well as when the infant was female, had neonatal problems or was admitted in the intensive care unit. Lower total bonding results were significantly predicted when the mother was depressed and had a lower educational level; being depressed, unemployed and single predicted more negative emotions toward the infant as well. No significant differences in the mother-to-infant emotional involvement were obtained for events related to childbirth, such as type of delivery, pain and partner support, or early experiences with the newborn; these events do not predict mother’s bonding results either. Conclusion. The study results support the need for screening and supporting depressed, unemployed and single mothers, in order to prevent bonding difficulties with the newborn at birth.

Postpartum Depression: Is it a condition affecting the mother-infant interaction and the development of the child across the first year of life?

Noting that maternal depression is common during a baby's first year, this study examined the interaction of depressed and non-depressed mother-child dyads. A sample of 26 first-time mothers with postpartum depression at the third month after birth and their 3-month-old infants was compared to a sample of 25 first-time mothers with no postpartum depression at the third month after birth and their 3-month-old infants. The observations were repeated at 6 months and again at 12 months postpartum. The samples were compared for differences in mother interaction behavior, mother's infant care, mother's concern with the baby, infant behavioral difficulties, infant mental and motor development, and infant behavior with the observer. Among the findings are the following: (1) depressed mothers' interaction behavior and care of their infants are less adequate than the non-depressed mothers' interaction behavior and care of their infants at 3, 6, and 12 months postpartum; (2) infants' interaction behaviors during feeding and face-to-face interaction with depressed mothers are less adequate than infants' interactions with non-depressed mothers at 3, 6, and 12 months postpartum; (3) mother-infant interactions are less adequate in the depressed mother dyads than the non-depressed dyads at 3, 6, and 12 months postpartum; (4) depressed mothers are less concerned about their infants than non-depressed mothers at 3, 6, and 12 months postpartum; (5) infants of depressed mothers have more behavioral difficulties at 3, 6, and 12 months postpartum than infants of non-depressed mothers; (6) infants of depressed mothers had lower mental and motor development rates at 6 and 12 months postpartum than infants of non-depressed mothers; and (7) infants of non-depressed mothers behaved in a more positive way with the observer than the infants of depressed mothers. (AS)