About using serious games to teach (Portuguese) sign language

Sign language is the form of communication used by Deaf people, which, in most cases have been learned since childhood. The problem arises when a non-Deaf tries to contact with a Deaf. For example, when non-Deaf parents try to communicate with their Deaf child. In most cases, this situation tends to happen when the parents did not have time to properly learn sign language. This dissertation proposes the teaching of sign language through the usage of serious games. Currently, similar solutions to this proposal do exist, however, those solutions are scarce and limited. For this reason, the proposed solution is composed of a natural user interface that is intended to create a new concept on this field. The validation of this work, consisted on the implementation of a serious game prototype, which can be used as a source for learning (Portuguese) sign language. On this validation, it was first implemented a module responsible for recognizing sign language. This first stage, allowed the increase of interaction and the construction of an algorithm capable of accurately recognizing sign language. On a second stage of the validation, the proposal was studied so that the pros and cons can be determined and considered on future works.

Vision-based portuguese sign language recognition system

Vision-based hand gesture recognition is an area of active current research in computer vision and machine learning. Being a natural way of human interaction, it is an area where many researchers are working on, with the goal of making human computer interaction (HCI) easier and natural, without the need for any extra devices. So, the primary goal of gesture recognition research is to create systems, which can identify specific human gestures and use them, for example, to convey information. For that, vision-based hand gesture interfaces require fast and extremely robust hand detection, and gesture recognition in real time. Hand gestures are a powerful human communication modality with lots of potential applications and in this context we have sign language recognition, the communication method of deaf people. Sign lan- guages are not standard and universal and the grammars differ from country to coun- try. In this paper, a real-time system able to interpret the Portuguese Sign Language is presented and described. Experiments showed that the system was able to reliably recognize the vowels in real-time, with an accuracy of 99.4% with one dataset of fea- tures and an accuracy of 99.6% with a second dataset of features. Although the im- plemented solution was only trained to recognize the vowels, it is easily extended to recognize the rest of the alphabet, being a solid foundation for the development of any vision-based sign language recognition user interface system.

"Knock, knock! Is it english or portuguese sign language?"

A presente investigação mostra a importância do contacto de crianças muito jovens com línguas estrangeiras. Este trabalho concentra-se na tentativa de investigar, numa abordagem plurilingue, com enfoque para a Língua Inglesa e a Língua Gestual Portuguesa, a sensibilização de um grupo de alunos do 1º Ciclo do Ensino Básico para uma língua diferente da sua língua materna. Nesta pesquisa, adotou-se uma postura de investigação-ação, apoiando-se com grande particularidade numa metodologia qualitativa e com menor relevância numa metodologia quantitativa, onde os alunos, através das várias atividades que desenvolveram foram adquirindo diferentes competências nas duas línguas. Isto permitiu aos alunos despertarem todas as suas potencialidades para a aprendizagem destas duas línguas (Língua Inglesa e Língua Gestual Portuguesa), tendo como ponto de partida a sua sensibilização e a aprendizagem de alguns vocábulos. Acreditamos que esta abordagem plurilíngue poderá auxiliar os alunos no desenvolvimento de habilidades linguísticas, cognitivas e pessoais tais como: a intercompreensão, o conhecimento de características específicas de diferentes línguas existentes em seu redor, a comparação linguística entre elas, a sua compreensão lexical, e por fim a competência em relacionar as línguas a culturas, e acima de tudo, o respeito e valorização da diversidade linguística e cultural. Foram utilizadas nas aulas atividades de nível de compreensão e produção oral, num processo de sensibilização e aprendizagem de alguns vocábulos destas línguas, sendo que os resultados foram posteriormente analisados, através de grelhas de observação das atividades, de dois inquéritos por questionário e fotos. Das observações e conclusões retiradas desta análise, confirmou-se que a sensibilização quanto à Língua Inglesa assim como quanto à Língua Gestual Portuguesa promove o desenvolvimento da criança, assim como a valorização da respetiva diversidade linguística e cultural.

Generic system for human-computer gesture interaction: applications on sign language recognition and robotic soccer refereeing

Hand gestures are a powerful way for human communication, with lots of potential applications in the area of human computer interaction. Vision-based hand gesture recognition techniques have many proven advantages compared with traditional devices, giving users a simpler and more natural way to communicate with electronic devices. This work proposes a generic system architecture based in computer vision and machine learning, able to be used with any interface for human-computer interaction. The proposed solution is mainly composed of three modules: a pre-processing and hand segmentation module, a static gesture interface module and a dynamic gesture interface module. The experiments showed that the core of visionbased interaction systems could be the same for all applications and thus facilitate the implementation. For hand posture recognition, a SVM (Support Vector Machine) model was trained and used, able to achieve a final accuracy of 99.4%. For dynamic gestures, an HMM (Hidden Markov Model) model was trained for each gesture that the system could recognize with a final average accuracy of 93.7%. The proposed solution as the advantage of being generic enough with the trained models able to work in real-time, allowing its application in a wide range of human-machine applications. To validate the proposed framework two applications were implemented. The first one is a real-time system able to interpret the Portuguese Sign Language. The second one is an online system able to help a robotic soccer game referee judge a game in real time.

Synchronizing Brazilian Sign Language (LIBRAS) on videos for Open Digital TV with educational purposes

The TV is a potential midia of communication that affects all social classes and it is available in 98% of Brazilian homes. It has been used as a distributor of educational materials since the 1950s. By 2016 the Open Digital TV (ODTV) in Brazil might cover the entire national territory, replacing the TV with analog signals. Concerns about accessibility for People with Special Needs (PSN) in that media have become more intense since the 1990s worldwide. In Brazil, it is estimated 24.6 million of PSN, 23% of them having some type of hearing loss. Of these, 2.9% are reported as deafs. Brazilian Sign Language (LIBRAS) is considered the first literacy language for deaf people in Brazil. In this context, this paper presents a proposal to facilitate the generation of educational content for ODTV based on two components. One is called SynchrLIBRAS and allows subtitles synchronization in Portuguese and a LIBRAS translator window of videos downloaded from the Web. The second component allows the visualization of this content through the Brazilian System of Digital TV and IPTV - environments that implement the middleware Ginga-NCL. The main focus of this paper is the presentation of the first component: SynchrLIBRAS. This proposal has educational purposes, contributing to teach LIBRAS to people who may collaborate with social inclusion of the deaf people.

Sign language in Landau-Kleffner syndrome.

This article reviews the history of sign language (SL) and the rationale for its use in children with profound auditory agnosia due to Landau-Kleffner syndrome (LKS), illustrated by studies of children and adults followed for many years and rare cases from the literature. The reasons that SL was successful and brought some children out of isolation while it could not be implemented in others are discussed. The nowadays earlier recognition and treatment of LKS and better awareness of the crucial need to maintain communication have certainly improved the outcome of affected children. Alternatives to oral language, even for less severe cases, are increasingly accepted. SL can be learned at different ages with a clear benefit, but the ambivalence of the patients and their families with the world and culture of the deaf may sometimes explain its refusal or limited acceptance. There are no data to support the fear that SL learning may delay or prevent oral language recovery in children with LKS. On the contrary, SL may even facilitate this recovery by stimulating functionally connected core language networks and by helping speech therapy and auditory training.

The sight of silence : the conceptualisation of silence in the visual cultures of sign language communities

Deaf people are perceived by hearing people as living in a silent world. Yet, silence cannot exist without sound, so if sound is not heard, can there be silence? From a linguistic point of view silence is the absence of, or intermission in, communication. Silence can be communicative or noncommunicative. Thus, silence must exist in sign languages as well. Sign languages are based on visual perception and production through movement and sight. Silence must, therefore, be visually perceptible; and, if there is such a thing as visual silence, how does it look? The paper will analyse the topic of silence from a Deaf perspective. The main aspects to be explored are the perception and evaluation of acoustic noise and silence by Deaf people; the conceptualisation of silence in visual languages, such as sign languages; the qualities of visual silence; the meaning of silence as absence of communication (particularly between hearing and Deaf people); social rules for silence; and silencing strategies.

TR-2015001: A Survey and Critique of Facial Expression Synthesis in Sign Language Animation

Sign language animations can lead to better accessibility of information and services for people who are deaf and have low literacy skills in spoken/written languages. Due to the distinct word-order, syntax, and lexicon of the sign language from the spoken/written language, many deaf people find it difficult to comprehend the text on a computer screen or captions on a television. Animated characters performing sign language in a comprehensible way could make this information accessible. Facial expressions and other non-manual components play an important role in the naturalness and understandability of these animations. Their coordination to the manual signs is crucial for the interpretation of the signed message. Software to advance the support of facial expressions in generation of sign language animation could make this technology more acceptable for deaf people. In this survey, we discuss the challenges in facial expression synthesis and we compare and critique the state of the art projects on generating facial expressions in sign language animations. Beginning with an overview of facial expressions linguistics, sign language animation technologies, and some background on animating facial expressions, a discussion of the search strategy and criteria used to select the five projects that are the primary focus of this survey follows. This survey continues on to introduce the work from the five projects under consideration. Their contributions are compared in terms of support for specific sign language, categories of facial expressions investigated, focus range in the animation generation, use of annotated corpora, input data or hypothesis for their approach, and other factors. Strengths and drawbacks of individual projects are identified in the perspectives above. This survey concludes with our current research focus in this area and future prospects.

Recognition of altered segments in Brazilian Sign Language

The purpose of this study was to verify discriminative control by segments of signs in adolescents with deafness who use Brazilian Sign Language (BSL). Four adolescent with bilateral deafness, with 3 years of BSL teaching, saw a video presenting a children's tale in BSL. After showing accurate understanding of the story, participants saw another video of the same story with 12 signs altered in one of their segments (hand configuration, place of articulation, or movement). They apparently did not detect the alterations. However, when the signs were presented in isolation in a matching-to-sample test, they virtually always selected the picture corresponding to the unaltered signs. Three participants selected an unfamiliar picture in 50% or more trials with an altered sign as a sample, showing that they could detect the majority of the altered signs.

Interactive application in spanish sign language for a public transport environment

[ES] En este trabajo se presenta el diseño de una herramienta multimedia que traduce a la lengua de signos españolas los mensajes de avisos que puede proporcionar un sistema de megafonía. El objetivo del trabajo es proporcionar una herramienta que mejore la inclusión social de las personas con discapacidades auditivas. Con este propósito, se han seleccionado el entorno y los mensajes de audio habituales en un aeropuerto para desarrollar este proyecto piloto. Por último, los audios se han traducido a lengua de signos españolas sintetizando un avatar usando la técnica de animación de rotoscopía a partir de la grabación en vídeo de un traductor. Los resultados finales han sido evaluados por personas sordas.