Exploiting Phonological Constraints for Handshape Inference in ASL Video

Handshape is a key articulatory parameter in sign language, and thus handshape recognition from signing video is essential for sign recognition and retrieval. Handshape transitions within monomorphemic lexical signs (the largest class of signs in signed languages) are governed by phonological rules. For example, such transitions normally involve either closing or opening of the hand (i.e., to exclusively use either folding or unfolding of the palm and one or more fingers). Furthermore, akin to allophonic variations in spoken languages, both inter- and intra- signer variations in the production of specific handshapes are observed. We propose a Bayesian network formulation to exploit handshape co-occurrence constraints, also utilizing information about allophonic variations to aid in handshape recognition. We propose a fast non-rigid image alignment method to gain improved robustness to handshape appearance variations during computation of observation likelihoods in the Bayesian network. We evaluate our handshape recognition approach on a large dataset of monomorphemic lexical signs. We demonstrate that leveraging linguistic constraints on handshapes results in improved handshape recognition accuracy. As part of the overall project, we are collecting and preparing for dissemination a large corpus (three thousand signs from three native signers) of American Sign Language (ASL) video. The video have been annotated using SignStream® [Neidle et al.] with labels for linguistic information such as glosses, morphological properties and variations, and start/end handshapes associated with each ASL sign.

A rule-based grapheme-to-phone converter for TTS systems in european portuguese

In this paper, a linguistically rule-based grapheme-to-phone (G2P) transcription algorithm is described for European Portuguese. A complete set of phonological and phonetic transcription rules regarding the European Portuguese standard variety is presented. This algorithm was implemented and tested by using online newspaper articles. The obtained experimental results gave rise to 98.80% of accuracy rate. Future developments in order to increase this value are foreseen. Our purpose with this work is to develop a module/ tool that can improve synthetic speech naturalness in European Portuguese. Other applications of this system can be expected like language teaching/learning. These results, together with our perspectives of future improvements, have proved the dramatic importance of linguistic knowledge on the development of Text-to-Speech systems (TTS).

Rôle des circuits cortico-striataux dans la planification et l'exécution de règles lexicales

Des recherches, autant chez l’homme que chez l’animal, proposent qu’il existerait, au sein des réseaux cérébraux, une organisation anatomique parallèle de circuits qui coordonne l’activité des structures qui participent à la planification et à l’exécution d’une action. Dans cette foulée, un modèle émerge qui attribue au cortex préfrontal (CPF) latéral une spécificité anatomo-fonctionnelle basée sur les niveaux de traitement en mémoire de travail (MT). Il s’agit du modèle « niveaux de traitement-dépendant », qui accorde un rôle important au CPF latéral dans l’acquisition et la représentation de règles guidant nos comportements. Des études en neuroimagerie fonctionnelle, utilisant le Wisconsin Card Sorting Task (WCST) ont permis de corroborer ce modèle et de dissocier trois niveaux de traitement en MT non seulement au sein du CPF latéral mais encore aux structures sous- corticales, les ganglions de la base (GB). Ces études suggèrent que certains noyaux des GB seraient topographiquement organisés avec le CPF latéral et contriburaient, sous certaines conditions, à des processus cognitifs et moteurs semblables à leur homologue cortical. Le but de notre étude est d'explorer la généralisation de la contribution des GB et du CPF au modèle niveaux de traitement-dépendant afin de voir si ce dernier est indépendant de la nature des stimuli en mémoire de travail. À cet effet, nous avons modifié le WCST en l’appliquant à un autre domaine, celui du langage. Nous avons remplacé les pictogrammes par des mots et modifié les règles formes, couleurs, nombres, par des règles sémantiques et phonologiques. L’analyse des résultats a démontré que différentes parties des GB de concert avec différentes régions du CPF se différencient quant aux niveaux de traitement en MT et ce, indépendamment de la nature des stimuli. Une deuxième analyse a permis d’évaluer les patrons d’activations liés aux conditions sémantiques et phonologiques. Ces résultats ont mis en évidence que les réseaux préfrontaux semblent liés aux processus exécutifs nécessaires à la réalisation de la tâche, indépendamment de la condition tandis que les aires associatives se dissocient davantage et contiennent des réseaux propres à la sémantique et à la phonologie.

Haplologia no falar paraense

O presente trabalho trata do fenômeno da Haplologia na fala espontânea de cidadãos paraenses. O estudo refere-se mais especificamente ao que chamamos de Haplologia entre frases. Avaliam-se os contextos de frases compostas apenas por /d/ - /d/, /t/ - /d/,/t/ - /t/ e /d/ - /t/, exemplificados respectivamente por: la(du) dʒi fora, per(tu) du, a gen(t∫i) t∫inha medu e tu(du) t∫inha. Os fatores avaliados dividem-se em dois grupos: linguísticos e extralinguísticos com o objetivo de mostrar os contextos favoráveis e desfavoráveis à aplicação do fenômeno em estudo. Os grupos de fatores linguísticos são: Relação entre palatalização e haplologia; Qualidade das vogais; Classe de palavra da sílaba elidida; Tonicidade das sílabas confinantes; e Estrutura silábica. No que se refere aos fatores extralinguísticos, analisamos: Sexo, Faixa etária e Escolaridade, seguindo a estratificação proposta no projeto Atlas Linguístico do Pará (ALIPA). Os dados analisados integram o corpus de duas cidades paraenses: Belém, a capital do Estado do Pará, e Itaituba, cidade paraense que fica a 891 km da capital mencionada. A coleta dos dados seguiu a orientação da Sociolinguística Variacionista. Os dados foram submetidos ao Programa de regra variável VARBRUL. Os resultados apontaram a haplologia como regra variável, entretanto, o fenômeno é pouco produtivo entre os informantes das duas cidades. Nos pressupostos da Sociolinguística Variacionista (Labov, 2008) a palatalização, o alteamento da vogal e a desconstrução do grupo consoantal podem ser considerados um processo de encaixamento, enquanto que, do ponto de vista fonético-fonológico, seriam considerados regras alimentadoras da haplologia (BISOL, 1996).

A computational model of prosody for Yorùbá text-to-speech synthesis

This work examines prosody modelling for the Standard Yorùbá (SY) language in the context of computer text-to-speech synthesis applications. The thesis of this research is that it is possible to develop a practical prosody model by using appropriate computational tools and techniques which combines acoustic data with an encoding of the phonological and phonetic knowledge provided by experts. Our prosody model is conceptualised around a modular holistic framework. The framework is implemented using the Relational Tree (R-Tree) techniques (Ehrich and Foith, 1976). R-Tree is a sophisticated data structure that provides a multi-dimensional description of a waveform. A Skeletal Tree (S-Tree) is first generated using algorithms based on the tone phonological rules of SY. Subsequent steps update the S-Tree by computing the numerical values of the prosody dimensions. To implement the intonation dimension, fuzzy control rules where developed based on data from native speakers of Yorùbá. The Classification And Regression Tree (CART) and the Fuzzy Decision Tree (FDT) techniques were tested in modelling the duration dimension. The FDT was selected based on its better performance. An important feature of our R-Tree framework is its flexibility in that it facilitates the independent implementation of the different dimensions of prosody, i.e. duration and intonation, using different techniques and their subsequent integration. Our approach provides us with a flexible and extendible model that can also be used to implement, study and explain the theory behind aspects of the phenomena observed in speech prosody.

Intonation contour realisation for Standard Yorùbá text-to-speech synthesis:a fuzzy computational approach

This paper presents a novel intonation modelling approach and demonstrates its applicability using the Standard Yorùbá language. Our approach is motivated by the theory that abstract and realised forms of intonation and other dimensions of prosody should be modelled within a modular and unified framework. In our model, this framework is implemented using the Relational Tree (R-Tree) technique. The R-Tree is a sophisticated data structure for representing a multi-dimensional waveform in the form of a tree. Our R-Tree for an utterance is generated in two steps. First, the abstract structure of the waveform, called the Skeletal Tree (S-Tree), is generated using tone phonological rules for the target language. Second, the numerical values of the perceptually significant peaks and valleys on the S-Tree are computed using a fuzzy logic based model. The resulting points are then joined by applying interpolation techniques. The actual intonation contour is synthesised by Pitch Synchronous Overlap Technique (PSOLA) using the Praat software. We performed both quantitative and qualitative evaluations of our model. The preliminary results suggest that, although the model does not predict the numerical speech data as accurately as contemporary data-driven approaches, it produces synthetic speech with comparable intelligibility and naturalness. Furthermore, our model is easy to implement, interpret and adapt to other tone languages.

A Computational Model for the Acquisition and Use of Phonological Knowledge

Does knowledge of language consist of symbolic rules? How do children learn and use their linguistic knowledge? To elucidate these questions, we present a computational model that acquires phonological knowledge from a corpus of common English nouns and verbs. In our model the phonological knowledge is encapsulated as boolean constraints operating on classical linguistic representations of speech sounds in term of distinctive features. The learning algorithm compiles a corpus of words into increasingly sophisticated constraints. The algorithm is incremental, greedy, and fast. It yields one-shot learning of phonological constraints from a few examples. Our system exhibits behavior similar to that of young children learning phonological knowledge. As a bonus the constraints can be interpreted as classical linguistic rules. The computational model can be implemented by a surprisingly simple hardware mechanism. Our mechanism also sheds light on a fundamental AI question: How are signals related to symbols?

The effects of phonological and morphological training on speech perception scores and grammatical judgments in deaf and hard-of-hearing children

Seventeen primary school deaf and hard-of-hearing children were given two types of training for 9 weeks each. Phonological training involved practice of /s, z, t, d/ in word final position in monomorphemic words. Morphological training involved learning and practicing the rules for forming third-person singular, present tense, past tense, and plurals. The words used in the two training types were different (monomorphemic or polymorphemic) but both involved word final /s, z, t, d/. Grammatical judgments were tested before and after training using short sentences that were read aloud by the child (or by the presenter if the child was unable to read them). Perception was tested with 150 key words in sentences using the trained morphemes and phonemes in word final position. Grammatical judgments for sentences involving the trained morphemes improved significantly after each type of training. Both types of training needed to be completed before a significant improvement was found for speech perception scores. The results suggest that both phonological and morphological training are beneficial in improving speech perception and grammatical performance of deaf and hard-of-hearing children and that both types of training were required to obtain the maximum benefit.

"Sounds Funny? – Humor Effects of Phonological and Prosodic Figures of Speech"

Whereas semantic, logical, and narrative features of verbal humor are well-researched, phonological and prosodic dimensions of verbal funniness are hardly explored. In a 2 × 2 design we varied rhyme and meter in humorous couplets. Rhyme and meter enhanced funniness ratings and supported faster processing. Rhyming couplets also elicited more intense and more positive affective responses, increased subjective comprehensibility and more accurate memory. The humor effect is attributed to special rhyme and meter features distinctive of humoristic poetry in several languages. Verses that employ these formal features make an artful use of typical poetic vices of amateurish poems written for birthday parties or other occasions. Their metrical patterning sounds “mechanical” rather than genuinely “poetic”; they also disregard rules for “good” rhymes. The processing of such verses is discussed in terms of a metacognitive integration of their poetically deviant features into an overall effect of processing ease. The study highlights the importance of nonsemantic rhetorical features in language processing.