Building Autonomous Sensitive Artificial Listeners

This paper describes a substantial effort to build a real-time interactive multimodal dialogue system with a focus on emotional and non-verbal interaction capabilities. The work is motivated by the aim to provide technology with competences in perceiving and producing the emotional and non-verbal behaviours required to sustain a conversational dialogue. We present the Sensitive Artificial Listener (SAL) scenario as a setting which seems particularly suited for the study of emotional and non- verbal behaviour, since it requires only very limited verbal understanding on the part of the machine. This scenario allows us to concentrate on non-verbal capabilities without having to address at the same time the challenges of spoken language understanding, task modeling etc. We first report on three prototype versions of the SAL scenario, in which the behaviour of the Sensitive Artificial Listener characters was determined by a human operator. These prototypes served the purpose of verifying the effectiveness of the SAL scenario and allowed us to collect data required for building system components for analysing and synthesising the respective behaviours. We then describe the fully autonomous integrated real-time system we created, which combines incremental analysis of user behaviour, dialogue management, and synthesis of speaker and listener behaviour of a SAL character displayed as a virtual agent. We discuss principles that should underlie the evaluation of SAL-type systems. Since the system is designed for modularity and reuse, and since it is publicly available, the SAL system has potential as a joint research tool in the affective computing research community.

Building autonomous sensitive artificial listeners (Extended abstract)

This paper describes a substantial effort to build a real-time interactive multimodal dialogue system with a focus on emotional and non-verbal interaction capabilities. The work is motivated by the aim to provide technology with competences in perceiving and producing the emotional and non-verbal behaviours required to sustain a conversational dialogue. We present the Sensitive Artificial Listener (SAL) scenario as a setting which seems particularly suited for the study of emotional and non-verbal behaviour, since it requires only very limited verbal understanding on the part of the machine. This scenario allows us to concentrate on non-verbal capabilities without having to address at the same time the challenges of spoken language understanding, task modeling etc. We first summarise three prototype versions of the SAL scenario, in which the behaviour of the Sensitive Artificial Listener characters was determined by a human operator. These prototypes served the purpose of verifying the effectiveness of the SAL scenario and allowed us to collect data required for building system components for analysing and synthesising the respective behaviours. We then describe the fully autonomous integrated real-time system we created, which combines incremental analysis of user behaviour, dialogue management, and synthesis of speaker and listener behaviour of a SAL character displayed as a virtual agent. We discuss principles that should underlie the evaluation of SAL-type systems. Since the system is designed for modularity and reuse, and since it is publicly available, the SAL system has potential as a joint research tool in the affective computing research community.

Syntactic Structures in Irish-Language Proverbs

In recent years, linguistic studies into the concept of ‘proverbiality’ have provided paremiologists with a more comprehensive understanding of the form and function of proverbial markers in a wide range of languages. Unfortunately, the Irish language has, until now, not featured in any of these linguistic analyses. This paper seeks to re-address this imbalance and to bring the unique structure and style of Irish-language proverbs to the attention of the international community of paremiologistsfor the first time. This research study applies the general methodology adopted by such scholars as Mahgoub (1948), Silverman-Weinreich (1978) and Arora (1984), toa corpus sample of Irish proverbs and provides both a qualitative and quantitative account of the most salient syntactic structures contained in Irish-language proverbs. Proverbial patterns as well as the collocation of proverbial markers are also discussed.

Beyond the Sign. Henri Meschonnic's Poetics of Rhythm and Continuum. Towards an Anthropological Theory of Language

Basing the conception of language on the sign represents also an obstacle to the awareness of certain elements of human life, especially to a full understanding of what language or art do, Henri Meschonnic’s poetics of the continuum and of rhythm criticizes the sign based on Benveniste’s terms of rhythm and discourse, developing an anthropology of language. Rhythm, for Meschonnic, is no formal metrical but a semantic principle, each time unique and unforeseeable. As for Humboldt, his starting point is not the word but the ensemble of speech; language is not ergon but energeia. The poem then is not a literary form but a process of transformation that Meschonnic defines as the invention of a form of life by a form of language and vice versa. Thus a poem is a way of thinking and rhythm is form in movement. The particular subject of art and literature is consequently not the author but a process of subjectivation – this is the contrary of the conception of the sign. By demonstrating the limits of the sign, Meschonnic’s poetics attempts to thematize the intelligibility of presence. Art and literature raise our awareness of this element of human life we cannot grasp conceptually. This poetical thinking is a necessary counterforce against all institutionalization.

Language development at 3 years in pre-term children of birth weight below 1000 g

Language development at 3 years of pre-term children born below 1000 g birth weight was compared with full-term controls matched for social background. The pre-term group used less complex expressive language and showed lower receptive understanding, auditory memory and verbal reasoning. Language outcome was related to intraventricular haemorrhage but not to global indication of postnatal illness such as number of days on the ventilator. Average verbal intelligence in environmentally low risk, extremely low birth weight children is an insufficient indicator of complex language functioning.

Children’s essentialist reasoning about language and race

Across four studies, we directly compared children’s essentialist reasoning about the stability of race and language throughout an individual’s lifespan. Monolingual English-speaking children were presented with a series of images of children who were either White or Black; each face was paired with a voice clip in either English or French. Participants were asked which of two adults each target child would grow up to be – one who was a ‘match’ to the target child in race but not language, and the other a ‘match’ in language but not race. Nine- to 10-year-old European American children chose the race-match, rather than the language-match. In contrast, 5–6-year-old European American children in both urban, racially diverse, and rural, racially homogeneous environments chose the language-match, even though this necessarily meant that the target child would transform racial categories. Although surprising in light of adult reasoning, these young children demonstrated an intuition about the relative stability of an individual’s language compared to her racial group membership. Yet, 5–6-year-old African American children, similar to the older European American children, chose the race-match, suggesting that membership in a racial minority group may highlight children’s reasoning about race as a stable category. Theoretical implications for our understanding of children’s categorization of human kinds are discussed.

Perception of Filtered Speech by Children with Developmental Dyslexia and Children with Specific Language Impairments

Here we use two filtered speech tasks to investigate children’s processing of slow (<4 Hz) versus faster (∼33 Hz) temporal modulations in speech. We compare groups of children with either developmental dyslexia (Experiment 1) or speech and language impairments (SLIs, Experiment 2) to groups of typically-developing (TD) children age-matched to each disorder group. Ten nursery rhymes were filtered so that their modulation frequencies were either low-pass filtered (<4 Hz) or band-pass filtered (22 – 40 Hz). Recognition of the filtered nursery rhymes was tested in a picture recognition multiple choice paradigm. Children with dyslexia aged 10 years showed equivalent recognition overall to TD controls for both the low-pass and band-pass filtered stimuli, but showed significantly impaired acoustic learning during the experiment from low-pass filtered targets. Children with oral SLIs aged 9 years showed significantly poorer recognition of band pass filtered targets compared to their TD controls, and showed comparable acoustic learning effects to TD children during the experiment. The SLI samples were also divided into children with and without phonological difficulties. The children with both SLI and phonological difficulties were impaired in recognizing both kinds of filtered speech. These data are suggestive of impaired temporal sampling of the speech signal at different modulation rates by children with different kinds of developmental language disorder. Both SLI and dyslexic samples showed impaired discrimination of amplitude rise times. Implications of these findings for a temporal sampling framework for understanding developmental language disorders are discussed.