Intonation modelling and adaptation for emotional prosody generation

This paper proposes an HMM-based approach to generating emotional intonation patterns. A set of models were built to represent syllable-length intonation units. In a classification framework, the models were able to detect a sequence of intonation units from raw fundamental frequency values. Using the models in a generative framework, we were able to synthesize smooth and natural sounding pitch contours. As a case study for emotional intonation generation, Maximum Likelihood Linear Regression (MLLR) adaptation was used to transform the neutral model parameters with a small amount of happy and sad speech data. Perceptual tests showed that listeners could identify the speech with the sad intonation 80% of the time. On the other hand, listeners formed a bimodal distribution in their ability to detect the system generated happy intontation and on average listeners were able to detect happy intonation only 46% of the time. © Springer-Verlag Berlin Heidelberg 2005.

Understanding and production of prosody in children with Williams syndrome: A developmental trajectory approach

This study investigated the development of three aspects of linguistic prosody in a group of children with Williams syndrome compared to typically developing children. The prosodic abilities investigated were: (1) the ability to understand and use prosody to make specific words or syllables stand out in an utterance (focus); (2) the ability to understand and use prosody to disambiguate complex noun phrases (chunking); (3) the ability to understand and use prosody to regulate conversational behaviour (turn-end). The data were analysed using a cross-sectional developmental trajectory approach. The results showed that, relative to chronological age, there was a delayed onset in the development of the ability of children with WS to use prosody to signal the most important word in an utterance (the focus function). Delayed rate of development was found for all the other aspects of expressive and receptive prosody under investigation. However, when non-verbal mental age was taken into consideration, there were no differences between the children with WS and the controls neither with the onset nor with the rate of development for any of the prosodic skills under investigation apart from the ability to use prosody in order to regulate conversational behaviour. We conclude that prosody is not a ‘preserved’ cognitive skill in WS. The genetic factors, development in other cognitive domains and environmental influences affect developmental pathways and as a result, development proceeds along an atypical trajectory.

Affective prosody in children with Williams syndrome

The aim of the current study was to investigate expressive affect in children with Williams syndrome ( WS) in comparison to typically developing children in an experimental task and in spontaneous speech. Fourteen children with WS, 14 typically developing children matched to the WS group for receptive language ( LA) and 15 typically developing children matched to the WS groups for chronological age ( CA) were recruited. Affect was investigated using an experimental Output Affect task from the Profiling Elements of Prosodic Systems-Child version ( PEPS-C) battery, and by measuring pitch range and vowel durations from a spontaneous speech task. The children were also rated for level of emotional involvement by phonetically naive listeners. The WS group performed similarly to the LA and CA groups on the Output Affect task. With regard to vowel durations, the WS group was no different from the LA group; however both the WS and the LA groups were found to use significantly longer vowels than the CA group. The WS group differed significantly from both control groups on their range of pitch range and was perceived as being significantly more emotionally involved than the two control groups.

Neural response to emotional prosody in schizophrenia and in bipolar affective disorder

Background Evidence suggests a reversal of the normal left-lateralised response to speech in schizophrenia. Aims To test the brain's response to emotional prosody in schizophrenia and bipolar disorder. Method BOLD contrast functional magnetic resonance imaging of subjects while they passively listened or attended to sentences that differed in emotional prosody Results Patients with schizophrenia exhibited normal right-lateralisation of the passive response to 'pure' emotional prosody and relative left-lateralisation of the response to unfiltered emotional prosody When attending to emotional prosody, patients with schizophrenia activated the left insula more than healthy controls. When listening passively, patients with bipolar disorder demonstrated less activation of the bilateral superior temporal gyri in response to pure emotional prosody, and greater activation of the left superior temporal gyrus in response to unfiltered emotional prosody In both passive experiments, the patient groups activated different lateral temporal lobe regions. Conclusions Patients with schizophrenia and bipolar disorder may display some left-lateralisation of the normal right-lateralised temporal lobe response to emotional prosody. Declaration of interest R.M. received a studentship from Neuraxis,, and funding from the Neuroscience and Psychiatry Unit, University of Manchester.

The neural response to emotional prosody, as revealed by functional magnetic resonance imaging

Prosody is an important feature of language, comprising intonation, loudness, and tempo. Emotional prosodic processing forms an integral part of our social interactions. The main aim of this study was to use bold contrast fMRI to clarify the normal functional neuroanatomy of emotional prosody, in passive and active contexts. Subjects performed six separate scanning studies, within which two different conditions were contrasted: (1) "pure" emotional prosody versus rest; (2) congruent emotional prosody versus 'neutral' sentences; (3) congruent emotional prosody versus rest; (4) incongruent emotional prosody versus rest; (5) congruent versus incongruent emotional prosody; and (6) an active experiment in which subjects were instructed to either attend to the emotion conveyed by semantic content or that conveyed by tone of voice. Data resulting from these contrasts were analysed using SPM99. Passive listening to emotional prosody consistently activated the lateral temporal lobe (superior and/or middle temporal gyri). This temporal lobe response was relatively right-lateralised with or without semantic information. Both the separate and direct comparisons of congruent and incongruent emotional prosody revealed that subjects used fewer brain regions to process incongruent emotional prosody than congruent. The neural response to attention to semantics, was left lateralised, and recruited an extensive network not activated by attention to emotional prosody. Attention to emotional prosody modulated the response to speech, and induced right-lateralised activity, including the middle temporal gyrus. In confirming the results of lesion and neuropsychological studies, the current study emphasises the importance of the right hemisphere in the processing of emotional prosody, specifically the lateral temporal lobes. (C) 2003 Elsevier Science Ltd. All rights reserved.

Decoding emotional prosody in Parkinson's disease and its potential neuropsychological basis

Parkinson's disease patients may have difficulty decoding prosodic emotion cues. These data suggest that the basal ganglia are involved, but may reflect dorsolateral prefrontal cortex dysfunction. An auditory emotional n-back task and cognitive n-back task were administered to 33 patients and 33 older adult controls, as were an auditory emotional Stroop task and cognitive Stroop task. No deficit was observed on the emotion decoding tasks; this did not alter with increased frontal lobe load. However, on the cognitive tasks, patients performed worse than older adult controls, suggesting that cognitive deficits may be more prominent. The impact of frontal lobe dysfunction on prosodic emotion cue decoding may only become apparent once frontal lobe pathology rises above a threshold.

fMRI delineation of working memory for emotional prosody in the brain: commonalities with the lexico-semantic emotion network

Decoding emotional prosody is crucial for successful social interactions, and continuous monitoring of emotional intent via prosody requires working memory. It has been proposed by Ross and others that emotional prosody cognitions in the right hemisphere are organized in an analogous fashion to propositional language functions in the left hemisphere. This study aimed to test the applicability of this model in the context of prefrontal cortex working memory functions. BOLD response data were therefore collected during performance of two emotional working memory tasks by participants undergoing fMRI. In the prosody task, participants identified the emotion conveyed in pre-recorded sentences, and working memory load was manipulated in the style of an N-back task. In the matched lexico-semantic task, participants identified the emotion conveyed by sentence content. Block-design neuroimaging data were analyzed parametrically with SPM5. At first, working memory for emotional prosody appeared to be right-lateralized in the PFC, however, further analyses revealed that it shared much bilateral prefrontal functional neuroanatomy with working memory for lexico-semantic emotion. Supplementary separate analyses of males and females suggested that these language functions were less bilateral in females, but their inclusion did not alter the direction of laterality. It is concluded that Ross et al.'s model is not applicable to prefrontal cortex working memory functions, that evidence that working memory cannot be subdivided in prefrontal cortex according to material type is increased, and that incidental working memory demands may explain the frontal lobe involvement in emotional prosody comprehension as revealed by neuroimaging studies. (c) 2007 Elsevier Inc. All rights reserved.

Age-related decline in the ability to decode emotional prosody: primary or secondary phenomenon?

Emotion processing deficits can cause catastrophic damage to a person's ability to interact socially. While it is known that older adults have difficulty identifying facial emotions, it is still not clear whether this difficulty extends to identification of the emotion conveyed by prosody. This study investigated whether the ability of older adults to decode emotional prosody falls below that of young adults after controlling for loss of hearing sensitivity and key features of cognitive ageing. Apart from frontal lobe load, only verbal IQ was associated with the age-related reduction in performance displayed by older participants, but a notable deficit existed after controlling for its effects. It is concluded that older adults may indeed have difficulty deducing the emotion conveyed by prosody, and that while this difficulty can be exaggerated by some aspects of cognitive ageing, it is primary in origin.