A neural correlate of syntactic encoding during speech production

Spoken language is one of the most compact and structured ways to convey information. The linguistic ability to structure individual words into larger sentence units permits speakers to express a nearly unlimited range of meanings. This ability is rooted in speakers' knowledge of syntax and in the corresponding process of syntactic encoding. Syntactic encoding is highly automatized, operates largely outside of conscious awareness, and overlaps closely in time with several other processes of language production. With the use of positron emission tomography we investigated the cortical activations during spoken language production that are related to the syntactic encoding process. In the paradigm of restrictive scene description, utterances varying in complexity of syntactic encoding were elicited. Results provided evidence that the left Rolandic operculum, caudally adjacent to Broca's area, is involved in both sentence-level and local (phrase-level) syntactic encoding during speaking.

Preserved speech abilities and compensation following prefrontal damage.

Lesions to left frontal cortex in humans produce speech production impairments (nonfluent aphasia). These impairments vary from subject to subject and performance on certain speech production tasks can be relatively preserved in some patients. A possible explanation for preservation of function under these circumstances is that areas outside left prefrontal cortex are used to compensate for the injured brain area. We report here a direct demonstration of preserved language function in a stroke patient (LF1) apparently due to the activation of a compensatory brain pathway. We used functional brain imaging with positron emission tomography (PET) as a basis for this study.

Research in speech communication.

Advances in digital speech processing are now supporting application and deployment of a variety of speech technologies for human/machine communication. In fact, new businesses are rapidly forming about these technologies. But these capabilities are of little use unless society can afford them. Happily, explosive advances in microelectronics over the past two decades have assured affordable access to this sophistication as well as to the underlying computing technology. The research challenges in speech processing remain in the traditionally identified areas of recognition, synthesis, and coding. These three areas have typically been addressed individually, often with significant isolation among the efforts. But they are all facets of the same fundamental issue--how to represent and quantify the information in the speech signal. This implies deeper understanding of the physics of speech production, the constraints that the conventions of language impose, and the mechanism for information processing in the auditory system. In ongoing research, therefore, we seek more accurate models of speech generation, better computational formulations of language, and realistic perceptual guides for speech processing--along with ways to coalesce the fundamental issues of recognition, synthesis, and coding. Successful solution will yield the long-sought dictation machine, high-quality synthesis from text, and the ultimate in low bit-rate transmission of speech. It will also open the door to language-translating telephony, where the synthetic foreign translation can be in the voice of the originating talker.

Recovery of hearing and vocal behavior after hair-cell regeneration

Postmitotic hair-cell regeneration in the inner ear of birds provides an opportunity to study the effect of renewed auditory input on auditory perception, vocal production, and vocal learning in a vertebrate. We used behavioral conditioning to test both perception and vocal production in a small Australian parrot, the budgerigar. Results show that both auditory perception and vocal production are disrupted when hair cells are damaged or lost but that these behaviors return to near normal over time. Precision in vocal production completely recovers well before recovery of full auditory function. These results may have particular relevance for understanding the relation between hearing loss and human speech production especially where there is consideration of an auditory prosthetic device. The present results show, at least for a bird, that even limited recovery of auditory input soon after deafening can support full recovery of vocal precision.

Song selectivity and sensorimotor signals in vocal learning and production

Bird song, like human speech, is a learned vocal behavior that requires auditory feedback. Both as juveniles, while they learn to sing, and as adults, songbirds use auditory feedback to compare their own vocalizations with an internal model of a target song. Here we describe experiments that explore a role for the songbird anterior forebrain pathway (AFP), a basal ganglia-forebrain circuit, in evaluating song feedback and modifying vocal output. First, neural recordings in anesthetized, juvenile birds show that single AFP neurons are specialized to process the song stimuli that are compared during sensorimotor learning. AFP neurons are tuned to both the bird's own song and the tutor song, even when these stimuli are manipulated to be very different from each other. Second, behavioral experiments in adult birds demonstrate that lesions to the AFP block the deterioration of song that normally follows deafening. This observation suggests that deafening results in an instructive signal, indicating a mismatch between feedback and the internal song model, and that the AFP is involved in generating or transmitting this instructive signal. Finally, neural recordings from behaving birds reveal robust singing-related activity in the AFP. This activity is likely to originate from premotor areas and could be modulated by auditory feedback of the bird's own voice. One possibility is that this activity represents an efference copy, predicting the sensory consequences of motor commands. Overall, these studies illustrate that sensory and motor processes are highly interrelated in this circuit devoted to vocal learning, as is true for brain areas involved in speech.

Training and search methods for speech recognition.

Speech recognition involves three processes: extraction of acoustic indices from the speech signal, estimation of the probability that the observed index string was caused by a hypothesized utterance segment, and determination of the recognized utterance via a search among hypothesized alternatives. This paper is not concerned with the first process. Estimation of the probability of an index string involves a model of index production by any given utterance segment (e.g., a word). Hidden Markov models (HMMs) are used for this purpose [Makhoul, J. & Schwartz, R. (1995) Proc. Natl. Acad. Sci. USA 92, 9956-9963]. Their parameters are state transition probabilities and output probability distributions associated with the transitions. The Baum algorithm that obtains the values of these parameters from speech data via their successive reestimation will be described in this paper. The recognizer wishes to find the most probable utterance that could have caused the observed acoustic index string. That probability is the product of two factors: the probability that the utterance will produce the string and the probability that the speaker will wish to produce the utterance (the language model probability). Even if the vocabulary size is moderate, it is impossible to search for the utterance exhaustively. One practical algorithm is described [Viterbi, A. J. (1967) IEEE Trans. Inf. Theory IT-13, 260-267] that, given the index string, has a high likelihood of finding the most probable utterance.