Invariance between subjects of brain wave representations of language

In three experiments, electric brain waves of 19 subjects were recorded under several different experimental conditions for two purposes. One was to test how well we could recognize which sentence, from a set of 24 or 48 sentences, was being processed in the cortex. The other was to study the invariance of brain waves between subjects. As in our earlier work, the analysis consisted of averaging over trials to create prototypes and test samples, to both of which Fourier transforms were applied, followed by filtering and an inverse transformation to the time domain. A least-squares criterion of fit between prototypes and test samples was used for classification. In all three experiments, averaging over subjects improved the recognition rates. The most significant finding was the following. When brain waves were averaged separately for two nonoverlapping groups of subjects, one for prototypes and the other for test samples, we were able to recognize correctly 90% of the brain waves generated by 48 different sentences about European geography.

Invariance of brain-wave representations of simple visual images and their names

In two experiments, electric brain waves of 14 subjects were recorded under several different conditions to study the invariance of brain-wave representations of simple patches of colors and simple visual shapes and their names, the words blue, circle, etc. As in our earlier work, the analysis consisted of averaging over trials to create prototypes and test samples, to both of which Fourier transforms were applied, followed by filtering and an inverse transformation to the time domain. A least-squares criterion of fit between prototypes and test samples was used for classification. The most significant results were these. By averaging over different subjects, as well as trials, we created prototypes from brain waves evoked by simple visual images and test samples from brain waves evoked by auditory or visual words naming the visual images. We correctly recognized from 60% to 75% of the test-sample brain waves. The general conclusion is that simple shapes such as circles and single-color displays generate brain waves surprisingly similar to those generated by their verbal names. These results, taken together with extensive psychological studies of auditory and visual memory, strongly support the solution proposed for visual shapes, by Bishop Berkeley and David Hume in the 18th century, to the long-standing problem of how the mind represents simple abstract ideas.

Brain wave recognition of words

Electrical and magnetic brain waves of seven subjects under three experimental conditions were recorded for the purpose of recognizing which one of seven words was processed. The analysis consisted of averaging over trials to create prototypes and test samples, to both of which Fourier transforms were applied, followed by filtering and an inverse transformation to the time domain. The filters used were optimal predictive filters, selected for each subject and condition. Recognition rates, based on a least-squares criterion, varied widely, but all but one of 24 were significantly different from chance. The two best were above 90%. These results show that brain waves carry substantial information about the word being processed under experimental conditions of conscious awareness.

Brain-wave recognition of sentences

Electrical and magnetic brain waves of two subjects were recorded for the purpose of recognizing which one of 12 sentences or seven words auditorily presented was processed. The analysis consisted of averaging over trials to create prototypes and test samples, to each of which a Fourier transform was applied, followed by filtering and an inverse transformation to the time domain. The filters used were optimal predictive filters, selected for each subject. A still further improvement was obtained by taking differences between recordings of two electrodes to obtain bipolar pairs that then were used for the same analysis. Recognition rates, based on a least-squares criterion, varied, but the best were above 90%. The first words of prototypes of sentences also were cut and pasted to test, at least partially, the invariance of a word’s brain wave in different sentence contexts. The best result was above 80% correct recognition. Test samples made up only of individual trials also were analyzed. The best result was 134 correct of 288 (47%), which is promising, given that the expected recognition number by chance is just 24 (or 8.3%). The work reported in this paper extends our earlier work on brain-wave recognition of words only. The recognition rates reported here further strengthen the case that recordings of electric brain waves of words or sentences, together with extensive mathematical and statistical analysis, can be the basis of new developments in our understanding of brain processing of language.

Syntax processing by auditory cortical neurons in the FM–FM area of the mustached bat Pteronotus parnellii

Syntax denotes a rule system that allows one to predict the sequencing of communication signals. Despite its significance for both human speech processing and animal acoustic communication, the representation of syntactic structure in the mammalian brain has not been studied electrophysiologically at the single-unit level. In the search for a neuronal correlate for syntax, we used playback of natural and temporally destructured complex species-specific communication calls—so-called composites—while recording extracellularly from neurons in a physiologically well defined area (the FM–FM area) of the mustached bat’s auditory cortex. Even though this area is known to be involved in the processing of target distance information for echolocation, we found that units in the FM–FM area were highly responsive to composites. The finding that neuronal responses were strongly affected by manipulation in the time domain of the natural composite structure lends support to the hypothesis that syntax processing in mammals occurs at least at the level of the nonprimary auditory cortex.

Pitch perception: A dynamical-systems perspective

Two and a half millennia ago Pythagoras initiated the scientific study of the pitch of sounds; yet our understanding of the mechanisms of pitch perception remains incomplete. Physical models of pitch perception try to explain from elementary principles why certain physical characteristics of the stimulus lead to particular pitch sensations. There are two broad categories of pitch-perception models: place or spectral models consider that pitch is mainly related to the Fourier spectrum of the stimulus, whereas for periodicity or temporal models its characteristics in the time domain are more important. Current models from either class are usually computationally intensive, implementing a series of steps more or less supported by auditory physiology. However, the brain has to analyze and react in real time to an enormous amount of information from the ear and other senses. How is all this information efficiently represented and processed in the nervous system? A proposal of nonlinear and complex systems research is that dynamical attractors may form the basis of neural information processing. Because the auditory system is a complex and highly nonlinear dynamical system, it is natural to suppose that dynamical attractors may carry perceptual and functional meaning. Here we show that this idea, scarcely developed in current pitch models, can be successfully applied to pitch perception.

Stopped-flow NMR spectroscopy: real-time unfolding studies of 6-19F-tryptophan-labeled Escherichia coli dihydrofolate reductase.

Escherichia coli dihydrofolate reductase (DHFR; EC 1.5.1.3) contains five tryptophan residues that have been replaced with 6-19F-tryptophan. The 19F NMR assignments are known in the native, unliganded form and the unfolded form. We have used these assignments with stopped-flow 19F NMR spectroscopy to investigate the behavior of specific regions of the protein in real time during urea-induced unfolding. The NMR data show that within 1.5 sec most of the intensities of the native 19F resonances of the protein are lost but only a fraction (approximately 20%) of the intensities of the unfolded resonances appears. We postulate that the early disappearance of the native resonances indicates that most of the protein rapidly forms an intermediate in which the side chains have considerable mobility. Stopped-flow far-UV circular dichroism measurements indicate that this intermediate retains native-like secondary structure. Eighty percent of the intensities of the NMR resonances assigned to the individual tryptophans in the unfolded state appear with similar rate constants (k approximately 0.14 sec-1), consistent with the major phase of unfolding observed by stopped-flow circular dichroism (representing 80% of total amplitude). These data imply that after formation of the intermediate, which appears to represent an expanded structural form, all regions of the protein unfold at the same rate. Stopped-flow measurements of the fluorescence and circular dichroism changes associated with the urea-induced unfolding show a fast phase (half-time of about 1 sec) representing 20% of the total amplitude in addition to the slow phase mentioned above. The NMR data show that approximately 20% of the total intensity for each of the unfolded tryptophan resonances is present at 1.5 sec, indicating that these two phases may represent the complete unfolding of the two different populations of the native protein.