Neuromorphic model for sound source segregation

While humans can easily segregate and track a speaker's voice in a loud noisy environment, most modern speech recognition systems still perform poorly in loud background noise. The computational principles behind auditory source segregation in humans is not yet fully understood. In this dissertation, we develop a computational model for source segregation inspired by auditory processing in the brain. To support the key principles behind the computational model, we conduct a series of electro-encephalography experiments using both simple tone-based stimuli and more natural speech stimulus. Most source segregation algorithms utilize some form of prior information about the target speaker or use more than one simultaneous recording of the noisy speech mixtures. Other methods develop models on the noise characteristics. Source segregation of simultaneous speech mixtures with a single microphone recording and no knowledge of the target speaker is still a challenge. Using the principle of temporal coherence, we develop a novel computational model that exploits the difference in the temporal evolution of features that belong to different sources to perform unsupervised monaural source segregation. While using no prior information about the target speaker, this method can gracefully incorporate knowledge about the target speaker to further enhance the segregation.Through a series of EEG experiments we collect neurological evidence to support the principle behind the model. Aside from its unusual structure and computational innovations, the proposed model provides testable hypotheses of the physiological mechanisms of the remarkable perceptual ability of humans to segregate acoustic sources, and of its psychophysical manifestations in navigating complex sensory environments. Results from EEG experiments provide further insights into the assumptions behind the model and provide motivation for future single unit studies that can provide more direct evidence for the principle of temporal coherence.

Time-varying bispectral analysis of auditory evoked multi-channel scalp EEG

Time-varying bispectra, computed using a classical sliding window short-time Fourier approach, are analyzed for scalp EEG potentials evoked by an auditory stimulus and new observations are presented. A single, short duration tone is presented from the left or the right, direction unknown to the test subject. The subject responds by moving the eyes to the direction of the sound. EEG epochs sampled at 200 Hz for repeated trials are processed between -70 ms and +1200 ms with reference to the stimulus. It is observed that for an ensemble of correctly recognized cases, the best matching timevarying bispectra at (8 Hz, 8Hz) are for PZ-FZ channels and this is also largely the case for grand averages but not for power spectra at 8 Hz. Out of 11 subjects, the only exception for time-varying bispectral match was a subject with family history of Alzheimer’s disease and the difference was in bicoherence, not biphase.

Neurochemical regulation of auditory information processing studied with EEG/MEG: application to schizophrenia

Cognitive impairments of attention, memory and executive functions are a fundamental feature of the pathophysiology of schizophrenia. The neurophysiological and neurochemical changes in the auditory cortex are shown to underlie cognitive impairmentsin schizophrenia patients. Functional state of the neural substrate of auditory information processing could be objectively and non-invasively probed with auditory event-related potentials (ERPs) and event- related fields (ERFs). In the current work, we explored the neurochemical effect on the neural origins of auditory information processing in relation to schizophrenia. By means of ERPs/ERFs we aimed to determine how neural substrates of auditory information processing are modulated by antipsychotic medication in schizophrenia spectrum patients (Studies I, II) and by neuropharmacological challenges in healthy human subjects (Studies III, IV). First, with auditory ERPs we investigated the effects of olanzapine (Study I) and risperidone (Study II) in a group of patients with schizophrenia spectrum disorders. After 2 and 4 weeks of treatment, olanzapine has no significant effects on mismatch negativity(MMN) and P300, which, as it has been suggested, respectively reflect preattentive and attention-dependent information processing. After 2 weeks of treatment, risperidone has no significant effect on P300, however risperidone reduces P200 amplitude. This latter effect of risperidone on neural resources responsible for P200 generation could be partly explained through the action of dopamine. Subsequently, we used simultaneous EEG/MEG to investigate the effects of memantine (Study III) and methylphenidate (Study IV) in healthy subjects. We found that memantine modulates MMN response without changing other ERP components. This could be interpreted as being due to the possible influence of memantine through the NMDA receptors on auditory change- detection mechanism, with processing of auditory stimuli remaining otherwise unchanged. Further, we found that methylphenidate does not modulate the MMN response. This finding could indicate no association between catecholaminergic activities and electrophysiological measures of preattentive auditory discrimination processes reflected in the MMN. However, methylphenidate decreases the P200 amplitudes. This could be interpreted as a modulation of auditory information processing reflected in P200 by dopaminergic and noradrenergic systems. Taken together, our set of studies indicates a complex pattern of neurochemical influences produced by the antipsychotic drugs in the neural substrate of auditory information processing in patients with schizophrenia spectrum disorders and by the pharmacological challenges in healthy subjects studied with ERPs and ERFs.

Charting developmental EEG gamma changes in the auditory modality /

The oscillation of neuronal circuits reflected in the EEG gamma frequency may be fundamental to the perceptual process referred to as binding (the integration of various thoughts and perceptions into a coherent picture). The aim of our study was to expand our knowledge of the developmental course ofEEG gamma in the auditory modality. 2 We investigated EEG 40 Hz gamma band responses (35.2 to 43.0 Hz) using an auditory novelty oddball paradigm alone and with a visual-number-series distracter task in 208 participants as a function of age (7 years to adult) at 9 sites across the sagital and lateral axes (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4). Gamma responses were operationally defined as change in power or a change in phase synchrony level from baseline within two time windows. The evoked gamma response was defined as a significant change from baseline occurring between 0 to 150 ms after stimulus onset; the induced gamma response was measured from 250 to 750 ms after stimulus onset. A significant evoked gamma band response was found when measuring changes in both power and phase synchrony. The increase in both measures was maximal at frontal regions. Decreases in both measures were found when participants were distracted by a secondary task. For neither measure were developmental effects noted. However, evoked gamma power was significantly enhanced with the presentation of a novel stimulus, especially at the right frontal site (F4); frontal evoked gamma phase synchrony also showed enhancement for novel stimuli but only for our two oldest age groups (16-18 year olds and adults). Induced gamma band responses also varied with task-dependent cognitive stimulus properties. In the induced gamma power response in all age groups, target stimuli generated the highest power values at the parietal region, while the novel stimuli were always below baseline. Target stimuli increased induced synchrony in all regions for all participants, but the novel stimulus selectively affected participants dependent on their age and gender. Adult participants, for example, exhibited a reduction in gamma power, but an increase in synchrony to the novel stimulus within the same region. Induced gamma synchrony was more sensitive to the gender of the participant than was induced gamma power. While induced gamma power produced little effects of age, gamma synchrony did have age effects. These results confirm that the perceptual process which regulates gamma power is distinct from that which governs the synchronization for neuronal firing, and both gamma power and synchrony are important factors to be considered for the "binding" hypothesis. However, there is surprisingly little effect of age on the absolute levels of or distribution of EEG gamma in the age range investigated.

Resting-state EEG in schizophrenia: auditory verbal hallucinations are related to shortening of specific microstates

Abnormal perceptions and cognitions in schizophrenia might be related to abnormal resting states of the brain. Previous research found that a specific class (class D) of sub-second electroencephalography (EEG) microstates was shortened in schizophrenia. This shortening correlated with positive symptoms. We questioned if this reflected positive psychotic traits or present psychopathology.

Is gamma band EEG synchronization reduced during auditory driving in schizophrenia patients with auditory verbal hallucinations?

Auditory verbal hallucinations (AVH) in schizophrenia patients assumingly result from a state inadequate activation of the primary auditory system. We tested brain responsiveness to auditory stimulation in healthy controls (n=26), and in schizophrenia patients that frequently (n=18) or never (n=11) experienced AVH. Responsiveness was assessed by driving the EEG with click-tones at 20, 30 and 40Hz. We compared stimulus induced EEG changes between groups using spectral amplitude maps and a global measure of phase-locking (GFS). As expected, the 40Hz stimulation elicited the strongest changes. However, while controls and non-hallucinators increased 40Hz EEG activity during stimulation, a left-lateralized decrease was observed in the hallucinators. These differences were significant (p=.02). As expected, GFS increased during stimulation in controls (p=.08) and non-hallucinating patients (p=.06), which was significant when combining the two groups (p=.01). In contrast, GFS decreased with stimulation in hallucinating patients (p=0.13), resulting in a significantly different GFS response when comparing subjects with and without AVH (p<.01). Our data suggests that normally, 40Hz stimulation leads to the activation of a synchronized network representing the sensory input, but in hallucinating patients, the same stimulation partly disrupts ongoing activity in this network.

Functional neuroanatomy of auditory mismatch processing : an event-related fMRI study of duration-deviant oddballs

This study was designed to identify the neural networks underlying automatic auditory deviance detection in 10 healthy subjects using functional magnetic resonance imaging. We measured blood oxygenation level-dependent contrasts derived from the comparison of blocks of stimuli presented as a series of standard tones (50 ms duration) alone versus blocks that contained rare duration-deviant tones (100 ms) that were interspersed among a series of frequent standard tones while subjects were watching a silent movie. Possible effects of scanner noise were assessed by a “no tone” condition. In line with previous positron emission tomography and EEG source modeling studies, we found temporal lobe and prefrontal cortical activation that was associated with auditory duration mismatch processing. Data were also analyzed employing an event-related hemodynamic response model, which confirmed activation in response to duration-deviant tones bilaterally in the superior temporal gyrus and prefrontally in the right inferior and middle frontal gyri. In line with previous electrophysiological reports, mismatch activation of these brain regions was significantly correlated with age. These findings suggest a close relationship of the event-related hemodynamic response pattern with the corresponding electrophysiological activity underlying the event-related “mismatch negativity” potential, a putative measure of auditory sensory memory.

Sensory auditory processing and intuitive sound detection : an investigation of musical experts and nonexperts

The auditory system can detect occasional changes (deviants) in acoustic regularities without the need for subjects to focus their attention on the sound material. Deviant detection is reflected in the elicitation of the mismatch negativity component (MMN) of the event-related potentials. In the studies presented in this thesis, the MMN is used to investigate the auditory abilities for detecting similarities and regularities in sound streams. To investigate the limits of these processes, professional musicians have been tested in some of the studies. The results show that auditory grouping is already more advanced in musicians than in nonmusicians and that the auditory system of musicians can, unlike that of nonmusicians, detect a numerical regularity of always four tones in a series. These results suggest that sensory auditory processing in musicians is not only a fine tuning of universal abilities, but is also qualitatively more advanced than in nonmusicians. In addition, the relationship between the auditory change-detection function and perception is examined. It is shown that, contrary to the generally accepted view, MMN elicitation does not necessarily correlate with perception. The outcome of the auditory change-detection function can be implicit and the implicit knowledge of the sound structure can, after training, be utilized for behaviorally correct intuitive sound detection. These results illustrate the automatic character of the sensory change detection function.

Human brain mechanisms of auditory and audiovisual selective attention

Selective attention refers to the process in which certain information is actively selected for conscious processing, while other information is ignored. The aim of the present studies was to investigate the human brain mechanisms of auditory and audiovisual selective attention with functional magnetic resonance imaging (fMRI), electroencephalography (EEG) and magnetoencephalography (MEG). The main focus was on attention-related processing in the auditory cortex. It was found that selective attention to sounds strongly enhances auditory cortex activity associated with processing the sounds. In addition, the amplitude of this attention-related modulation was shown to increase with the presentation rate of attended sounds. Attention to the pitch of sounds and to their location appeared to enhance activity in overlapping auditory-cortex regions. However, attention to location produced stronger activity than attention to pitch in the temporo-parietal junction and frontal cortical regions. In addition, a study on bimodal attentional selection found stronger audiovisual than auditory or visual attention-related modulations in the auditory cortex. These results were discussed in light of Näätänen s attentional-trace theory and other research concerning the brain mechanisms of selective attention.

Towards Optimal Testing of Auditory Memory : Methodological development of recording of the mismatch negativity (MMN) of the auditory event-related potential (ERP)

The overlapping sound pressure waves that enter our brain via the ears and auditory nerves must be organized into a coherent percept. Modelling the regularities of the auditory environment and detecting unexpected changes in these regularities, even in the absence of attention, is a necessary prerequisite for orientating towards significant information as well as speech perception and communication, for instance. The processing of auditory information, in particular the detection of changes in the regularities of the auditory input, gives rise to neural activity in the brain that is seen as a mismatch negativity (MMN) response of the event-related potential (ERP) recorded by electroencephalography (EEG). --- As the recording of MMN requires neither a subject s behavioural response nor attention towards the sounds, it can be done even with subjects with problems in communicating or difficulties in performing a discrimination task, for example, from aphasic and comatose patients, newborns, and even fetuses. Thus with MMN one can follow the evolution of central auditory processing from the very early, often critical stages of development, and also in subjects who cannot be examined with the more traditional behavioural measures of auditory discrimination. Indeed, recent studies show that central auditory processing, as indicated by MMN, is affected in different clinical populations, such as schizophrenics, as well as during normal aging and abnormal childhood development. Moreover, the processing of auditory information can be selectively impaired for certain auditory attributes (e.g., sound duration, frequency) and can also depend on the context of the sound changes (e.g., speech or non-speech). Although its advantages over behavioral measures are undeniable, a major obstacle to the larger-scale routine use of the MMN method, especially in clinical settings, is the relatively long duration of its measurement. Typically, approximately 15 minutes of recording time is needed for measuring the MMN for a single auditory attribute. Recording a complete central auditory processing profile consisting of several auditory attributes would thus require from one hour to several hours. In this research, I have contributed to the development of new fast multi-attribute MMN recording paradigms in which several types and magnitudes of sound changes are presented in both speech and non-speech contexts in order to obtain a comprehensive profile of auditory sensory memory and discrimination accuracy in a short measurement time (altogether approximately 15 min for 5 auditory attributes). The speed of the paradigms makes them highly attractive for clinical research, their reliability brings fidelity to longitudinal studies, and the language context is especially suitable for studies on language impairments such as dyslexia and aphasia. In addition I have presented an even more ecological paradigm, and more importantly, an interesting result in view of the theory of MMN where the MMN responses are recorded entirely without a repetitive standard tone. All in all, these paradigms contribute to the development of the theory of auditory perception, and increase the feasibility of MMN recordings in both basic and clinical research. Moreover, they have already proven useful in studying for instance dyslexia, Asperger syndrome and schizophrenia.

Bridging a yawning chasm: EEG investigations into the debate concerning the role of the human mirror neuron system in contagious yawning

Ongoing debate in the literature concerns whether there is a link between contagious yawning and the human mirror neuron system (hMNS). One way of examining this issue is with the use of the electroencephalogram (EEG) to measure changes in mu activation during the observation of yawns. Mu oscillations are seen in the alpha bandwidth of the EEG (8–12 Hz) over sensorimotor areas. Previous work has shown that mu suppression is a useful index of hMNS activation and is sensitive to individual differences in empathy. In two experiments, we presented participants with videos of either people yawning or control stimuli. We found greater mu suppression for yawns than for controls over right motor and premotor areas, particularly for those scoring higher on traits of empathy. In a third experiment, auditory recordings of yawns were compared against electronically scrambled versions of the same yawns. We observed greater mu suppression for yawns than for the controls over right lateral premotor areas. Again, these findings were driven by those scoring highly on empathy. The results from these experiments support the notion that the hMNS is involved in contagious yawning, emphasise the link between contagious yawning and empathy, and stress the importance of good control stimuli.

The effects of dexmedetomidine/remifentanil and midazolam/remifentanil on auditory-evoked potentials and electroencephalogram at light-to-moderate sedation levels in healthy subjects

Avoidance of excessively deep sedation levels is problematic in intensive care patients. Electrophysiologic monitoring may offer an approach to solving this problem. Since electroencephalogram (EEG) responses to different sedation regimens vary, we assessed electrophysiologic responses to two sedative drug regimens in 10 healthy volunteers. Dexmedetomidine/remifentanil (dex/remi group) and midazolam/remifentanil (mida/remi group) were infused 7 days apart. Each combination of medications was given at stepwise intervals to reach Ramsay scores (RS) 2, 3, and 4. Resting EEG, bispectral index (BIS), and the N100 amplitudes of long-latency auditory-evoked potentials (ERP) were recorded at each level of sedation. During dex/remi, resting EEG was characterized by a recurrent high-power low-frequency pattern which became more pronounced at deeper levels of sedation. BIS Index decreased uniformly in only the dex/remi group (from 94 +/- 3 at baseline to 58 +/- 14 at RS 4) compared to the mida/remi group (from 94 +/- 2 to 76 +/- 10; P = 0.029 between groups). The ERP amplitudes decreased from 5.3 +/- 1.3 at baseline to 0.4 +/- 1.1 at RS 4 (P = 0.003) in only the mida/remi group. We conclude that ERPs in volunteers sedated with dex/remi, in contrast to mida/remi, indicate a cortical response to acoustic stimuli, even when sedation reaches deeper levels. Consequently, ERP can monitor sedation with midazolam but not with dexmedetomidine. The reverse is true for BIS.

Event-related potential and EEG measures in Parkinson's disease without and with dementia

Nondemented Parkinson’s disease (PD) patients showed increased amplitude of event-related potential component P3. We recorded 18-channel spontaneous eyes-closed resting EEG and auditory oddball event-related potentials in 29 PD patients and 11 age-matched controls. Combining Mini-Mental State Examination score and oddball P3 counting performance, 15 patients were intellectually normal, 7 moderately, and 7 severely demented. P3 and N1 amplitude and latency, mean amplitude of 1,024 ms post-stimulus (separate after rare and after frequent stimuli), and resting EEG total power for 40 s were computed, and linearly regressed for age, sex, and L-dopa dosage. In nondemented PD patients, increased P3 amplitude was confirmed, but N1 amplitude and mean amplitude after rare and frequent stimuli were also increased as well as – most important – resting EEG total power. With increasing dementia, amplitude and power decreased, and P3 latency increased. Task demands cannot explain increased P3 amplitude, since similarly increased EEG total power was found during no-task resting. Prospective studies must determine whether P3 amplitude and EEG power in nondemented PD patients can serve as predictors of dementia.

Agency and ownership are independent components of 'sensing the self' in the auditory-verbal domain

'Sensing the self' relies on the ability to distinguish self-generated from external stimuli. It requires functioning mechanisms to establish feelings of agency and ownership. Agency is defined causally, where the subjects action is followed by an effect. Ownership is defined by the features of the effect, independent from the action. In our study, we manipulated these qualities separately. 13 right-handed healthy individuals performed the experiment while 76-channel EEG was recorded. Stimuli consisted of visually presented words, read aloud by the subject. The experiment consisted of six conditions: (a) subjects saw a word, read it aloud, heard it in their own voice; (b) like a, but the word was heard in an unfamiliar voice; (c) subject heard a word in his/her own voice without speaking; (d) like c, but the word was heard in an unfamiliar voice; (e) like a, but subjects heard the word with a delay; (f) subjects read without hearing. ERPs and difference maps were computed for all conditions. Effects were analysed topographically. The N100 (86-172 ms) displayed significant main effects of agency and ownership. The topographies of the two effects shared little common variance, suggesting independent effects. Later effects (174-400 ms) of agency and ownership were topographically similar, suggesting common mechanisms. Replicating earlier studies, significant N100 suppression was observed, with a topography resembling the agency effect. 'Sensing the self' appears to recruit from at least two very distinct processes: an agency assessment that represents causality and an ownership assessment that compares stimulus features with memory content.

Effects of binaural stimulation in attention and EEG

When two pure tones of slightly different frequency are presented separately to each ear, the listener perceives a third single tone with amplitude variations at a frequency that equals the difference between the two tones, this perceptual illusion is known as binaural auditory beat. There are anecdotal reports that suggest that the binaural beat can entrain EEG activity and may affect the arousal levels, although few studies have been published. There is a need for double-blind, well-designed studies in order to establish a solid foundation for these sounds, as most of the documented benefits come from self-reported cases that could be affected by placebo effect. As BB’s are a cheap technology (it even exists a free open source programmable bin aural-beat generator on the internet named Gnaural), any achievement in this area could be of public interest. The aim in our research was to explore the potential of BB’s in a particular field: tasks that require focus and concentration. In order to detect changes in the brain waves that could relate to any particular improvement, EEG recordings of a small sample of individuals were also obtained. In this study we compare the effect of different binaural stimulation in 7 EEG frequency ranges, 78 participants were exposed to 20 min binaural beat stimulation. The effects were obtained both qualitative with cognitive test and quantitative with EEG analysis. Results suggest no significant statistical improvement in 20 min stimulation.