Cortical processing of musical pitch as reflected by behavioural and electrophysiological evidence

In a musical context, the pitch of sounds is encoded according to domain-general principles not confined to music or even to audition overall but common to other perceptual and cognitive processes (such as multiple pattern encoding and feature integration), and to domain-specific and culture-specific properties related to a particular musical system only (such as the pitch steps of the Western tonal system). The studies included in this thesis shed light on the processing stages during which pitch encoding occurs on the basis of both domain-general and music-specific properties, and elucidate the putative brain mechanisms underlying pitch-related music perception. Study I showed, in subjects without formal musical education, that the pitch and timbre of multiple sounds are integrated as unified object representations in sensory memory before attentional intervention. Similarly, multiple pattern pitches are simultaneously maintained in non-musicians' sensory memory (Study II). These findings demonstrate the degree of sophistication of pitch processing at the sensory memory stage, requiring neither attention nor any special expertise of the subjects. Furthermore, music- and culture-specific properties, such as the pitch steps of the equal-tempered musical scale, are automatically discriminated in sensory memory even by subjects without formal musical education (Studies III and IV). The cognitive processing of pitch according to culture-specific musical-scale schemata hence occurs as early as at the sensory-memory stage of pitch analysis. Exposure and cortical plasticity seem to be involved in musical pitch encoding. For instance, after only one hour of laboratory training, the neural representations of pitch in the auditory cortex are altered (Study V). However, faulty brain mechanisms for attentive processing of fine-grained pitch steps lead to inborn deficits in music perception and recognition such as those encountered in congenital amusia (Study VI). These findings suggest that predispositions for exact pitch-step discrimination together with long-term exposure to music govern the acquisition of the automatized schematic knowledge of the music of a particular culture that even non-musicians possess.

Dynamics of contour, object and face processing in the human visual cortex

The neural basis of visual perception can be understood only when the sequence of cortical activity underlying successful recognition is known. The early steps in this processing chain, from retina to the primary visual cortex, are highly local, and the perception of more complex shapes requires integration of the local information. In Study I of this thesis, the progression from local to global visual analysis was assessed by recording cortical magnetoencephalographic (MEG) responses to arrays of elements that either did or did not form global contours. The results demonstrated two spatially and temporally distinct stages of processing: The first, emerging 70 ms after stimulus onset around the calcarine sulcus, was sensitive to local features only, whereas the second, starting at 130 ms across the occipital and posterior parietal cortices, reflected the global configuration. To explore the links between cortical activity and visual recognition, Studies II III presented subjects with recognition tasks of varying levels of difficulty. The occipito-temporal responses from 150 ms onwards were closely linked to recognition performance, in contrast to the 100-ms mid-occipital responses. The averaged responses increased gradually as a function of recognition performance, and further analysis (Study III) showed the single response strengths to be graded as well. Study IV addressed the attention dependence of the different processing stages: Occipito-temporal responses peaking around 150 ms depended on the content of the visual field (faces vs. houses), whereas the later and more sustained activity was strongly modulated by the observers attention. Hemodynamic responses paralleled the pattern of the more sustained electrophysiological responses. Study V assessed the temporal processing capacity of the human object recognition system. Above sufficient luminance, contrast and size of the object, the processing speed was not limited by such low-level factors. Taken together, these studies demonstrate several distinct stages in the cortical activation sequence underlying the object recognition chain, reflecting the level of feature integration, difficulty of recognition, and direction of attention.

Processing of Stimulus Repetition and Change in the Somatosensory System: Recordings of Electrical and Magnetic Brain Responses

In the present work, effects of stimulus repetition and change in a continuous stimulus stream on the processing of somatosensory information in the human brain were studied. Human scalp-recorded somatosensory event-related potentials (ERPs) and magnetoencephalographic (MEG) responses rapidly diminished with stimulus repetition when mechanical or electric stimuli were applied to fingers. On the contrary, when the ERPs and multi-unit a ctivity (MUA) were directly recorded from the primary (SI) and secondary (SII) somatosensory cortices in a monkey, there was no marked decrement in the somatosensory responses as a function of stimulus repetition. These results suggest that this rate effect is not due to the response diminution in the SI and SII cortices. Obviously the responses to the first stimulus after a long "silent" period are nhanced due to unspecific initial orientation, originating in more broadly distributed and/or deeper neural structures, perhaps in the prefrontal cortices. With fast repetition rates not only the late unspecific but also some early specific somatosensory ERPs were diminished in amplitude. The fast decrease of the ERPs as a function of stimulus repetition is mainly due to the disappearance of the orientation effect and with faster repetition rates additively due to stimulus specific refractoriness. A sudden infrequent change in the continuous stimulus stream also enhanced somatosensory MEG responses to electric stimuli applied to different fingers. These responses were quite similar to those elicited by the deviant stimuli alone when the frequent standard stimuli were omitted. This enhancement was obviously due to the release from refractoriness because the neural structures generating the responses to the infrequent deviants had more time to recover from the refractoriness than the respective structures for the standards. Infrequent deviant mechanical stimuli among frequent standard stimuli also enhanced somatosensory ERPs and, in addition, they elicited a new negative wave which did not occur in the deviants-alone condition. This extra negativity could be recorded to deviations in the stimulation site and in the frequency of the vibratory stimuli. This response is probably a somatosensory analogue of the auditory mismatch negativity (MMN) which has been suggested to reflect a neural mismatch process between the sensory input and the sensory memory trace.

Neurochemical regulation of auditory information processing studied with EEG/MEG: application to schizophrenia

Cognitive impairments of attention, memory and executive functions are a fundamental feature of the pathophysiology of schizophrenia. The neurophysiological and neurochemical changes in the auditory cortex are shown to underlie cognitive impairmentsin schizophrenia patients. Functional state of the neural substrate of auditory information processing could be objectively and non-invasively probed with auditory event-related potentials (ERPs) and event- related fields (ERFs). In the current work, we explored the neurochemical effect on the neural origins of auditory information processing in relation to schizophrenia. By means of ERPs/ERFs we aimed to determine how neural substrates of auditory information processing are modulated by antipsychotic medication in schizophrenia spectrum patients (Studies I, II) and by neuropharmacological challenges in healthy human subjects (Studies III, IV). First, with auditory ERPs we investigated the effects of olanzapine (Study I) and risperidone (Study II) in a group of patients with schizophrenia spectrum disorders. After 2 and 4 weeks of treatment, olanzapine has no significant effects on mismatch negativity(MMN) and P300, which, as it has been suggested, respectively reflect preattentive and attention-dependent information processing. After 2 weeks of treatment, risperidone has no significant effect on P300, however risperidone reduces P200 amplitude. This latter effect of risperidone on neural resources responsible for P200 generation could be partly explained through the action of dopamine. Subsequently, we used simultaneous EEG/MEG to investigate the effects of memantine (Study III) and methylphenidate (Study IV) in healthy subjects. We found that memantine modulates MMN response without changing other ERP components. This could be interpreted as being due to the possible influence of memantine through the NMDA receptors on auditory change- detection mechanism, with processing of auditory stimuli remaining otherwise unchanged. Further, we found that methylphenidate does not modulate the MMN response. This finding could indicate no association between catecholaminergic activities and electrophysiological measures of preattentive auditory discrimination processes reflected in the MMN. However, methylphenidate decreases the P200 amplitudes. This could be interpreted as a modulation of auditory information processing reflected in P200 by dopaminergic and noradrenergic systems. Taken together, our set of studies indicates a complex pattern of neurochemical influences produced by the antipsychotic drugs in the neural substrate of auditory information processing in patients with schizophrenia spectrum disorders and by the pharmacological challenges in healthy subjects studied with ERPs and ERFs.

Tarkkaavaisuuden vaikutus puheen merkityssisällön käsittelyyn: Aivojen herätevastetutkimus

Semantic processing can be studied with semantic priming. Target words that are preceded by semantically related prime words are recognized faster and more accurately than targets preceded by unrelated prime words. Semantic priming also affects the magnitude of the N400 event-related potential. The response is smaller to a target word when it is preceded by a related than an unrelated prime word. The effect is called the N400 effect. It is not yet clear, however, how attention modulates semantic priming and the N400 effect. This study investigated how the direction of attention affects the semantic processing of speech. The N400 effect was studied in experimental conditions in which the subjects attention was directed 1) away from the speech stimuli, 2) to phonological features of the speech stimuli, and 3) to semantic features of the speech stimuli. The first aim of the study was to investigate whether the N400 effect for spoken words is dependent on attention to the auditory information. The second aim was to study the differences in the N400 effect when attention is directed to the semantic or other features of speech stimuli. The results showed an N400 effect even when attention was directed away from the speech stimuli. The N400 effect was, however, stronger in conditions during which the speech stimuli were attended. The magnitude of the behavioral semantic priming and the N400 effect did not differ between the conditions during which attention was directed to the semantic or phonological features of the words. The findings indicate that the semantic processing of spoken words is not dependent on attention to auditory information. Furthermore, the results suggest that whether or not semantic processing is relevant for the task performance does not affect the semantic processing of attended spoken words.

Tarinakontekstin vaikutus sanojen muistamiseen kompleksisessa työmuistitehtävässä

Objectives. The sentence span task is a complex working memory span task used for estimating total working memory capacity for both processing (sentence comprehension) and storage (remembering a set of words). Several traditional models of working memory suggest that performance on these tasks relies on phonological short-term storage. However, long-term memory effects as well as the effects of expertise and strategies have challenged this view. This study uses a working memory task that aids the creation of retrieval structures in the form of stories, which have been shown to form integrated structures in longterm memory. The research question is whether sentence and story contexts boost memory performance in a complex working memory task. The hypothesis is that storage of the words in the task takes place in long-term memory. Evidence of this would be better recall for words as parts of sentences than for separate words, and, particularly, a beneficial effect for words as part of an organized story. Methods. Twenty stories consisting of five sentences each were constructed, and the stimuli in all experimental conditions were based on these sentences and sentence-final words, reordered and recombined for the other conditions. Participants read aloud sets of five sentences that either formed a story or not. In one condition they had to report all the last words at the end of the set, in another, they memorised an additional separate word with each sentence. The sentences were presented on the screen one word at a time (500 ms). After the presentation of each sentence, the participant verified a statement about the sentence. After five sentences, the participant repeated back the words in correct positions. Experiment 1 (n=16) used immediate recall, experiment 2 (n=21) both immediate recall and recall after a distraction interval (the operation span task). In experiment 2 a distracting mental arithmetic task was presented instead of recall in half of the trials, and an individual word was added before each sentence in the two experimental conditions when the participants were to memorize the sentence final words. Subjects also performed a listening span task (in exp.1) or an operation span task (exp.2) to allow comparison of the estimated span and performance in the story task. Results were analysed using correlations, repeated measures ANOVA and a chi-square goodness of fit test on the distribution of errors. Results and discussion. Both the relatedness of the sentences (the story condition) and the inclusion of the words into sentences helped memory. An interaction showed that the story condition had a greater effect on last words than separate words. The beneficial effect of the story was shown in all serial positions. The effects remained in delayed recall. When the sentences formed stories, performance in verification of the statements about sentence context was better. This, as well as the differing distributions of errors in different experimental conditions, suggest different levels of representation are in use in the different conditions. In the story condition, the nature of these representations could be in the form of an organized memory structure, a situation model. The other working memory tasks had only few week correlations to the story task. This could indicate that different processes are in use in the tasks. The results do not support short-term phonological storage, but instead are compatible with the words being encoded to LTM during the task.

Role of heat treatment in the processing and quality of oat flakes

This thesis reports on investigations into the influence of heat treatment on the manufacturing of oat flakes. Sources of variation in the oat flake quality are reviewed, including the whole chain from the farm to the consumer. The most important quality parameters of oat flakes are the absence of lipid hydrolysing enzymes, specific weight, thickness, breakage (fines), water absorption. Flavour, colour and pasting properties are also important, but were not included in the experimental part of this study. Of particular interest was the role of heat processing. The first possible heat treatment may occur already during grain drying, which in Finland generally happens at the farm. At the mill, oats are often kilned to stabilise the product by inactivating lipid hydrolysing enzymes. Almost invariably steaming is used during flaking, to soften the groats and reduce flake breakage. This thesis presents the use of a material science approach to investigating a complex system, typical of food processes. A combination of fundamental and empirical rheological measurements was used together with a laboratory scale process to simulate industrial processing. The results were verified by means of industrial trials. Industrially produced flakes at three thickness levels (nominally 0.75, 0.85 and 0.90 mm) were produced from kilned and unkilned oat groats, and the flake strength was measured at different moisture contents. Kilning was not found to significantly affect the force required to puncture a flake with a 2mm cylindrical probe, which was taken as a measure of flake strength. To further investigate how heat processing contributes to flake quality, dynamic mechanical analysis was used to characterise the effect of heat on the mechanical properties of oats. A marked stiffening of the groat, of up to about 50% increase in storage modulus, was observed during first heating at around 36 to 57°C. This was also observed in tablets prepared from ground groats and extracted oat starch. This stiffening was thus attributed to increased adhesion between starch granules. Groats were steamed in a laboratory steamer and were tempered in an oven at 80 110°C for 30 90 min. The maximum force required to compress the steamed groats to 50% strain increased from 50.7 N to 57.5 N as the tempering temperature was increased from 80 to 110°C. Tempering conditions also affected water absorption. A significantly higher moisture content was observed for kilned (18.9%) compared to unkilned (17.1%) groats, but otherwise had no effect on groat height, maximum force or final force after a 5 s relaxation time. Flakes were produced from the tempered groats using a laboratory flaking machine, using a roll gap of 0.4 mm. Apart from specific weight, flake properties were not influenced by kilning. Tempering conditions however had significant effects on the specific weight, thickness and water absorption of the flakes, as well as on the amount of fine material (<2 mm) produced during flaking. Flake strength correlated significantly with groat strength and flake thickness. Trial flaking at a commercial mill confirmed that groat temperature after tempering influenced water absorption. Variation in flake strength was observed , but at the groat temperatures required to inactivate lipase, it was rather small. Cold flaking of groats resulted in soft, floury flakes. The results presented in this thesis suggest that heating increased the adhesion between starch granules. This resulted in an increase in the stiffness and brittleness of the groat. Brittle fracture, rather than plastic flow, during flaking could result in flaws and cracks in the flake. These would be expected to increase water absorption. This was indeed observed as tempering temperature increased. Industrial trials, conducted with different groat temperatures, confirmed the main findings of the laboratory experiments. The approach used in the present study allowed the systematic study of the effect of interacting process parameters on product quality. There have been few scientific studies of oat processing, and these results can be used to understand the complex effects of process variables on flake quality. They also offer an insight into what happens as the oat groat is deformed into a flake.

Microfiltration in cheese and whey processing

Milk microfiltration (0.05-0.2 um) is a membrane separation technique which divides milk components into casein-enriched and native whey fractions. Hitherto the effect of intensive microfiltration including a diafiltration step for both cheese and whey processing has not been studied. The microfiltration performance of skimmed milk was studied with polymeric and ceramic MF membranes. The changes caused by decreased concentration of milk lactose, whey protein and ash content for cheese milk quality and ripening were studied. The effects of cheese milk modification on the milk coagulation properties, cheese recovery yield, cheese composition, ripening and sensory quality as well as on the whey recovery yield and composition by microfiltration were studied. The functional properties of whey protein concentrate from native whey were studied and the detailed composition of whey protein concentrate powders made from cheese wheys after cheese milk pretreatments such as high temperature heat treatment (HH), microfiltration (MF) and ultrafiltration (UF) were compared. The studied polymeric spiral wound microfiltration membranes had 38.5% lower energy consumption, 30.1% higher retention of whey proteins to milk retentate and 81.9% lower permeate flux values compared to ceramic membranes. All studied microfiltration membranes were able to separate main whey proteins from skimmed milk. The optimal lactose content of Emmental cheese milk exceeded 3.2% and reduction of whey proteins and ash content of cheese milk with high concentration factor (CF) values increased the rate of cheese ripening. Reduction of whey protein content in cheese milk increased the concentration of caseinomacropeptide (CMP) of total proteins in cheese whey. Reduction of milk whey protein, lactose and ash content reduces milk rennet clotting time and increased the firmness of the coagulum. Cheese yield calculated from raw milk to cheese was lower with microfiltrated milks due to native whey production. Amounts of a-lactalbumin (a-LA) and b-lactoglobulin (b-LG) were significantly higher in the reference whey, indicating that HH, MF and UF milk pretreatments decrease the amounts of these valuable whey proteins in whey. Even low CF values in milk microfiltration (CF 1.4) reduced nutritional value of cheese whey. From the point of view of utilization of milk components it would be beneficial if the amount of native whey and the CMP content of cheese whey could be maximized. Whey protein concentrate powders made of native whey had excellent functional properties and their detailed amino acid composition differed from those of cheese whey protein concentrate powders.

Folates in rye: Determination and enhancement by food processing

The average daily intake of folate, one of the B vitamins, falls below recommendations among the Finnish population. Bread and cereals are the main sources of folate, rye being the most significant single source. Processing is a prerequisite for the consumption of whole grain rye; however, little is known about the effect of processing on folates. Moreover, data on the bioavailability of endogenous cereal folates are scarce. The aim of this study was to examine the variation in as well as the effect of fermentation, germination, and thermal processes on folate contents in rye. Bioavailability of endogenous rye folates was investigated in a four-week human intervention study. One of the objectives throughout the work was to optimise and evaluate analytical methods for determining folate contents in cereals. Affinity chromatographic purification followed by high-performance liquid chromatography (HPLC) was a suitable method for analysing cereal products for folate vitamers, and microbiological assay with Lactobacillus rhamnosus reliably quantified the total folate. However, HPLC gave approximately 30% lower results than the microbiological assay. The folate content of rye was high and could be further increased by targeted processing. The vitamer distribution of whole grain rye was characterised by a large proportion of formylated vitamers followed by 5-methyltetrahydrofolate. In sourdough fermentation of rye, the studied yeasts synthesized and lactic acid bacteria mainly depleted folate. Two endogenous bacteria isolated from rye flour were found to produce folate during fermentation. Inclusion of baker s yeast in sourdough fermentation raised the folate level so that the bread could contain more folate than the flour it was made of. Germination markedly increased the folate content of rye, with particularly high folate concentrations in hypocotylar roots. Thermal treatments caused significant folate losses but the preceding germination compensated well for the losses. In the bioavailability study, moderate amounts of endogenous folates in the form of different rye products and orange juice incorporated in the diet improved the folate status among healthy adults. Endogenous folates from rye and orange juice showed similar bioavailability to folic acid from fortified white bread. In brief, it was shown that the folate content of rye can be enhanced manifold by optimising and combining food processing techniques. This offers some practical means to increase the daily intake of folate in a bioavailable form.

Enhanced processing of SPOT multispectral satellite imagery for environmental monitoring and modelling

The Taita Hills in southeastern Kenya form the northernmost part of Africa’s Eastern Arc Mountains, which have been identified by Conservation International as one of the top ten biodiversity hotspots on Earth. As with many areas of the developing world, over recent decades the Taita Hills have experienced significant population growth leading to associated major changes in land use and land cover (LULC), as well as escalating land degradation, particularly soil erosion. Multi-temporal medium resolution multispectral optical satellite data, such as imagery from the SPOT HRV, HRVIR, and HRG sensors, provides a valuable source of information for environmental monitoring and modelling at a landscape level at local and regional scales. However, utilization of multi-temporal SPOT data in quantitative remote sensing studies requires the removal of atmospheric effects and the derivation of surface reflectance factor. Furthermore, for areas of rugged terrain, such as the Taita Hills, topographic correction is necessary to derive comparable reflectance throughout a SPOT scene. Reliable monitoring of LULC change over time and modelling of land degradation and human population distribution and abundance are of crucial importance to sustainable development, natural resource management, biodiversity conservation, and understanding and mitigating climate change and its impacts. The main purpose of this thesis was to develop and validate enhanced processing of SPOT satellite imagery for use in environmental monitoring and modelling at a landscape level, in regions of the developing world with limited ancillary data availability. The Taita Hills formed the application study site, whilst the Helsinki metropolitan region was used as a control site for validation and assessment of the applied atmospheric correction techniques, where multiangular reflectance field measurements were taken and where horizontal visibility meteorological data concurrent with image acquisition were available. The proposed historical empirical line method (HELM) for absolute atmospheric correction was found to be the only applied technique that could derive surface reflectance factor within an RMSE of < 0.02 ps in the SPOT visible and near-infrared bands; an accuracy level identified as a benchmark for successful atmospheric correction. A multi-scale segmentation/object relationship modelling (MSS/ORM) approach was applied to map LULC in the Taita Hills from the multi-temporal SPOT imagery. This object-based procedure was shown to derive significant improvements over a uni-scale maximum-likelihood technique. The derived LULC data was used in combination with low cost GIS geospatial layers describing elevation, rainfall and soil type, to model degradation in the Taita Hills in the form of potential soil loss, utilizing the simple universal soil loss equation (USLE). Furthermore, human population distribution and abundance were modelled with satisfactory results using only SPOT and GIS derived data and non-Gaussian predictive modelling techniques. The SPOT derived LULC data was found to be unnecessary as a predictor because the first and second order image texture measurements had greater power to explain variation in dwelling unit occurrence and abundance. The ability of the procedures to be implemented locally in the developing world using low-cost or freely available data and software was considered. The techniques discussed in this thesis are considered equally applicable to other medium- and high-resolution optical satellite imagery, as well the utilized SPOT data.

Learning Nonlinear Visual Processing from Natural Images

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.