Dynamics of contour, object and face processing in the human visual cortex

The neural basis of visual perception can be understood only when the sequence of cortical activity underlying successful recognition is known. The early steps in this processing chain, from retina to the primary visual cortex, are highly local, and the perception of more complex shapes requires integration of the local information. In Study I of this thesis, the progression from local to global visual analysis was assessed by recording cortical magnetoencephalographic (MEG) responses to arrays of elements that either did or did not form global contours. The results demonstrated two spatially and temporally distinct stages of processing: The first, emerging 70 ms after stimulus onset around the calcarine sulcus, was sensitive to local features only, whereas the second, starting at 130 ms across the occipital and posterior parietal cortices, reflected the global configuration. To explore the links between cortical activity and visual recognition, Studies II III presented subjects with recognition tasks of varying levels of difficulty. The occipito-temporal responses from 150 ms onwards were closely linked to recognition performance, in contrast to the 100-ms mid-occipital responses. The averaged responses increased gradually as a function of recognition performance, and further analysis (Study III) showed the single response strengths to be graded as well. Study IV addressed the attention dependence of the different processing stages: Occipito-temporal responses peaking around 150 ms depended on the content of the visual field (faces vs. houses), whereas the later and more sustained activity was strongly modulated by the observers attention. Hemodynamic responses paralleled the pattern of the more sustained electrophysiological responses. Study V assessed the temporal processing capacity of the human object recognition system. Above sufficient luminance, contrast and size of the object, the processing speed was not limited by such low-level factors. Taken together, these studies demonstrate several distinct stages in the cortical activation sequence underlying the object recognition chain, reflecting the level of feature integration, difficulty of recognition, and direction of attention.

Social perception and cognition : processing of gestures, postures and facial expressions in the human brain

Humans are a social species with the internal capability to process social information from other humans. To understand others behavior and to react accordingly, it is necessary to infer their internal states, emotions and aims, which are conveyed by subtle nonverbal bodily cues such as postures, gestures, and facial expressions. This thesis investigates the brain functions underlying the processing of such social information. Studies I and II of this thesis explore the neural basis of perceiving pain from another person s facial expressions by means of functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). In Study I, observing another s facial expression of pain activated the affective pain system (previously associated with self-experienced pain) in accordance with the intensity of the observed expression. The strength of the response in anterior insula was also linked to the observer s empathic abilities. The cortical processing of facial pain expressions advanced from the visual to temporal-lobe areas at similar latencies (around 300 500 ms) to those previously shown for emotional expressions such as fear or disgust. Study III shows that perceiving a yawning face is associated with middle and posterior STS activity, and the contagiousness of a yawn correlates negatively with amygdalar activity. Study IV explored the brain correlates of interpreting social interaction between two members of the same species, in this case human and canine. Observing interaction engaged brain activity in very similar manner for both species. Moreover, the body and object sensitive brain areas of dog experts differentiated interaction from noninteraction in both humans and dogs whereas in the control subjects, similar differentiation occurred only for humans. Finally, Study V shows the engagement of the brain area associated with biological motion when exposed to the sounds produced by a single human being walking. However, more complex pattern of activation, with the walking sounds of several persons, suggests that as the social situation becomes more complex so does the brain response. Taken together, these studies demonstrate the roles of distinct cortical and subcortical brain regions in the perception and sharing of others internal states via facial and bodily gestures, and the connection of brain responses to behavioral attributes.

Half-occlusion processing in stereoscopic vision

This study investigated questions related to half-occlusion processing in human stereoscopic vision: (1) How does the depth location of a half-occluding figure affect the depth localization of adjacent monocular objects? (2) Is three-dimensional slant around vertical axis (geometric effect) affected by half-occlusion constraints? and (3) How the half-occlusion constraints and surface formation processes are manifested in stereoscopic capture? Our results showed that the depth localization of binocular objects affects the depth localization of discrete monocular objects. We also showed that the visual system has a preference for a frontoparallel surface interpretation if the half-occlusion configuration allows multiple interpretation alternatives. When the surface formation was constrained by textures, our results showed that a process of rematching spreading determines the resulting perception and that the spreading can be limited by illusory contours that support the presence of binocularly unmatched figures. The unmatched figures could be present, if the inducing figures producing the illusory surface contained binocular image differences that provided cues for quantitative da Vinci stereopsis. These findings provide evidence of the significant role of half-occlusions in stereoscopic processing.

Sensory auditory processing and intuitive sound detection : an investigation of musical experts and nonexperts

The auditory system can detect occasional changes (deviants) in acoustic regularities without the need for subjects to focus their attention on the sound material. Deviant detection is reflected in the elicitation of the mismatch negativity component (MMN) of the event-related potentials. In the studies presented in this thesis, the MMN is used to investigate the auditory abilities for detecting similarities and regularities in sound streams. To investigate the limits of these processes, professional musicians have been tested in some of the studies. The results show that auditory grouping is already more advanced in musicians than in nonmusicians and that the auditory system of musicians can, unlike that of nonmusicians, detect a numerical regularity of always four tones in a series. These results suggest that sensory auditory processing in musicians is not only a fine tuning of universal abilities, but is also qualitatively more advanced than in nonmusicians. In addition, the relationship between the auditory change-detection function and perception is examined. It is shown that, contrary to the generally accepted view, MMN elicitation does not necessarily correlate with perception. The outcome of the auditory change-detection function can be implicit and the implicit knowledge of the sound structure can, after training, be utilized for behaviorally correct intuitive sound detection. These results illustrate the automatic character of the sensory change detection function.

Cortical processing of musical pitch as reflected by behavioural and electrophysiological evidence

In a musical context, the pitch of sounds is encoded according to domain-general principles not confined to music or even to audition overall but common to other perceptual and cognitive processes (such as multiple pattern encoding and feature integration), and to domain-specific and culture-specific properties related to a particular musical system only (such as the pitch steps of the Western tonal system). The studies included in this thesis shed light on the processing stages during which pitch encoding occurs on the basis of both domain-general and music-specific properties, and elucidate the putative brain mechanisms underlying pitch-related music perception. Study I showed, in subjects without formal musical education, that the pitch and timbre of multiple sounds are integrated as unified object representations in sensory memory before attentional intervention. Similarly, multiple pattern pitches are simultaneously maintained in non-musicians' sensory memory (Study II). These findings demonstrate the degree of sophistication of pitch processing at the sensory memory stage, requiring neither attention nor any special expertise of the subjects. Furthermore, music- and culture-specific properties, such as the pitch steps of the equal-tempered musical scale, are automatically discriminated in sensory memory even by subjects without formal musical education (Studies III and IV). The cognitive processing of pitch according to culture-specific musical-scale schemata hence occurs as early as at the sensory-memory stage of pitch analysis. Exposure and cortical plasticity seem to be involved in musical pitch encoding. For instance, after only one hour of laboratory training, the neural representations of pitch in the auditory cortex are altered (Study V). However, faulty brain mechanisms for attentive processing of fine-grained pitch steps lead to inborn deficits in music perception and recognition such as those encountered in congenital amusia (Study VI). These findings suggest that predispositions for exact pitch-step discrimination together with long-term exposure to music govern the acquisition of the automatized schematic knowledge of the music of a particular culture that even non-musicians possess.

Neurochemical regulation of auditory information processing studied with EEG/MEG: application to schizophrenia

Cognitive impairments of attention, memory and executive functions are a fundamental feature of the pathophysiology of schizophrenia. The neurophysiological and neurochemical changes in the auditory cortex are shown to underlie cognitive impairmentsin schizophrenia patients. Functional state of the neural substrate of auditory information processing could be objectively and non-invasively probed with auditory event-related potentials (ERPs) and event- related fields (ERFs). In the current work, we explored the neurochemical effect on the neural origins of auditory information processing in relation to schizophrenia. By means of ERPs/ERFs we aimed to determine how neural substrates of auditory information processing are modulated by antipsychotic medication in schizophrenia spectrum patients (Studies I, II) and by neuropharmacological challenges in healthy human subjects (Studies III, IV). First, with auditory ERPs we investigated the effects of olanzapine (Study I) and risperidone (Study II) in a group of patients with schizophrenia spectrum disorders. After 2 and 4 weeks of treatment, olanzapine has no significant effects on mismatch negativity(MMN) and P300, which, as it has been suggested, respectively reflect preattentive and attention-dependent information processing. After 2 weeks of treatment, risperidone has no significant effect on P300, however risperidone reduces P200 amplitude. This latter effect of risperidone on neural resources responsible for P200 generation could be partly explained through the action of dopamine. Subsequently, we used simultaneous EEG/MEG to investigate the effects of memantine (Study III) and methylphenidate (Study IV) in healthy subjects. We found that memantine modulates MMN response without changing other ERP components. This could be interpreted as being due to the possible influence of memantine through the NMDA receptors on auditory change- detection mechanism, with processing of auditory stimuli remaining otherwise unchanged. Further, we found that methylphenidate does not modulate the MMN response. This finding could indicate no association between catecholaminergic activities and electrophysiological measures of preattentive auditory discrimination processes reflected in the MMN. However, methylphenidate decreases the P200 amplitudes. This could be interpreted as a modulation of auditory information processing reflected in P200 by dopaminergic and noradrenergic systems. Taken together, our set of studies indicates a complex pattern of neurochemical influences produced by the antipsychotic drugs in the neural substrate of auditory information processing in patients with schizophrenia spectrum disorders and by the pharmacological challenges in healthy subjects studied with ERPs and ERFs.

Integrated semantic processing of complex pictures and spoken sentences - Evidence from event-related potentials

It has been suggested that semantic information processing is modularized according to the input form (e.g., visual, verbal, non-verbal sound). A great deal of research has concentrated on detecting a separate verbal module. Also, it has traditionally been assumed in linguistics that the meaning of a single clause is computed before integration to a wider context. Recent research has called these views into question. The present study explored whether it is reasonable to assume separate verbal and nonverbal semantic systems in the light of the evidence from event-related potentials (ERPs). The study also provided information on whether the context influences processing of a single clause before the local meaning is computed. The focus was on an ERP called N400. Its amplitude is assumed to reflect the effort required to integrate an item to the preceding context. For instance, if a word is anomalous in its context, it will elicit a larger N400. N400 has been observed in experiments using both verbal and nonverbal stimuli. Contents of a single sentence were not hypothesized to influence the N400 amplitude. Only the combined contents of the sentence and the picture were hypothesized to influence the N400. The subjects (n = 17) viewed pictures on a computer screen while hearing sentences through headphones. Their task was to judge the congruency of the picture and the sentence. There were four conditions: 1) the picture and the sentence were congruent and sensible, 2) the sentence and the picture were congruent, but the sentence ended anomalously, 3) the picture and the sentence were incongruent but sensible, 4) the picture and the sentence were incongruent and anomalous. Stimuli from the four conditions were presented in a semi-randomized sequence. Their electroencephalography was simultaneously recorded. ERPs were computed for the four conditions. The amplitude of the N400 effect was largest in the incongruent sentence-picture -pairs. The anomalously ending sentences did not elicit a larger N400 than the sensible sentences. The results suggest that there is no separate verbal semantic system, and that the meaning of a single clause is not processed independent of the context.

Microfiltration in cheese and whey processing

Milk microfiltration (0.05-0.2 um) is a membrane separation technique which divides milk components into casein-enriched and native whey fractions. Hitherto the effect of intensive microfiltration including a diafiltration step for both cheese and whey processing has not been studied. The microfiltration performance of skimmed milk was studied with polymeric and ceramic MF membranes. The changes caused by decreased concentration of milk lactose, whey protein and ash content for cheese milk quality and ripening were studied. The effects of cheese milk modification on the milk coagulation properties, cheese recovery yield, cheese composition, ripening and sensory quality as well as on the whey recovery yield and composition by microfiltration were studied. The functional properties of whey protein concentrate from native whey were studied and the detailed composition of whey protein concentrate powders made from cheese wheys after cheese milk pretreatments such as high temperature heat treatment (HH), microfiltration (MF) and ultrafiltration (UF) were compared. The studied polymeric spiral wound microfiltration membranes had 38.5% lower energy consumption, 30.1% higher retention of whey proteins to milk retentate and 81.9% lower permeate flux values compared to ceramic membranes. All studied microfiltration membranes were able to separate main whey proteins from skimmed milk. The optimal lactose content of Emmental cheese milk exceeded 3.2% and reduction of whey proteins and ash content of cheese milk with high concentration factor (CF) values increased the rate of cheese ripening. Reduction of whey protein content in cheese milk increased the concentration of caseinomacropeptide (CMP) of total proteins in cheese whey. Reduction of milk whey protein, lactose and ash content reduces milk rennet clotting time and increased the firmness of the coagulum. Cheese yield calculated from raw milk to cheese was lower with microfiltrated milks due to native whey production. Amounts of a-lactalbumin (a-LA) and b-lactoglobulin (b-LG) were significantly higher in the reference whey, indicating that HH, MF and UF milk pretreatments decrease the amounts of these valuable whey proteins in whey. Even low CF values in milk microfiltration (CF 1.4) reduced nutritional value of cheese whey. From the point of view of utilization of milk components it would be beneficial if the amount of native whey and the CMP content of cheese whey could be maximized. Whey protein concentrate powders made of native whey had excellent functional properties and their detailed amino acid composition differed from those of cheese whey protein concentrate powders.

Learning Nonlinear Visual Processing from Natural Images

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.

Tactile processing in human somatosensory and auditory cortices

Tactile sensation plays an important role in everyday life. While the somatosensory system has been studied extensively, the majority of information has come from studies using animal models. Recent development of high-resolution anatomical and functional imaging techniques has enabled the non-invasive study of human somatosensory cortex and thalamus. This thesis provides new insights into the functional organization of the human brain areas involved in tactile processing using magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). The thesis also demonstrates certain optimizations of MEG and fMRI methods. Tactile digit stimulation elicited stimulus-specific responses in a number of brain areas. Contralateral activation was observed in somatosensory thalamus (Study II), primary somatosensory cortex (SI; I, III, IV), and post-auditory belt area (III). Bilateral activation was observed in secondary somatosensory cortex (SII; II, III, IV). Ipsilateral activation was found in the post-central gyrus (area 2 of SI cortex; IV). In addition, phasic deactivation was observed within ipsilateral SI cortex and bilateral primary motor cortex (IV). Detailed investigation of the tactile responses demonstrated that the arrangement of distal-proximal finger representations in area 3b of SI in humans is similar to that found in monkeys (I). An optimized MEG approach was sufficient to resolve such fine detail in functional organization. The SII region appeared to contain double representations for fingers and toes (II). The detection of activations in the SII region and thalamus improved at the individual and group levels when cardiac-gated fMRI was used (II). Better detection of body part representations at the individual level is an important improvement, because identification of individual representations is crucial for studying brain plasticity in somatosensory areas. The posterior auditory belt area demonstrated responses to both auditory and tactile stimuli (III), implicating this area as a physiological substrate for the auditory-tactile interaction observed in earlier psychophysical studies. Comparison of different smoothing parameters (III) demonstrated that proper evaluation of co-activation should be based on individual subject analysis with minimal or no smoothing. Tactile input consistently influenced area 3b of the human ipsilateral SI cortex (IV). The observed phasic negative fMRI response is proposed to result from interhemispheric inhibition via trans-callosal connections. This thesis contributes to a growing body of human data suggesting that processing of tactile stimuli involves multiple brain areas, with different spatial patterns of cortical activation for different stimuli.

Information Technology in Construction – Domain Definition and Research Issues

This article discusses the scope of research on the application of information technology in construction (ITC). A model of the information and material activities which together constitute the construction process is presented, using the IDEF0 activity modelling methodology. Information technology is defined to include all kinds of technology used for the storage, transfer and manipulation of information, thus also including devices such as copying machines, faxes and mobile phones. Using the model the domain of ITC research is defined as the use of information technology to facilitate and re-engineer the information process component of construction. Developments during the last decades in IT use in construction is discussed against a background of a simplified model of generic information processing tasks. The scope of ITC is compared with the scopes of research in related areas such as design methodology, construction management and facilities management. Health care is proposed as an interesting alternative (to the often used car manufacturing industry), as an IT application domain to compare with. Some of the key areas of ITC research in recent years; expert systems, company IT strategies, and product modelling are shortly discussed. The article finishes with a short discussion of the problems of applying standard scientific methodology in ITC research, in particular in product model research.

Learning Challenges in Biogas Production for Sustainability : An activity theoretical study of a network from a swine industry chain

This study is about the challenges of learning in the creation and implementation of new sustainable technologies. The system of biogas production in the Programme of Sustainable Swine Production (3S Programme) conducted by the Sadia food processing company in Santa Catarina State, Brazil, is used as a case example for exploring the challenges, possibilities and obstacles of learning in the use of biogas production as a way to increase the environmental sustainability of swine production. The aim is to contribute to the discussion about the possibilities of developing systems of biogas production for sustainability (BPfS). In the study I develop hypotheses concerning the central challenges and possibilities for developing systems of BPfS in three phases. First, I construct a model of the network of activities involved in the BP for sustainability in the case study. Next, I construct a) an idealised model of the historically evolved concepts of BPfS through an analysis of the development of forms of BP and b) a hypothesis of the current central contradictions within and between the activity systems involved in BP for sustainability in the case study. This hypothesis is further developed through two actual empirical analyses: an analysis of the actors senses in taking part in the system, and an analysis of the disturbance processes in the implementation and operation of the BP system in the 3S Programme. The historical analysis shows that BP for sustainability in the 3S Programme emerged as a feasible solution for the contradiction between environmental protection and concentration, intensification and specialisation in swine production. This contradiction created a threat to the supply of swine to the food processing company. In the food production activity, the contradiction was expressed as a contradiction between the desire of the company to become a sustainable company and the situation in the outsourced farms. For the swine producers the contradiction was expressed between the contradictory rules in which the market exerted pressure which pushed for continual increases in scale, specialisation and concentration to keep the production economically viable, while the environmental rules imposed a limit to this expansion. Although the observed disturbances in the biogas system seemed to be merely technical and localised within the farms, the analysis proposed that these disturbances were formed in and between the activity systems involved in the network of BPfS during the implementation. The disturbances observed could be explained by four contradictions: a) contradictions between the new, more expanded activity of sustainable swine production and the old activity, b) a contradiction between the concept of BP for carbon credits and BP for local use in the BPfS that was implemented, c) contradictions between the new UNFCCC1 methodology for applying for carbon credits and the small size of the farms, and d) between the technologies of biogas use and burning available in the market and the small size of the farms. The main finding of this study relates to the zone of proximal development (ZPD) of the BPfS in Sadia food production chain. The model is first developed as a general model of concepts of BPfS and further developed here to the specific case of the BPfS in the 3S Programme. The model is composed of two developmental dimensions: societal and functional integration. The dimension of societal integration refers to the level of integration with other activities outside the farm. At one extreme, biogas production is self-sufficient and highly independent and the products of BP are consumed within the farm, while at the other extreme BP is highly integrated in markets and networks of collaboration, and BP products are exchanged within the markets. The dimension of functional integration refers to the level of integration between products and production processes so that economies of scope can be achieved by combining several functions using the same utility. At one extreme, BP is specialised in only one product, which allows achieving economies of scale, while at the other extreme there is an integrated production in which several biogas products are produced in order to maximise the outcomes from the BP system. The analysis suggests that BP is moving towards a societal integration, towards the market and towards a functional integration in which several biogas products are combined. The model is a hypothesis to be further tested through interventions by collectively constructing the new proposed concept of BPfS. Another important contribution of this study refers to the concept of the learning challenge. Three central learning challenges for developing a sustainable system of BP in the 3S Programme were identified: 1) the development of cheaper and more practical technologies of burning and measuring the gas, as well as the reduction of costs of the process of certification, 2) the development of new ways of using biogas within farms, and 3) the creation of new local markets and networks for selling BP products. One general learning challenge is to find more varied and synergic ways of using BP products than solely for the production of carbon credits. Both the model of the ZPD of BPfS and the identified learning challenges could be used as learning tools to facilitate the development of biogas production systems. The proposed model of the ZPD could be used to analyse different types of agricultural activities that face a similar contradiction. The findings could be used in interventions to help actors to find their own expansive actions and developmental projects for change. Rather than proposing a standardised best concept of BPfS, the idea of these learning tools is to facilitate the analysis of local situations and to help actors to make their activities more sustainable.

Neurocognitive processing of inflected and derived words

The representation of morphologically complex words in the mental lexicon and their neurocognitive processing has been a vigorously debated topic in psycholinguistics and the cognitive neuroscience of language. This thesis investigates the effect of stimulus modality on morphological processing, the spatiotemporal dynamics of the neural processing of inflected (e.g., work+ed ) and derived (e.g., work+er ) words and their interaction, using the Finnish language. Overall, the results suggest that the constituent morphemes of isolated written and spoken inflected words are accessed separately, whereas spoken derived words activate both their full form and the constituent morphemes. The processing of both spoken and written inflected words elicited larger N400 responses than monomorphemic words (Study I), whereas the responses to spoken derived words did not differ from those to monomorphemic words (Study IV). Spoken inflected words elicited a larger left-lateralized negativity and greater source strengths in the left temporal cortices than derived words (Study IV). Thus, the results suggest different cortical processing for derived and inflected words. Moreover, the neural mechanisms underlying inflection and derivation seem to be not only different, but also independent as indexed by the linear summation of the responses to derived and inflected stimuli in a combined (derivation+inflection) condition (Study III). Furthermore, the processing of meaningless, spoken derived pseudowords was more difficult than for existing derived words, indexed by a larger N400-type effect for the pseudowords. However, no differences were observed between meaningful derived pseudowords and existing derived words (Study II). The results of Study II suggest that semantic compatibility between morphemes seems to have a crucial role in a successful morphological analysis. As a methodological note, time-locking the auditory event-related potentials/fields (ERP/ERF) to the suffix onset revealed the processes related to morphological analysis more precisely (Studies II and IV), which also enables comparison of the neural processes in different modalities (Study I).

Linguistic and perceptual processing of communicative cues in Asperger Syndrome

Asperger Syndrome (AS) belongs to autism spectrum disorders where both verbal and non-verbal communication difficulties are at the core of the impairment. Social communication requires a complex use of affective, linguistic-cognitive and perceptual processes. In the four studies included in the current thesis, some of the linguistic and perceptual factors that are important for face-to-face communication were studied using behavioural methods. In all four studies the results obtained from individuals with AS were compared with typically developed age, gender and IQ matched controls. First, the language skills of school-aged children were characterized in detail with standardized tests that measured different aspects of receptive and expressive language (Study I). The children with AS were found to be worse than the controls in following complex verbal instructions. Next, the visual perception of facial expressions of emotion with varying degrees of visual detail was examined (Study II). Adults with AS were found to have impaired recognition of facial expressions on the basis of very low spatial frequencies which are important for processing global information. Following that, multisensory perception was investigated by looking at audiovisual speech perception (Studies III and IV). Adults with AS were found to perceive audiovisual speech qualitatively differently from typically developed adults, although both groups were equally accurate in recognizing auditory and visual speech presented alone. Finally, the effect of attention on audiovisual speech perception was studied by registering eye gaze behaviour (Study III) and by studying the voluntary control of visual attention (Study IV). The groups did not differ in eye gaze behaviour or in the voluntary control of visual attention. The results of the study series demonstrate that many factors underpinning face-to-face social communication are atypical in AS. In contrast with previous assumptions about intact language abilities, the current results show that children with AS have difficulties in understanding complex verbal instructions. Furthermore, the study makes clear that deviations in the perception of global features in faces expressing emotions as well as in the multisensory perception of speech are likely to harm face-to-face social communication.