From inference to affordance : the problem of visual depth-perception in the optical writings of Descartes, Berkeley, and Gibson

This thesis explores the debate and issues regarding the status of visual ;,iferellces in the optical writings of Rene Descartes, George Berkeley and James 1. Gibson. It gathers arguments from across their works and synthesizes an account of visual depthperception that accurately reflects the larger, metaphysical implications of their philosophical theories. Chapters 1 and 2 address the Cartesian and Berkelean theories of depth-perception, respectively. For Descartes and Berkeley the debate can be put in the following way: How is it possible that we experience objects as appearing outside of us, at various distances, if objects appear inside of us, in the representations of the individual's mind? Thus, the Descartes-Berkeley component of the debate takes place exclusively within a representationalist setting. Representational theories of depthperception are rooted in the scientific discovery that objects project a merely twodimensional patchwork of forms on the retina. I call this the "flat image" problem. This poses the problem of depth in terms of a difference between two- and three-dimensional orders (i.e., a gap to be bridged by one inferential procedure or another). Chapter 3 addresses Gibson's ecological response to the debate. Gibson argues that the perceiver cannot be flattened out into a passive, two-dimensional sensory surface. Perception is possible precisely because the body and the environment already have depth. Accordingly, the problem cannot be reduced to a gap between two- and threedimensional givens, a gap crossed with a projective geometry. The crucial difference is not one of a dimensional degree. Chapter 3 explores this theme and attempts to excavate the empirical and philosophical suppositions that lead Descartes and Berkeley to their respective theories of indirect perception. Gibson argues that the notion of visual inference, which is necessary to substantiate representational theories of indirect perception, is highly problematic. To elucidate this point, the thesis steps into the representationalist tradition, in order to show that problems that arise within it demand a tum toward Gibson's information-based doctrine of ecological specificity (which is to say, the theory of direct perception). Chapter 3 concludes with a careful examination of Gibsonian affordallces as the sole objects of direct perceptual experience. The final section provides an account of affordances that locates the moving, perceiving body at the heart of the experience of depth; an experience which emerges in the dynamical structures that cross the body and the world.

Visual information processing during conscious and non-conscious face perception

Les stimuli naturels projetés sur nos rétines nous fournissent de l’information visuelle riche. Cette information varie le long de propriétés de « bas niveau » telles que la luminance, le contraste, et les fréquences spatiales. Alors qu’une partie de cette information atteint notre conscience, une autre partie est traitée dans le cerveau sans que nous en soyons conscients. Les propriétés de l’information influençant l’activité cérébrale et le comportement de manière consciente versus non-consciente demeurent toutefois peu connues. Cette question a été examinée dans les deux derniers articles de la présente thèse, en exploitant les techniques psychophysiques développées dans les deux premiers articles. Le premier article présente la boîte à outils SHINE (spectrum, histogram, and intensity normalization and equalization), développée afin de permettre le contrôle des propriétés de bas niveau de l'image dans MATLAB. Le deuxième article décrit et valide la technique dite des bulles fréquentielles, qui a été utilisée tout au long des études de cette thèse pour révéler les fréquences spatiales utilisées dans diverses tâches de perception des visages. Cette technique offre les avantages d’une haute résolution au niveau des fréquences spatiales ainsi que d’un faible biais expérimental. Le troisième et le quatrième article portent sur le traitement des fréquences spatiales en fonction de la conscience. Dans le premier cas, la méthode des bulles fréquentielles a été utilisée avec l'amorçage par répétition masquée dans le but d’identifier les fréquences spatiales corrélées avec les réponses comportementales des observateurs lors de la perception du genre de visages présentés de façon consciente versus non-consciente. Les résultats montrent que les mêmes fréquences spatiales influencent de façon significative les temps de réponse dans les deux conditions de conscience, mais dans des sens opposés. Dans le dernier article, la méthode des bulles fréquentielles a été combinée à des enregistrements intracrâniens et au Continuous Flash Suppression (Tsuchiya & Koch, 2005), dans le but de cartographier les fréquences spatiales qui modulent l'activation de structures spécifiques du cerveau (l'insula et l'amygdale) lors de la perception consciente versus non-consciente des expressions faciales émotionnelles. Dans les deux régions, les résultats montrent que la perception non-consciente s'effectue plus rapidement et s’appuie davantage sur les basses fréquences spatiales que la perception consciente. La contribution de cette thèse est donc double. D’une part, des contributions méthodologiques à la recherche en perception visuelle sont apportées par l'introduction de la boîte à outils SHINE ainsi que de la technique des bulles fréquentielles. D’autre part, des indications sur les « corrélats de la conscience » sont fournies à l’aide de deux approches différentes.

Influence of silence and attention on tinnitus perception

OBJECTIVE: The purpose of this study was to study the effect of attention and sustained silence on the emergence of auditory phantom perception in normal-hearing adults. STUDY DESIGN: Cross-sectional survey. SUBJECTS AND METHODS: While sitting in a sound booth, 66 volunteers (age range. 18-65; mean age, 37.3) performed 3 experiments of 5 minutes each, consecutively and randomly presented. Two deviated attention from auditory system (Hanoi and visual attention experiments). and 1 drove attention to the auditory system (auditory attention). After each experiment, participants were asked about their auditory and visual perception. No sound or light change was given at any moment. RESULTS: Of the participants, 19.7% experienced tinnitus during Hanoi, 45.5% during visual attention, and 68.2% during auditory attention experiment. with no significant differences for studied variables. CONCLUSION: Tinnitus-like perceptions may occur in a non-clinical population in a silent environment. Concomitant auditory attention plays an important role on the emergence of tinnitus. (C) 2008 American Academy of Otolaryngology-Head and Neck Surgery Foundation. All rights reserved.

Visual Biology of Hawaiian Coral Reef Fishes. I. Ocular Transmission and Visual Pigments

The visual biology of Hawaiian reef fishes was explored by examining their eyes for spectral sensitivity of their visual pigments and for transmission of light through the ocular media to the retina. The spectral absorption curves for the visual pigments of 38 species of Hawaiian fish were recorded using microspectrophotometry. The peak absorption wavelength (lambda(max)) of the rods varied from 477-502 nm and the lambda(max) of individual species conformed closely to values for the same species previously reported using a whole retina extraction procedure. The visual pigments of single cone photoreceptors were categorized, dependent on their lambda(max)-values, as ultraviolet (347-376 nm), violet (398-431 nm) or blue (439-498 nm) sensitive cones. Eight species possessed ultraviolet-sensitive cones and 14 species violet-sensitive cones. Thus, 47% of the species examined displayed photosensitivity to the short-wavelength region of the spectrum. Both identical and nonidentical paired and double cones were found with blue sensitivity or green absorption peaks (> 500 nm). Spectrophotometry of the lens, cornea, and humors for 195 species from 49 families found that the spectral composition of the light transmitted to the retina was most often limited by the lens (73% of species examined). Except for two unusual species with humor-limited eyes, Acanthocybium solandri (Scombridae) and the priacanthid fish, Heteropriacanthus cruentatus, the remainder had corneal-limited eyes. The wavelength at which 50% of the light was blocked (T50) was classified according to a system modified from Douglas and McGuigan (1989) as Type I, T50 < = 355 nm, (32 species); Type IIa, 355 < T50 < = 380 nm (30 species); Type IIb, 380 < T50 405 nm (84 species). Possession of UV-transmitting ocular media follows both taxonomic and functional lines and, if the ecology of the species is considered, is correlated with the short-wavelength visual pigments found in the species. Three types of short-wavelength vision in fishes are hypothesized: UV-sensitive, UV-specialized, and violet-specialized. UV-sensitive eyes lack UV blockers (Type I and IIa) and can sense UV light with the secondary absorption peak or beta peak of their longer wavelength visual pigments but do not possess specialized UV receptor cells and, therefore, probably lack UV hue discrimination. UV-specialized eyes allow transmission of UV light to the retina (Type I and IIa) and also possess UV-sensitive cone receptors with peak absorption between 300 and 400 nm. Given the appropriate perceptual mechanisms, these species could possess true UV-color vision and hue discrimination. Violet-specialized eyes extend into Type IIb eyes and possess violet-sensitive cone cells. UV-sensitive eyes are found throughout the fishes from at least two species of sharks to modern bony fishes. Eyes with specialized short-wavelength sensitivity are common in tropical reef fishes and must be taken into consideration when performing research involving the visual perception systems of these fishes. Because most glass and plastics are UV-opaque, great care must be taken to ensure that aquarium dividers, specimen holding containers, etc., are UV-transparent or at least to report the types of materials in use.

Perceção visual na avaliação diagnóstica em mamografia: uma revisão sistemática

Introdução – Na avaliação diagnóstica em mamografia, o desempenho do radiologista pode estar sujeito a erros de diagnóstico. Objetivo – Descrever a importância da perceção visual na análise da mamografia, identificando os principais fatores que contribuem para a perceção visual do radiologista e que condicionam a acuidade diagnóstica. Metodologia – Estudo descritivo baseado numa revisão sistemática de literatura através da PubMed e da Science Direct. Foram incluídos 42 artigos que respeitavam, pelo menos, um dos critérios de inclusão no estudo. Para a seleção das referências foi utilizada a metodologia PRISMA, constituída por 4 fases: identificação, seleção preliminar, elegibilidade e estudos incluídos. Resultados – Na avaliação diagnóstica em mamografia, a perceção visual está intimamente relacionada com: 1) diferentes parâmetros visuais e da motilidade ocular (acuidade visual, sensibilidade ao contraste e à luminância e movimentos oculares); 2) com condições de visualização de uma imagem (iluminância da sala e luminância do monitor); e 3) fadiga ocular provocada pela observação diária consecutiva de imagens. Conclusões – A perceção visual pode ser influenciada por 3 categorias de erros observados: erros de pesquisa (lesões não são fixadas pela fóvea), erros de reconhecimento (lesões fixadas, mas não durante o tempo suficiente) e erros de decisão (lesões fixadas, mas não identificadas como suspeitas). Os estudos analisados sobre perceção visual, atenção visual e estratégia visual, bem como os estudos sobre condições de visualização não caracterizam a função visual dos observadores. Para uma avaliação correta da perceção visual em mamografia deverão ser efetuados estudos que correlacionem a função visual com a qualidade diagnóstica. ABSTRACT - Introduction – Diagnostic evaluation in mammography could be influenced by the radiologist performance that could be under diagnostic errors. Aims – To describe the importance of radiologist visual perception in mammographic diagnostic evaluation and to identify the main factors that contribute to diagnostic accuracy. Methods – In this systematic review 42 references were included based on inclusion criteria (PubMed and Science Direct). PRISMA method was used to select the references following 4 steps: identification, screening, eligibility and included references. Results – Visual perception in mammography diagnostic evaluation is related with: 1) visual parameters and ocular motility (visual acuity, contrast sensitivity and luminance and ocular movements); 2) image visualization environment (room iluminance and monitor luminance); and 3) eyestrain caused by image daily consecutive observation. Conclusions – Visual perception can be influenced by three errors categories: search errors (lesions are never looked at with high-resolution foveal vision), recognition errors (lesions are looked at, but not long enough to detect or recognize) and decision errors (lesions are looked at for long periods of time but are still missed). The reviewed studies concerning visual perception, visual attention, visual strategies and image visualization environment do not describe observer’s visual function. An accurate evaluation of visual perception in mammography must include visual function analysis.

O treino da perceção visual e da consciência fonológica como promotor da fluência da leitura num aluno do 2º ciclo do ensino básico

Aprender a ler é um dos maiores desafios que as crianças enfrentam quando entram para a escola. A dificuldade no domínio do código alfabético, nos níveis da consciência fonológica e a falta de fluência na leitura são fatores que interferem em larga escala na aprendizagem global dos alunos. Habilitar um aluno para a prática da leitura é um estímulo que tem vindo a dar origem a várias investigações e intervenções no campo da educação. Este projeto descreve dois programas de treino: “Programa de treino da percepção Visual” e “Programa de promoção do desenvolvimento da consciência fonológica”, num aluno do 2º ciclo do ensino básico com dificuldade de fluência na leitura, ao longo de quinze aulas de 90 minutos. No que respeita aos resultados do primeiro estudo, que teve por base o “Programa de treino da percepção visual”, não foram encontradas diferenças relevantes quanto ao seu efeito na fluência da leitura do aluno. No entanto, no segundo estudo, que se centrou na aplicação do “Programa de promoção do desenvolvimento da consciência fonológica” em complemento com o “Programa de treino da percepção visual”, mostrou que o aluno ficou mais fluente na leitura diminuindo o número de erros de precisão (substituições, omissões, inversões, adições e erros complexos). Assim, sugere-se uma monotorização sistemática das aprendizagens dos alunos para que as intervenções possam ser cada vez mais precoces e direcionadas para as suas necessidades.

Contributos das TIC para as Necessidades Educativas Especiais - Treino da Perceção Visual

As novas tecnologias de informação e comunicação vêm introduzir novas abordagens e orientações nas relações das pessoas no mundo global, ao mesmo tempo que vêm redefinir paradigmas diversos em várias áreas disciplinares, nomeadamente nas áreas da saúde e da educação. A massificação das tecnologias vem assim aproximar a informação e o conhecimento das pessoas e a sua utilização no apoio aos processos de ensino e de aprendizagem dos alunos com necessidades educativas especiais e dificuldades de aprendizagem específicas tem vindo a ser discutida cada vez mais com uma maior relevância. Pretende-se através da presente dissertação contribuir com soluções que apoiem a desmaterialização dos processos e desburocratização dos relacionamentos entres os diversos intervenientes num processo de terapia de uma criança. Apesar de poder ser extensível a outros cenários, o projeto desenvolvido aplica-se ao caso concreto de crianças com necessidades educativas especiais e dificuldades de aprendizagem específicas, motivadas por dificuldades de perceção visual, que obrigam, além da adoção de programas de treino, como jogos interativos, a um acompanhamento integrado de todos os intervenientes na sua terapia: terapeutas, professores, pais e educadores, assistentes sociais entre outros, tornando-se fundamental a utilização de uma plataforma universal que permita a troca e sistematização de informações. Inicialmente foi necessário um enquadramento desta temática por via da leitura, pesquisa e reuniões com elementos ligados a diferentes áreas que intervém no tratamento de necessidades educativas especiais. Depois de uma pesquisa bibliográfica inicial sobre as dimensões a explorar, recolheu-se informação sobre estudos e projetos desenvolvidos para apoiar esta área, e desenvolveu-se um projeto destinado a contribuir para o trabalho positivo de todos os profissionais que se dedicam à terapia/tratamento das crianças com necessidades educativas especiais, particularmente derivadas de dificuldades de perceção visual. Posteriormente, realizou-se a avaliação do protótipo de forma a validar o real contributo do sistema na melhoria da comunicação e partilha de informação entre todos os intervenientes no processo de terapia de crianças com necessidades educativas especiais e da utilização das tecnologias interativas no treino da perceção visual. Com vista a uma futura implementação de um recurso educativo deste âmbito, foram também recolhidos os pontos negativos e sugestões de melhoria a incorporar. Em suma, este trabalho valida os contributos das TIC, e deste sistema em particular, na relação dos intervenientes num processo de terapia interdisciplinar e no treino da perceção visual.

The Rank Lecture: Has imaging told us anything new about human visual organisation?

The functional architecture of the occipital cortex is being studied with increasing detail. Functional and structural MR based imaging are altering views about the organisation of the human visual system. Recent advances have ranged from comparative studies with non-human primates to predictive scanning. The latter multivariate technique describes with sub-voxel resolution patterns of activity that are characteristic of specific visual experiences. One can deduce what a subject experienced visually from the pattern of cortical activity recorded. The challenge for the future is to understand visual functions in terms of cerebral computations at a mesoscopic level of description and to relate this information to electrophysiology. The principal medical application of this new knowledge has focused to a large extent on plasticity and the capacity for functional reorganisation. Crossmodality visual-sensory interactions and cross-correlations between visual and other cerebral areas in the resting state are areas of considerable current interest. The lecture will review findings over the last two decades and reflect on possible roles for imaging studies in the future.

Visual representation of food in the brain : effects of repeated exposure and gut hormone secretion

The thesis at hand is concerned with the spatio-temporal brain mechanisms of visual food perception as investigated by electrical neuroimaging. Due to the increasing prevalence of obesity and its associated challenges for public health care, there is a need to better understand behavioral and brain processes underlying food perception and food-based decision-making. The first study (Study A) of this thesis was concerned with the role of repeated exposure to visual food cues. In our everyday lives we constantly and repeatedly encounter food and these exposures influence our food choices and preferences. In Study A, we therefore applied electrical neuroimaging analyses of visual evoked potentials to investigate the spatio-temporal brain dynamics linked to the repeated viewing of high- and low-energy food cues (published manuscript: "The role of energetic value in dynamic brain response adaptation during repeated food image viewing" (Lietti et al., 2012)). In this study, we found that repetitions differentially affect behavioral and brain mechanisms when high-energy, as opposed to low-energy foods and non-food control objects, were viewed. The representation of high-energy food remained invariant between initial and repeated exposures indicating that the sight of high-energy dense food induces less behavioral and neural adaptation than the sight of low-energy food and non-food control objects. We discuss this finding in the context of the higher salience (due to greater motivation and higher reward or hedonic valuation) of energy- dense food that likely generates a more mnemonically stable representation. In turn, this more invariant representation of energy-dense food is supposed to (partially) explain why these foods are over-consumed despite of detrimental health consequences. In Study Β we investigated food responsiveness in patients who had undergone Roux-en-Y gastric bypass surgery to overcome excessive obesity. This type of gastric bypass surgery is not only known to alter food appreciation, but also the secretion patterns of adipokines and gut peptides. Study Β aimed at a comprehensive and interdisciplinary investigation of differences along the gut-brain axis in bypass-operated patients as opposed to weight-matched non-operated controls. On the one hand, the spatio-temporal brain dynamics to the visual perception of high- vs. low-energy foods under differing states of motivation towards food intake (i.e. pre- and post-prandial) were assessed and compared between groups. On the other hand, peripheral gut hormone measures were taken in pre- and post-prandial nutrition state and compared between groups. In order to evaluate alterations in the responsiveness along the gut-brain-axis related to gastric bypass surgery, correlations between both measures were compared between both participant groups. The results revealed that Roux-en- Y gastric bypass surgery alters the spatio-temporal brain dynamics to the perception of high- and low-energy food cues, as well as the responsiveness along the gut-brain-axis. The potential role of these response alterations is discussed in relation to previously observed changes in physiological factors and food intake behavior post-Roux-en-Y gastric bypass surgery. By doing so, we highlight potential behavioral, neural and endocrine (i.e. gut hormone) targets for the future development of intervention strategies for deviant eating behavior and obesity. Together, the studies showed that the visual representation of foods in the brain is plastic and that modulations in neural activity are already noted at early stages of visual processing. Different factors of influence such as a repeated exposure, Roux-en-Y gastric bypass surgery, motivation (nutrition state), as well as the energy density of the visually perceived food were identified. En raison de la prévalence croissante de l'obésité et du défi que cela représente en matière de santé publique, une meilleure compréhension des processus comportementaux et cérébraux liés à la nourriture sont nécessaires. En particulier, cette thèse se concentre sur l'investigation des mécanismes cérébraux spatio-temporels liés à la perception visuelle de la nourriture. Nous sommes quotidiennement et répétitivement exposés à des images de nourriture. Ces expositions répétées influencent nos choix, ainsi que nos préférences alimentaires. La première étude (Study A) de cette thèse investigue donc l'impact de ces exposition répétée à des stimuli visuels de nourriture. En particulier, nous avons comparé la dynamique spatio-temporelle de l'activité cérébrale induite par une exposition répétée à des images de nourriture de haute densité et de basse densité énergétique. (Manuscrit publié: "The role of energetic value in dynamic brain response adaptation during repeated food image viewing" (Lietti et al., 2012)). Dans cette étude, nous avons pu constater qu'une exposition répétée à des images représentant de la nourriture de haute densité énergétique, par opposition à de la nourriture de basse densité énergétique, affecte les mécanismes comportementaux et cérébraux de manière différente. En particulier, la représentation neurale des images de nourriture de haute densité énergétique est similaire lors de l'exposition initiale que lors de l'exposition répétée. Ceci indique que la perception d'images de nourriture de haute densité énergétique induit des adaptations comportementales et neurales de moindre ampleur par rapport à la perception d'images de nourriture de basse densité énergétique ou à la perception d'une « catégorie contrôle » d'objets qui ne sont pas de la nourriture. Notre discussion est orientée sur les notions prépondérantes de récompense et de motivation qui sont associées à la nourriture de haute densité énergétique. Nous suggérons que la nourriture de haute densité énergétique génère une représentation mémorielle plus stable et que ce mécanisme pourrait (partiellement) être sous-jacent au fait que la nourriture de haute densité énergétique soit préférentiellement consommée. Dans la deuxième étude (Study Β) menée au cours de cette thèse, nous nous sommes intéressés aux mécanismes de perception de la nourriture chez des patients ayant subi un bypass gastrique Roux- en-Y, afin de réussir à perdre du poids et améliorer leur santé. Ce type de chirurgie est connu pour altérer la perception de la nourriture et le comportement alimentaire, mais également la sécrétion d'adipokines et de peptides gastriques. Dans une approche interdisciplinaire et globale, cette deuxième étude investigue donc les différences entre les patients opérés et des individus « contrôles » de poids similaire au niveau des interactions entre leur activité cérébrale et les mesures de leurs hormones gastriques. D'un côté, nous avons investigué la dynamique spatio-temporelle cérébrale de la perception visuelle de nourriture de haute et de basse densité énergétique dans deux états physiologiques différent (pre- et post-prandial). Et de l'autre, nous avons également investigué les mesures physiologiques des hormones gastriques. Ensuite, afin d'évaluer les altérations liées à l'intervention chirurgicale au niveau des interactions entre la réponse cérébrale et la sécrétion d'hormone, des corrélations entre ces deux mesures ont été comparées entre les deux groupes. Les résultats révèlent que l'intervention chirurgicale du bypass gastrique Roux-en-Y altère la dynamique spatio-temporelle de la perception visuelle de la nourriture de haute et de basse densité énergétique, ainsi que les interactions entre cette dernière et les mesures périphériques des hormones gastriques. Nous discutons le rôle potentiel de ces altérations en relation avec les modulations des facteurs physiologiques et les changements du comportement alimentaire préalablement déjà démontrés. De cette manière, nous identifions des cibles potentielles pour le développement de stratégies d'intervention future, au niveau comportemental, cérébral et endocrinien (hormones gastriques) en ce qui concerne les déviances du comportement alimentaire, dont l'obésité. Nos deux études réunies démontrent que la représentation visuelle de la nourriture dans le cerveau est plastique et que des modulations de l'activité neurale apparaissent déjà à un stade très précoce des mécanismes de perception visuelle. Différents facteurs d'influence comme une exposition repetee, le bypass gastrique Roux-en-Y, la motivation (état nutritionnel), ainsi que la densité énergétique de la nourriture qui est perçue ont pu être identifiés.

Selective integration of auditory-visual looming cues by humans.

An object's motion relative to an observer can confer ethologically meaningful information. Approaching or looming stimuli can signal threats/collisions to be avoided or prey to be confronted, whereas receding stimuli can signal successful escape or failed pursuit. Using movement detection and subjective ratings, we investigated the multisensory integration of looming and receding auditory and visual information by humans. While prior research has demonstrated a perceptual bias for unisensory and more recently multisensory looming stimuli, none has investigated whether there is integration of looming signals between modalities. Our findings reveal selective integration of multisensory looming stimuli. Performance was significantly enhanced for looming stimuli over all other multisensory conditions. Contrasts with static multisensory conditions indicate that only multisensory looming stimuli resulted in facilitation beyond that induced by the sheer presence of auditory-visual stimuli. Controlling for variation in physical energy replicated the advantage for multisensory looming stimuli. Finally, only looming stimuli exhibited a negative linear relationship between enhancement indices for detection speed and for subjective ratings. Maximal detection speed was attained when motion perception was already robust under unisensory conditions. The preferential integration of multisensory looming stimuli highlights that complex ethologically salient stimuli likely require synergistic cooperation between existing principles of multisensory integration. A new conceptualization of the neurophysiologic mechanisms mediating real-world multisensory perceptions and action is therefore supported.

Auditory-visual multisensory interactions in humans: timing, topography, directionality, and sources.

Current models of brain organization include multisensory interactions at early processing stages and within low-level, including primary, cortices. Embracing this model with regard to auditory-visual (AV) interactions in humans remains problematic. Controversy surrounds the application of an additive model to the analysis of event-related potentials (ERPs), and conventional ERP analysis methods have yielded discordant latencies of effects and permitted limited neurophysiologic interpretability. While hemodynamic imaging and transcranial magnetic stimulation studies provide general support for the above model, the precise timing, superadditive/subadditive directionality, topographic stability, and sources remain unresolved. We recorded ERPs in humans to attended, but task-irrelevant stimuli that did not require an overt motor response, thereby circumventing paradigmatic caveats. We applied novel ERP signal analysis methods to provide details concerning the likely bases of AV interactions. First, nonlinear interactions occur at 60-95 ms after stimulus and are the consequence of topographic, rather than pure strength, modulations in the ERP. AV stimuli engage distinct configurations of intracranial generators, rather than simply modulating the amplitude of unisensory responses. Second, source estimations (and statistical analyses thereof) identified primary visual, primary auditory, and posterior superior temporal regions as mediating these effects. Finally, scalar values of current densities in all of these regions exhibited functionally coupled, subadditive nonlinear effects, a pattern increasingly consistent with the mounting evidence in nonhuman primates. In these ways, we demonstrate how neurophysiologic bases of multisensory interactions can be noninvasively identified in humans, allowing for a synthesis across imaging methods on the one hand and species on the other.

Cognitive disorganisation in schizotypy is associated with deterioration in visual backward masking.

To understand the causes of schizophrenia, a search for stable markers (endophenotypes) is ongoing. In previous years, we have shown that the shine-through visual backward masking paradigm meets the most important characteristics of an endophenotype. Here, we tested masking performance differences between healthy students with low and high schizotypy scores as determined by the self-report O-Life questionnaire assessing schizotypy along three dimensions, i.e. positive schizotypy (unusual experiences), cognitive disorganisation, and negative schizotypy (introvertive anhedonia). Forty participants performed the shine-through backward masking task and a classical cognitive test, the Wisconsin Card Sorting Task (WCST). We found that visual backward masking was impaired for students scoring high as compared to low on the cognitive disorganisation dimension, whereas the positive and negative schizotypy dimensions showed no link to masking performance. We also found group differences for students scoring high and low on the cognitive disorganisation factor for the WCST. These findings indicate that the shine-through paradigm is sensitive to differences in schizotypy which are closely linked with the pathological expression in schizophrenia.

Visual activity predicts auditory recovery from deafness after adult cochlear implantation.

Modern cochlear implantation technologies allow deaf patients to understand auditory speech; however, the implants deliver only a coarse auditory input and patients must use long-term adaptive processes to achieve coherent percepts. In adults with post-lingual deafness, the high progress of speech recovery is observed during the first year after cochlear implantation, but there is a large range of variability in the level of cochlear implant outcomes and the temporal evolution of recovery. It has been proposed that when profoundly deaf subjects receive a cochlear implant, the visual cross-modal reorganization of the brain is deleterious for auditory speech recovery. We tested this hypothesis in post-lingually deaf adults by analysing whether brain activity shortly after implantation correlated with the level of auditory recovery 6 months later. Based on brain activity induced by a speech-processing task, we found strong positive correlations in areas outside the auditory cortex. The highest positive correlations were found in the occipital cortex involved in visual processing, as well as in the posterior-temporal cortex known for audio-visual integration. The other area, which positively correlated with auditory speech recovery, was localized in the left inferior frontal area known for speech processing. Our results demonstrate that the visual modality's functional level is related to the proficiency level of auditory recovery. Based on the positive correlation of visual activity with auditory speech recovery, we suggest that visual modality may facilitate the perception of the word's auditory counterpart in communicative situations. The link demonstrated between visual activity and auditory speech perception indicates that visuoauditory synergy is crucial for cross-modal plasticity and fostering speech-comprehension recovery in adult cochlear-implanted deaf patients.

Recovery of biological motion perception and network plasticity after cerebellar tumor removal.

Visual perception of body motion is vital for everyday activities such as social interaction, motor learning or car driving. Tumors to the left lateral cerebellum impair visual perception of body motion. However, compensatory potential after cerebellar damage and underlying neural mechanisms remain unknown. In the present study, visual sensitivity to point-light body motion was psychophysically assessed in patient SL with dysplastic gangliocytoma (Lhermitte-Duclos disease) to the left cerebellum before and after neurosurgery, and in a group of healthy matched controls. Brain activity during processing of body motion was assessed by functional magnetic resonance imaging (MRI). Alterations in underlying cerebro-cerebellar circuitry were studied by psychophysiological interaction (PPI) analysis. Visual sensitivity to body motion in patient SL before neurosurgery was substantially lower than in controls, with significant improvement after neurosurgery. Functional MRI in patient SL revealed a similar pattern of cerebellar activation during biological motion processing as in healthy participants, but located more medially, in the left cerebellar lobules III and IX. As in normalcy, PPI analysis showed cerebellar communication with a region in the superior temporal sulcus, but located more anteriorly. The findings demonstrate a potential for recovery of visual body motion processing after cerebellar damage, likely mediated by topographic shifts within the corresponding cerebro-cerebellar circuitry induced by cerebellar reorganization. The outcome is of importance for further understanding of cerebellar plasticity and neural circuits underpinning visual social cognition.