Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.

Specific attentional impairments and complex visual hallucinations in eye disease

OBJECTIVE: To test the prediction by the Perception and Attention Deficit (PAD) model of complex visual hallucinations that cognitive impairment, specifically in visual attention, is a key risk factor for complex hallucinations in eye disease. METHODS: Two studies of elderly patients with acquired eye disease investigated the relationship between complex visual hallucinations (CVH) and impairments in general cognition and verbal attention (Study 1) and between CVH, selective visual attention and visual object perception (Study 2). The North East Visual Hallucinations Inventory was used to classify CVH. RESULTS: In Study 1, there was no relationship between CVH (n=10/39) and performance on cognitive screening or verbal attention tasks. In Study 2, participants with CVH (n=11/31) showed poorer performance on a modified Stroop task (p<0.05), a novel imagery-based attentional task (p<0.05) and picture (p<0.05) but not silhouette naming (p=0.13) tasks. Performance on these tasks correctly classified 83% of the participants as hallucinators or non-hallucinators. CONCLUSIONS: The results suggest that, consistent with the PAD model, complex visual hallucinations in people with acquired eye disease are associated with visual attention impairment.

Modulating the granularity of category formation by global cortical states

The unsupervised categorization of sensory stimuli is typically attributed to feedforward processing in a hierarchy of cortical areas. This purely sensory-driven view of cortical processing, however, ignores any internal modulation, e.g., by top-down attentional signals or neuromodulator release. To isolate the role of internal signaling on category formation, we consider an unbroken continuum of stimuli without intrinsic category boundaries. We show that a competitive network, shaped by recurrent inhibition and endowed with Hebbian and homeostatic synaptic plasticity, can enforce stimulus categorization. The degree of competition is internally controlled by the neuronal gain and the strength of inhibition. Strong competition leads to the formation of many attracting network states, each being evoked by a distinct subset of stimuli and representing a category. Weak competition allows more neurons to be co-active, resulting in fewer but larger categories. We conclude that the granularity of cortical category formation, i.e., the number and size of emerging categories, is not simply determined by the richness of the stimulus environment, but rather by some global internal signal modulating the network dynamics. The model also explains the salient non-additivity of visual object representation observed in the monkey inferotemporal (IT) cortex. Furthermore, it offers an explanation of a previously observed, demand-dependent modulation of IT activity on a stimulus categorization task and of categorization-related cognitive deficits in schizophrenic patients.

Imagery of a moving object: the role of occipital cortex and human MT/V5+

Visual imagery – similar to visual perception – activates feature-specific and category-specific visual areas. This is frequently observed in experiments where the instruction is to imagine stimuli that have been shown immediately before the imagery task. Hence, feature-specific activation could be related to the short-term memory retrieval of previously presented sensory information. Here, we investigated mental imagery of stimuli that subjects had not seen before, eliminating the effects of short-term memory. We recorded brain activation using fMRI while subjects performed a behaviourally controlled guided imagery task in predefined retinotopic coordinates to optimize sensitivity in early visual areas. Whole brain analyses revealed activation in a parieto-frontal network and lateral–occipital cortex. Region of interest (ROI) based analyses showed activation in left hMT/V5+. Granger causality mapping taking left hMT/V5+ as source revealed an imagery-specific directed influence from the left inferior parietal lobule (IPL). Interestingly, we observed a negative BOLD response in V1–3 during imagery, modulated by the retinotopic location of the imagined motion trace. Our results indicate that rule-based motion imagery can activate higher-order visual areas involved in motion perception, with a role for top-down directed influences originating in IPL. Lower-order visual areas (V1, V2 and V3) were down-regulated during this type of imagery, possibly reflecting inhibition to avoid visual input from interfering with the imagery construction. This suggests that the activation in early visual areas observed in previous studies might be related to short- or long-term memory retrieval of specific sensory experiences.

Brain responses to auditory and visual stimulus offset: Shared representations of temporal edges

Edges are crucial for the formation of coherent objects from sequential sensory inputs within a single modality. Moreover, temporally coincident boundaries of perceptual objects across different sensory modalities facilitate crossmodal integration. Here, we used functional magnetic resonance imaging in order to examine the neural basis of temporal edge detection across modalities. Onsets of sensory inputs are not only related to the detection of an edge but also to the processing of novel sensory inputs. Thus, we used transitions from input to rest (offsets) as convenient stimuli for studying the neural underpinnings of visual and acoustic edge detection per se. We found, besides modality-specific patterns, shared visual and auditory offset-related activity in the superior temporal sulcus and insula of the right hemisphere. Our data suggest that right hemispheric regions known to be involved in multisensory processing are crucial for detection of edges in the temporal domain across both visual and auditory modalities. This operation is likely to facilitate cross-modal object feature binding based on temporal coincidence. Hum Brain Mapp, 2008. (c) 2008 Wiley-Liss, Inc.

Functional imbalance of visual pathways indicates alternative face processing strategies in autism

OBJECTIVE: To investigate whether autistic subjects show a different pattern of neural activity than healthy individuals during processing of faces and complex patterns. METHODS: Blood oxygen level-dependent (BOLD) signal changes accompanying visual processing of faces and complex patterns were analyzed in an autistic group (n = 7; 25.3 [6.9] years) and a control group (n = 7; 27.7 [7.8] years). RESULTS: Compared with unaffected subjects, autistic subjects demonstrated lower BOLD signals in the fusiform gyrus, most prominently during face processing, and higher signals in the more object-related medial occipital gyrus. Further signal increases in autistic subjects vs controls were found in regions highly important for visual search: the superior parietal lobule and the medial frontal gyrus, where the frontal eye fields are located. CONCLUSIONS: The cortical activation pattern during face processing indicates deficits in the face-specific regions, with higher activations in regions involved in visual search. These findings reflect different strategies for visual processing, supporting models that propose a predisposition to local rather than global modes of information processing in autism.

A comparison of pixel and object-based land cover classification. A case study of the Asmara region, Eritrea

In this paper we compare the performance of two image classification paradigms (object- and pixel-based) for creating a land cover map of Asmara, the capital of Eritrea and its surrounding areas using a Landsat ETM+ imagery acquired in January 2000. The image classification methods used were maximum likelihood for the pixel-based approach and Bhattacharyya distance for the object-oriented approach available in, respectively, ArcGIS and SPRING software packages. Advantages and limitations of both approaches are presented and discussed. Classifications outputs were assessed using overall accuracy and Kappa indices. Pixel- and object-based classification methods result in an overall accuracy of 78% and 85%, respectively. The Kappa coefficient for pixel- and object-based approaches was 0.74 and 0.82, respectively. Although pixel-based approach is the most commonly used method, assessment and visual interpretation of the results clearly reveal that the object-oriented approach has advantages for this specific case-study.

Taking an Object-Centric View on Dynamic Information with Object Flow Analysis

A large body of research analyzes the runtime execution of a system to extract abstract behavioral views. Those approaches primarily analyze control flow by tracing method execution events or they analyze object graphs of heap snapshots. However, they do not capture how objects are passed through the system at runtime. We refer to the exchange of objects as the object flow, and we claim that object flow is necessary to analyze if we are to understand the runtime of an object-oriented application. We propose and detail Object Flow Analysis, a novel dynamic analysis technique that takes this new information into account. To evaluate its usefulness, we present a visual approach that allows a developer to study classes and components in terms of how they exchange objects at runtime. We illustrate our approach on three case studies.

Complex visual hallucinations and attentional performance in eye disease and dementia: a test of the Perception and Attention Deficit model

OBJECTIVE This study aimed to test the prediction from the Perception and Attention Deficit model of complex visual hallucinations (CVH) that impairments in visual attention and perception are key risk factors for complex hallucinations in eye disease and dementia. METHODS Two studies ran concurrently to investigate the relationship between CVH and impairments in perception (picture naming using the Graded Naming Test) and attention (Stroop task plus a novel Imagery task). The studies were in two populations-older patients with dementia (n = 28) and older people with eye disease (n = 50) with a shared control group (n = 37). The same methodology was used in both studies, and the North East Visual Hallucinations Inventory was used to identify CVH. RESULTS A reliable relationship was found for older patients with dementia between impaired perceptual and attentional performance and CVH. A reliable relationship was not found in the population of people with eye disease. CONCLUSIONS The results add to previous research that object perception and attentional deficits are associated with CVH in dementia, but that risk factors for CVH in eye disease are inconsistent, suggesting that dynamic rather than static impairments in attentional processes may be key in this population.

Visual perception in Parkinson disease dementia and dementia with Lewy bodies

OBJECTIVE To quantify visual discrimination, space-motion, and object-form perception in patients with Parkinson disease dementia (PDD), dementia with Lewy bodies (DLB), and Alzheimer disease (AD). METHODS The authors used a cross-sectional study to compare three demented groups matched for overall dementia severity (PDD: n = 24; DLB: n = 20; AD: n = 23) and two age-, sex-, and education-matched control groups (PD: n = 24, normal controls [NC]: n = 25). RESULTS Visual perception was globally more impaired in PDD than in nondemented controls (NC, PD), but was not different from DLB. Compared to AD, PDD patients tended to perform worse in all perceptual scores. Visual perception of patients with PDD/DLB and visual hallucinations was significantly worse than in patients without hallucinations. CONCLUSIONS Parkinson disease dementia (PDD) is associated with profound visuoperceptual impairments similar to dementia with Lewy bodies (DLB) but different from Alzheimer disease. These findings are consistent with previous neuroimaging studies reporting hypoactivity in cortical areas involved in visual processing in PDD and DLB.

Atypical visual processing in posttraumatic stress disorder

BACKGROUND: Many patients with Posttraumatic Stress Disorder (PTSD) feel overwhelmed in situations with high levels of sensory input, as in crowded situations with complex sensory characteristics. These difficulties might be related to subtle sensory processing deficits similar to those that have been found for sounds in electrophysiological studies. METHOD: Visual processing was investigated with functional magnetic resonance imaging in trauma-exposed participants with (N = 18) and without PTSD (N = 21) employing a picture-viewing task. RESULTS: Activity observed in response to visual scenes was lower in PTSD participants 1) in the ventral stream of the visual system, including striate and extrastriate, inferior temporal, and entorhinal cortices, and 2) in dorsal and ventral attention systems (P < 0.05, FWE-corrected). These effects could not be explained by the emotional salience of the pictures. CONCLUSIONS: Visual processing was substantially altered in PTSD in the ventral visual stream, a component of the visual system thought to be responsible for object property processing. Together with previous reports of subtle auditory deficits in PTSD, these findings provide strong support for potentially important sensory processing deficits, whose origins may be related to dysfunctional attention processes.

A new method to measure higher visual functions in an immersive environment

BACKGROUND: Higher visual functions can be defined as cognitive processes responsible for object recognition, color and shape perception, and motion detection. People with impaired higher visual functions after unilateral brain lesion are often tested with paper pencil tests, but such tests do not assess the degree of interaction between the healthy brain hemisphere and the impaired one. Hence, visual functions are not tested separately in the contralesional and ipsilesional visual hemifields. METHODS: A new measurement setup, that involves real-time comparisons of shape and size of objects, orientation of lines, speed and direction of moving patterns, in the right or left visual hemifield, has been developed. The setup was implemented in an immersive environment like a hemisphere to take into account the effects of peripheral and central vision, and eventual visual field losses. Due to the non-flat screen of the hemisphere, a distortion algorithm was needed to adapt the projected images to the surface. Several approaches were studied and, based on a comparison between projected images and original ones, the best one was used for the implementation of the test. Fifty-seven healthy volunteers were then tested in a pilot study. A Satisfaction Questionnaire was used to assess the usability of the new measurement setup. RESULTS: The results of the distortion algorithm showed a structural similarity between the warped images and the original ones higher than 97%. The results of the pilot study showed an accuracy in comparing images in the two visual hemifields of 0.18 visual degrees and 0.19 visual degrees for size and shape discrimination, respectively, 2.56° for line orientation, 0.33 visual degrees/s for speed perception and 7.41° for recognition of motion direction. The outcome of the Satisfaction Questionnaire showed a high acceptance of the battery by the participants. CONCLUSIONS: A new method to measure higher visual functions in an immersive environment was presented. The study focused on the usability of the developed battery rather than the performance at the visual tasks. A battery of five subtasks to study the perception of size, shape, orientation, speed and motion direction was developed. The test setup is now ready to be tested in neurological patients.

Visual acuity of dentists in their respective clinical conditions.

OBJECTIVES This study examined the impact of age and magnification on the near visual acuity of dentists in their private practice under simulated clinical conditions. MATERIALS AND METHODS Miniaturized visual tests were fixed in posterior teeth of a dental phantom head and brought to 31 dentists in their respective private practice. The visual acuity of these dentists (n = 19, ≥40 years; n = 12, <40 years) was measured in a clinical setting under the following conditions: (a) natural visual acuity, distance of 300 mm; (b) natural visual acuity, free choice of the distance; and (c) loupe and additional light source, if available. RESULTS The visual acuity under the different clinical conditions varied widely between individuals. The older group of dentists had a lower median visual acuity value under all clinical conditions. This difference was highly significant for natural visual acuity at a free choice of distance (p < 0.0001). For younger dentists (<40 years), visual acuity could be significantly improved by reducing the eye-object distance (p = 0.001) or by using loupes (p = 0.008). For older dentists (≥40 years), visual acuity could be significantly improved by using loupes (p = 0.0005). CONCLUSIONS Visual performance decreased with increasing age under the specific clinical conditions of each dentist's private practice. Magnification aids can compensate for visual deficiencies. CLINICAL RELEVANCE The question of whether findings obtained under standardized conditions are valuable for the habitual setting of each dentist's private practice seems clinically relevant.

Age-dependent visual exploration during simulated day- and night driving on a motorway: a cross-sectional study

BACKGROUND: Central and peripheral vision is needed for object detection. Previous research has shown that visual target detection is affected by age. In addition, light conditions also influence visual exploration. The aim of the study was to investigate the effects of age and different light conditions on visual exploration behavior and on driving performance during simulated driving. METHODS: A fixed-base simulator with 180 degree field of view was used to simulate a motorway route under daylight and night conditions to test 29 young subjects (25-40 years) and 27 older subjects (65-78 years). Drivers' eye fixations were analyzed and assigned to regions of interests (ROI) such as street, road signs, car ahead, environment, rear view mirror, side mirror left, side mirror right, incoming car, parked car, road repair. In addition, lane-keeping and driving speed were analyzed as a measure of driving performance. RESULTS: Older drivers had longer fixations on the task relevant ROI, but had a lower frequency of checking mirrors when compared to younger drivers. In both age groups, night driving led to a less fixations on the mirror. At the performance level, older drivers showed more variation in driving speed and lane-keeping behavior, which was especially prominent at night. In younger drivers, night driving had no impact on driving speed or lane-keeping behavior. CONCLUSIONS: Older drivers' visual exploration behavior are more fixed on the task relevant ROI, especially at night, when driving performance becomes more heterogeneous than in younger drivers.

Detecting single-target changes in multiple object tracking: The case of peripheral vision

In the current study it is investigated whether peripheral vision can be used to monitor multi-ple moving objects and to detect single-target changes. For this purpose, in Experiment 1, a modified MOT setup with a large projection and a constant-position centroid phase had to be checked first. Classical findings regarding the use of a virtual centroid to track multiple ob-jects and the dependency of tracking accuracy on target speed could be successfully replicat-ed. Thereafter, the main experimental variations regarding the manipulation of to-be-detected target changes could be introduced in Experiment 2. In addition to a button press used for the detection task, gaze behavior was assessed using an integrated eye-tracking system. The anal-ysis of saccadic reaction times in relation to the motor response shows that peripheral vision is naturally used to detect motion and form changes in MOT because the saccade to the target occurred after target-change offset. Furthermore, for changes of comparable task difficulties, motion changes are detected better by peripheral vision than form changes. Findings indicate that capabilities of the visual system (e.g., visual acuity) affect change detection rates and that covert-attention processes may be affected by vision-related aspects like spatial uncertainty. Moreover, it is argued that a centroid-MOT strategy might reduce the amount of saccade-related costs and that eye-tracking seems to be generally valuable to test predictions derived from theories on MOT. Finally, implications for testing covert attention in applied settings are proposed.