Seeing music performance: Visual influences on perception and experience

Drawing from ethnographic, empirical, and historical/cultural perspectives, we examine the extent to which visual aspects of music contribute to the communication that takes place between performers and their listeners. First, we introduce a framework for understanding how media and genres shape aural and visual experiences of music. Second, we present case studies of two performances, and describe the relation between visual and aural aspects of performance. Third, we report empirical evidence that visual aspects of performance reliably influence perceptions of musical structure (pitch related features) and affective interpretations of music. Finally, we trace new and old media trajectories of aural and visual dimensions of music, and highlight how our conceptions, perceptions and appreciation of music are intertwined with technological innovation and media deployment strategies.

Avoidance of obstacles in the absence of visual awareness

The spatial character of our reaching movements is extremely sensitive to potential obstacles in the workspace. We recently found that this sensitivity was retained by most patients with left visual neglect when reaching between two objects, despite the fact that they tended to ignore the leftward object when asked to bisect the space between them. This raises the possibility that obstacle avoidance does not require a conscious awareness of the obstacle avoided. We have now tested this hypothesis in a patient with visual extinction following right temporoparietal damage. Extinction is an attentional disorder in which patients fail to report stimuli on the side of space opposite a brain lesion under conditions of bilateral stimulation. Our patient avoided obstacles during reaching, to exactly the same degree, regardless of whether he was able to report their presence. This implicit processing of object location, which may depend on spared superior parietal-lobe pathways, demonstrates that conscious awareness is not necessary for normal obstacle avoidance.

"mind the gap": The size-distance dissociation in visual neglect is a cueing effect

There is a growing body of evidence that the processes mediating the allocation of spatial attention within objects may be separable from those governing attentional distribution between objects. In the neglect literature, a related proposal has been made regarding the perception of (within-object) sizes and (between-object) distances. This proposal follows observations that, in size-matching and bisection tasks, neglect is more strongly expressed when patients are required to attend to the sizes of discrete objects than to the (unfilled) distances between objects. These findings are consistent with a partial dissociation between size and distance processing, but a simpler alternative must also be considered. Whilst a neglect patient may fail to explore the full extent of a solid stimulus, the estimation of an unfilled distance requires that both endpoints be inspected before the task can be attempted at all. The attentional cueing implicit in distance estimation tasks might thus account for their superior performance by neglect patients. We report two bisection studies that address this issue. The first confirmed, amongst patients with left visual neglect, a reliable reduction of rightward error for unfilled gap stimuli as compared with solid lines. The second study assessed the cause of this reduction, deconfounding the effects of stimulus type (lines vs. gaps) and attentional cueing, by applying an explicit cueing manipulation to line and gap bisection tasks. Under these matched cueing conditions, all patients performed similarly on line and gap bisection tasks, suggesting that the reduction of neglect typically observed for gap stimuli may be attributable entirely to cueing effects. We found no evidence that a spatial extent, once fully attended, is judged any differently according to whether it is filled or unfilled.

Perception of faces and bodies: Similar or different?

Human faces and bodies are both complex and interesting perceptual objects, and both convey important social information. Given these similarities between faces and bodies, we can ask how similar are the visual processing mechanisms used to recognize them. It has long been argued that faces are subject to dedicated and unique perceptual processes, but until recently, relatively little research has focused on how we perceive the human. body. Some recent paradigms indicate that faces and bodies are processed differently; others show similarities in face and body perception. These similarities and differences depend on the type of perceptual task and the level of processing involved. Future research should take these issues into account.

Visual-auditory integration during speech imitation in autism

Children with autistic spectrum disorder (ASD) may have poor audio-visual integration, possibly reflecting dysfunctional 'mirror neuron' systems which have been hypothesised to be at the core of the condition. In the present study, a computer program, utilizing speech synthesizer software and a 'virtual' head (Baldi), delivered speech stimuli for identification in auditory, visual or bimodal conditions. Children with ASD were poorer than controls at recognizing stimuli in the unimodal conditions, but once performance on this measure was controlled for, no group difference was found in the bimodal condition. A group of participants with ASD were also trained to develop their speech-reading ability. Training improved visual accuracy and this also improved the children's ability to utilize visual information in their processing of speech. Overall results were compared to predictions from mathematical models based on integration and non-integration, and were most consistent with the integration model. We conclude that, whilst they are less accurate in recognizing stimuli in the unimodal condition, children with ASD show normal integration of visual and auditory speech stimuli. Given that training in recognition of visual speech was effective, children with ASD may benefit from multi-modal approaches in imitative therapy and language training. (C) 2004 Elsevier Ltd. All rights reserved.

Capacity limits for the detection of changing visual features

Capacity limits in visual attention have traditionally been studied using static arrays of elements from which an observer must detect a target defined by a certain visual feature or combination of features. In the current study we use this visual search paradigm, with accuracy as the dependent variable, to examine attentional capacity limits for different visual features undergoing change over time. In Experiment 1, detectability of a single changing target was measured under conditions where the type of change (size, speed, colour), the magnitude of change, the set size and homogeneity of the unchanging distractors were all systematically varied. Psychometric function slopes were calculated for different experimental conditions and ‘change thresholds’extracted from these slopes were used in Experiment 2, in which multiple supra-threshold changes were made, simultaneously, either to a single or to two or three different stimulus elements. These experiments give an objective psychometric paradigm for measuring changes in visual features over time. Results favour object-based accounts of visual attention, and show consistent differences in the allocation of attentional capacity to different perceptual dimensions.