Seeing music performance: Visual influences on perception and experience

Drawing from ethnographic, empirical, and historical/cultural perspectives, we examine the extent to which visual aspects of music contribute to the communication that takes place between performers and their listeners. First, we introduce a framework for understanding how media and genres shape aural and visual experiences of music. Second, we present case studies of two performances, and describe the relation between visual and aural aspects of performance. Third, we report empirical evidence that visual aspects of performance reliably influence perceptions of musical structure (pitch related features) and affective interpretations of music. Finally, we trace new and old media trajectories of aural and visual dimensions of music, and highlight how our conceptions, perceptions and appreciation of music are intertwined with technological innovation and media deployment strategies.

Effects of visual occlusion and fatigue on motor performance in water

The purpose of this study was to test the effects of visual occlusion and fatigue on the motor performance of vertical skills in synchronized swimming. Experienced synchronized swimmers (n = 12) were randomly assigned to either an exercise or nonexercise (control) activity group. Subjective ratings of fatigue were obtained from the swimmers who then each performed four vertical skills under alternating conditions of vision and visual occlusion before and after either a swimming (designed to induce fatigue) or nonphysical activity. A main effect of activity (p < .03) was found for two measures of performance accuracy (lateral and anterior total distance traveled) but not for lateral and anterior maximum deviation from vertical, indicating that fatigue played a role in executing the skills. The data also indicate that the maintenance of a stationary position is a skill of greater difficulty than maintaining a true vertical. In contrast with previous research findings on synchronized swimmers, a significant effect of vision in all conditions was found, with performance decrements in the conditions of visual occlusion showing that vision provided important sensory input for the swimmers.

Attentional blink modulation in a reaction time task: performance feedback, warning stimulus modality, and task difficulty

The present research investigated the effect of performance feedback on the modulation of the acoustic startle reflex in a Go/NoGo reaction time task. Experiment 1 (n = 120) crossed warning stimulus modality (acoustic, visual, and tactile) with the provision of feedback in a between subject design. Provision of performance feedback increased the number of errors committed and reduced reaction time, but did not affect blink modulation significantly. Attentional blink latency and magnitude modulation was larger during acoustic than during visual and larger during visual than during tactile warning stimuli. In comparison to control blinks, latency shortening was significant in all modality conditions whereas magnitude facilitation was not significant during tactile warning stimuli. Experiment 2 (n = 80) employed visual warning stimuli only and crossed the provision of feedback with task difficulty. Feedback and difficulty affected accuracy and reaction time. Whereas blink latency shortening was not affected, blink magnitude modulation was smallest in the Easy/No Feedback and the Difficult/Feedback conditions. (C) 2002 Elsevier Science B.V. All rights reserved.

Avoidance of obstacles in the absence of visual awareness

The spatial character of our reaching movements is extremely sensitive to potential obstacles in the workspace. We recently found that this sensitivity was retained by most patients with left visual neglect when reaching between two objects, despite the fact that they tended to ignore the leftward object when asked to bisect the space between them. This raises the possibility that obstacle avoidance does not require a conscious awareness of the obstacle avoided. We have now tested this hypothesis in a patient with visual extinction following right temporoparietal damage. Extinction is an attentional disorder in which patients fail to report stimuli on the side of space opposite a brain lesion under conditions of bilateral stimulation. Our patient avoided obstacles during reaching, to exactly the same degree, regardless of whether he was able to report their presence. This implicit processing of object location, which may depend on spared superior parietal-lobe pathways, demonstrates that conscious awareness is not necessary for normal obstacle avoidance.

Preserved obstacle avoidance during reaching in patients with left visual neglect

We asked 12 patients with left visual neglect to bisect the gap between two cylinders or to reach rapidly between them to a more distal target zone. Both tasks demanded a motor response but these responses were quite different in nature. The bisection response was a communicative act whereby the patient indicated the perceived midpoint. The reaching task carried no imperative to bisect the gap, only to maintain a safe distance from either cylinder while steering to the target zone. Optimal performance on either task could only be achieved by reference to the location of both cylinders. Our analysis focused upon the relative influence of the left and right cylinders on the lateral location of the response. In the bisection task, all neglect patients showed qualitatively the same asymmetry, with the left cylinder exerting less influence than the right. In the reaching task, the neglect group behaved like normal subjects, being influenced approximately equally by the two cylinders. This was true for all bar two of the patients, who showed clear neglect in both tasks. We conclude that the visuomotor processing underlying obstacle avoidance during reaching is preserved in most patients with left visual neglect. (C) 2004 Elsevier Ltd. All rights reserved.

"mind the gap": The size-distance dissociation in visual neglect is a cueing effect

There is a growing body of evidence that the processes mediating the allocation of spatial attention within objects may be separable from those governing attentional distribution between objects. In the neglect literature, a related proposal has been made regarding the perception of (within-object) sizes and (between-object) distances. This proposal follows observations that, in size-matching and bisection tasks, neglect is more strongly expressed when patients are required to attend to the sizes of discrete objects than to the (unfilled) distances between objects. These findings are consistent with a partial dissociation between size and distance processing, but a simpler alternative must also be considered. Whilst a neglect patient may fail to explore the full extent of a solid stimulus, the estimation of an unfilled distance requires that both endpoints be inspected before the task can be attempted at all. The attentional cueing implicit in distance estimation tasks might thus account for their superior performance by neglect patients. We report two bisection studies that address this issue. The first confirmed, amongst patients with left visual neglect, a reliable reduction of rightward error for unfilled gap stimuli as compared with solid lines. The second study assessed the cause of this reduction, deconfounding the effects of stimulus type (lines vs. gaps) and attentional cueing, by applying an explicit cueing manipulation to line and gap bisection tasks. Under these matched cueing conditions, all patients performed similarly on line and gap bisection tasks, suggesting that the reduction of neglect typically observed for gap stimuli may be attributable entirely to cueing effects. We found no evidence that a spatial extent, once fully attended, is judged any differently according to whether it is filled or unfilled.

Auditory-visual speech integration by prelinguistic infants: Perception of an emergent consonant in the McGurk effect

The McGurk effect, in which auditory [ba] dubbed onto [go] lip movements is perceived as da or tha, was employed in a real-time task to investigate auditory-visual speech perception in prelingual infants. Experiments 1A and 1B established the validity of real-time dubbing for producing the effect. In Experiment 2, 4(1)/(2)-month-olds were tested in a habituation-test paradigm, in which 2 an auditory-visual stimulus was presented contingent upon visual fixation of a live face. The experimental group was habituated to a McGurk stimulus (auditory [ba] visual [ga]), and the control group to matching auditory-visual [ba]. Each group was then presented with three auditory-only test trials, [ba], [da], and [deltaa] (as in then). Visual-fixation durations in test trials showed that the experimental group treated the emergent percept in the McGurk effect, [da] or [deltaa], as familiar (even though they had not heard these sounds previously) and [ba] as novel. For control group infants [da] and [deltaa] were no more familiar than [ba]. These results are consistent with infants'perception of the McGurk effect, and support the conclusion that prelinguistic infants integrate auditory and visual speech information. (C) 2004 Wiley Periodicals, Inc.

Visual-auditory integration during speech imitation in autism

Children with autistic spectrum disorder (ASD) may have poor audio-visual integration, possibly reflecting dysfunctional 'mirror neuron' systems which have been hypothesised to be at the core of the condition. In the present study, a computer program, utilizing speech synthesizer software and a 'virtual' head (Baldi), delivered speech stimuli for identification in auditory, visual or bimodal conditions. Children with ASD were poorer than controls at recognizing stimuli in the unimodal conditions, but once performance on this measure was controlled for, no group difference was found in the bimodal condition. A group of participants with ASD were also trained to develop their speech-reading ability. Training improved visual accuracy and this also improved the children's ability to utilize visual information in their processing of speech. Overall results were compared to predictions from mathematical models based on integration and non-integration, and were most consistent with the integration model. We conclude that, whilst they are less accurate in recognizing stimuli in the unimodal condition, children with ASD show normal integration of visual and auditory speech stimuli. Given that training in recognition of visual speech was effective, children with ASD may benefit from multi-modal approaches in imitative therapy and language training. (C) 2004 Elsevier Ltd. All rights reserved.

Attentional blink reflex modulation in a continuous performance task is modality specific

Four experiments investigated the attentional modulation of acoustic blinks during continuous spatial tracking tasks. Experiment 1 found blink magnitude inhibition in a visual tracking task. Experiment 2 replicated this finding and also found blink latency slowing. Experiment 3 varied the difficulty of the task and found larger blink inhibition in the easy condition. Blink latency slowing did not differ and was significant at both difficulty levels. Experiment 4 employed less difficult visual and acoustic tracking tasks at two levels of task load. Blink magnitude inhibition during the visual and facilitation during the acoustic task was significant during high load in both modality groups. Blink latency was slowed in all visual task conditions and shortened in the difficult acoustic task. These results indicate that attentional blink modulation in a continuous spatial tracking task is modality specific.

The effect of auditory and visual distracters on the useful field of view: Implications for the driving task

PURPOSE. The driving environment is becoming increasingly complex, including both visual and auditory distractions within the in- vehicle and external driving environments. This study was designed to investigate the effect of visual and auditory distractions on a performance measure that has been shown to be related to driving safety, the useful field of view. METHODS. A laboratory study recorded the useful field of view in 28 young visually normal adults (mean 22.6 +/- 2.2 years). The useful field of view was measured in the presence and absence of visual distracters (of the same angular subtense as the target) and with three levels of auditory distraction (none, listening only, listening and responding). RESULTS. Central errors increased significantly (P < 0.05) in the presence of auditory but not visual distracters, while peripheral errors increased in the presence of both visual and auditory distracters. Peripheral errors increased with eccentricity and were greatest in the inferior region in the presence of distracters. CONCLUSIONS. Visual and auditory distracters reduce the extent of the useful field of view, and these effects are exacerbated in inferior and peripheral locations. This result has significant ramifications for road safety in an increasingly complex in-vehicle and driving environment.

Capacity limits for the detection of changing visual features

Capacity limits in visual attention have traditionally been studied using static arrays of elements from which an observer must detect a target defined by a certain visual feature or combination of features. In the current study we use this visual search paradigm, with accuracy as the dependent variable, to examine attentional capacity limits for different visual features undergoing change over time. In Experiment 1, detectability of a single changing target was measured under conditions where the type of change (size, speed, colour), the magnitude of change, the set size and homogeneity of the unchanging distractors were all systematically varied. Psychometric function slopes were calculated for different experimental conditions and ‘change thresholds’extracted from these slopes were used in Experiment 2, in which multiple supra-threshold changes were made, simultaneously, either to a single or to two or three different stimulus elements. These experiments give an objective psychometric paradigm for measuring changes in visual features over time. Results favour object-based accounts of visual attention, and show consistent differences in the allocation of attentional capacity to different perceptual dimensions.