Digital Musical Interactions: Performer-System Relationships and their Perception by Spectators

This article adopts an ecological view of digital musical interactions, first considering the relationship between performers and digital systems, and then spectators’ perception of these interactions. We provide evidence that the relationships between performers and digital music systems are not necessarily instrumental in the same was as they are with acoustic systems, and nor should they always strive to be. Furthermore, we report results of a study suggesting that spectators may not perceive such interactions in the same way as performances with musical instruments. We present implications for the design of digital musical interactions, suggesting that designers should embrace the reality that digital systems are malleable and dynamic, and may engage performers and spectators in different modalities, sometimes simultaneously.

Music, rhythm, rise time perception and developmental dyslexia: Perception of musical meter predicts reading and phonology

Introduction: Rhythm organises musical events into patterns and forms, and rhythm perception in music is usually studied by using metrical tasks. Metrical structure also plays an organisational function in the phonology of language, via speech prosody, and there is evidence for rhythmic perceptual difficulties in developmental dyslexia. Here we investigate the hypothesis that the accurate perception of musical metrical structure is related to basic auditory perception of rise time, and also to phonological and literacy development in children. Methods: A battery of behavioural tasks was devised to explore relations between musical metrical perception, auditory perception of amplitude envelope structure, phonological awareness (PA) and reading in a sample of 64 typically-developing children and children with developmental dyslexia. Results: We show that individual differences in the perception of amplitude envelope rise time are linked to musical metrical sensitivity, and that musical metrical sensitivity predicts PA and reading development, accounting for over 60% of variance in reading along with age and I.Q. Even the simplest metrical task, based on a duple metrical structure, was performed significantly more poorly by the children with dyslexia. Conclusions: The accurate perception of metrical structure may be critical for phonological development and consequently for the development of literacy. Difficulties in metrical processing are associated with basic auditory rise time processing difficulties, suggesting a primary sensory impairment in developmental dyslexia in tracking the lower-frequency modulations in the speech envelope. © 2010 Elsevier.

Perception of patterns of musical beat distribution in phonological developmental dyslexia:significant longitudinal relations with word reading and reading comprehension

In a recent study, we reported that the accurate perception of beat structure in music ('perception of musical meter') accounted for over 40% of the variance in single word reading in children with and without dyslexia (Huss et al., 2011). Performance in the musical task was most strongly associated with the auditory processing of rise time, even though beat structure was varied by manipulating the duration of the musical notes.

Expertise is perceived from both sound and body movement in musical performance

Music is a rich form of nonverbal communication, in which the movements that expert musicians make during performance can influence the perception of expressive and structural features of the music. Whether the actual skill of a musician is perceivable from vision of movement was examined. In Experiment 1, musicians and non-musicians rated performances by novice, intermediate and expert clarinettists from point-light animations of their movements, sound recordings, or both. Performances by clarinettists of more advanced skill level were rated significantly higher from vision of movements, although this effect was stronger when sound was also presented. In Experiment 2, movements and sound from the novice and expert clarinettists' performances were switched for half the presentations, and were matched for the rest. Ratings of novice music were significantly higher when presented with expert movements, although the opposite was not found for expert sound presented with novice movements. No perceptual effect of raters' own level of musicianship was found in either experiment. These results suggest that expertise is perceivable from vision of musicians' body movements, although perception of skill from sound is dominant. The results from Experiment 2 further indicate a cross-modal effect of vision and audition on the perception of musical expertise. © 2012 Elsevier B.V.

(Dis‑)Harmony in movement: effects of musical dissonance on movement timing and form

While the origins of consonance and dissonance in terms of acoustics, psychoacoustics and physiology have been debated for centuries, their plausible effects on movement synchronization have largely been ignored. The present study aims to address this by investigating whether, and if so how, consonant/dissonant pitch intervals affect the spatiotemporal properties of regular reciprocal aiming movements. We compared movements synchronized either to consonant or to dissonant sounds, and showed that they were differently influenced by the degree of consonance of the sound presented. Interestingly, the difference was present after the sound stimulus was removed. In this case, the performance measured after consonant sound exposure was found to be more stable and accurate, with a higher percentage of information/movement coupling (tau-coupling) and a higher degree of movement circularity when compared to performance measured after the exposure to dissonant sounds. We infer that the neural resonance representing consonant tones leads to finer perception/action coupling which in turn may help explain the prevailing preference for these types of tones.

Language Experience Affects Grouping of Musical Instrument Sounds

Language experience clearly affects the perception of speech, but little is known about whether these differences in perception extend to non-speech sounds. In this study, we investigated rhythmic perception of non-linguistic sounds in speakers of French and German using a grouping task, in which complexity (variability in sounds, presence of pauses) was manipulated. In this task, participants grouped sequences of auditory chimeras formed from musical instruments. These chimeras mimic the complexity of speech without being speech. We found that, while showing the same overall grouping preferences, the German speakers showed stronger biases than the French speakers in grouping complex sequences. Sound variability reduced all participants' biases, resulting in the French group showing no grouping preference for the most variable sequences, though this reduction was attenuated by musical experience. In sum, this study demonstrates that linguistic experience, musical experience, and complexity affect rhythmic grouping of non-linguistic sounds and suggests that experience with acoustic cues in a meaningful context (language or music) is necessary for developing a robust grouping preference that survives acoustic variability.

Is perception of upper body orientation based on the inertia tensor? Normogravity versus microgravity conditions

During lateral leg raising, a synergistic inclination of the supporting leg and trunk in the opposite direction to the leg movement is performed in order to preserve equilibrium. As first hypothesized by Pagano and Turvey (J Exp Psychol Hum Percept Perform, 1995, 21:1070-1087), the perception of limb orientation could be based on the orientation of the limb's inertia tensor. The purpose of this study was thus to explore whether the final upper body orientation (trunk inclination relative to vertical) depends on changes in the trunk inertia tensor. We imposed a loading condition, with total mass of 4 kg added to the subject's trunk in either a symmetrical or asymmetrical configuration. This changed the orientation of the trunk inertia tensor while keeping the total trunk mass constant. In order to separate any effects of the inertia tensor from the effects of gravitational torque, the experiment was carried out in normo- and microgravity. The results indicated that in normogravity the same final upper body orientation was maintained irrespective of the loading condition. In microgravity, regardless of loading conditions the same (but different from the normogravity) orientation of the upper body was achieved through different joint organizations: two joints (the hip and ankle joints of the supporting leg) in the asymmetrical loading condition, and one (hip) in the symmetrical loading condition. In order to determine whether the different orientations of the inertia tensor were perceived during the movement, the interjoint coordination was quantified by performing a principal components analysis (PCA) on the supporting and moving hips and on the supporting ankle joints. It was expected that different loading conditions would modify the principal component of the PCA. In normogravity, asymmetrical loading decreased the coupling between joints, while in microgravity a strong coupling was preserved whatever the loading condition. It was concluded that the trunk inertia tensor did not play a role during the lateral leg raising task because in spite of the absence of gravitational torque the final upper body orientation and the interjoint coupling were not influenced.

An Enactive Approach to the Design of New Tangible Musical Instruments

In this paper, we propose a theoretical framework for the design of tangible interfaces for musical expression. The main insight for the proposed approach is the importance and utility of familiar sensorimotor experiences for the creation of engaging and playable new musical instruments. In particular, we suggest exploiting the commonalities between different natural interactions by varying the auditory response or tactile details of the instrument within certain limits. Using this principle, devices for classes of sounds such as coarse grain collision interactions or friction interactions can be designed. The designs we propose retain the familiar tactile aspect of the interaction so that the performer can take advantage of tacit knowledge gained through experiences with such phenomena in the real world.

Development of Illusory contour perception in infants

We investigated whether infants from 8 ^ 22 weeks of age were sensitive to the illusory contour created by aligned line terminators. Previous reports of illusory-contour detection in infants under 4 months old could be due to infants' preference for the presence of terminators rather than their configuration. We generated preferential-looking stimuli containing sinusoidal lines whose oscillating, abutting terminators give a strong illusory contour in adult perception. Our experiments demonstrated a preference in infants 8 weeks old and above for an oscillating illusory contour compared with a stimulus containing equal terminator density and movement. Control experiments excluded local line density, or attention to alignment in general, as the basis for this result. In the youngest age group (8 ^ 10 weeks) stimulus velocity appears to be critical in determining the visibility of illusory contours, which is consistent with other data on motion processing at this age. We conclude that, by 2 months of age, the infant's visual system contains the nonlinear mechanisms necessary to extract an illusory contour from aligned terminators.