Digital Musical Interactions: Performer-System Relationships and their Perception by Spectators

This article adopts an ecological view of digital musical interactions, first considering the relationship between performers and digital systems, and then spectators’ perception of these interactions. We provide evidence that the relationships between performers and digital music systems are not necessarily instrumental in the same was as they are with acoustic systems, and nor should they always strive to be. Furthermore, we report results of a study suggesting that spectators may not perceive such interactions in the same way as performances with musical instruments. We present implications for the design of digital musical interactions, suggesting that designers should embrace the reality that digital systems are malleable and dynamic, and may engage performers and spectators in different modalities, sometimes simultaneously.

A Framework for the Evaluation of Digital Musical Instruments

At the outset of a discussion of evaluating digital musical instruments, that is to say instruments whose sound generators are digital and separable though not necessarily separate from their control interfaces (Malloch, 2006), it is reasonable to ask what the term evaluation in this context really means. After all, there may be many perspectives from which to view the effectiveness or otherwise of the instruments we build. For most performers, performance on an instrument becomes a means of evaluating how well it functions in the context of live music making, and their measure of success is the response of the audience to their performance. Audiences evaluate performances on the basis of how engaged they feel they have been by what they have seen and heard. When questioned, they are likely to describe good performances as “exciting,” “skillful,” “musical.” Bad performances are “boring,” and those which are marred by technical malfunction are often dismissed out of hand. If performance is considered to be a valid means of evaluating a musical instrument, then it follows that, for the field of DMI design, a much broader definition of the term “evaluation” than that typically used in human-computer interaction (HCI) is required to reflect the fact that there are a number of stakeholders involved in the design and evaluation of DMIs. In addition to players and audiences, there are also composers, instrument builders, component manufacturers, and perhaps even customers, each of whom will have a different concept of what is meant by “evaluation.”

Music, rhythm, rise time perception and developmental dyslexia: Perception of musical meter predicts reading and phonology

Introduction: Rhythm organises musical events into patterns and forms, and rhythm perception in music is usually studied by using metrical tasks. Metrical structure also plays an organisational function in the phonology of language, via speech prosody, and there is evidence for rhythmic perceptual difficulties in developmental dyslexia. Here we investigate the hypothesis that the accurate perception of musical metrical structure is related to basic auditory perception of rise time, and also to phonological and literacy development in children. Methods: A battery of behavioural tasks was devised to explore relations between musical metrical perception, auditory perception of amplitude envelope structure, phonological awareness (PA) and reading in a sample of 64 typically-developing children and children with developmental dyslexia. Results: We show that individual differences in the perception of amplitude envelope rise time are linked to musical metrical sensitivity, and that musical metrical sensitivity predicts PA and reading development, accounting for over 60% of variance in reading along with age and I.Q. Even the simplest metrical task, based on a duple metrical structure, was performed significantly more poorly by the children with dyslexia. Conclusions: The accurate perception of metrical structure may be critical for phonological development and consequently for the development of literacy. Difficulties in metrical processing are associated with basic auditory rise time processing difficulties, suggesting a primary sensory impairment in developmental dyslexia in tracking the lower-frequency modulations in the speech envelope. © 2010 Elsevier.

Expertise is perceived from both sound and body movement in musical performance

Music is a rich form of nonverbal communication, in which the movements that expert musicians make during performance can influence the perception of expressive and structural features of the music. Whether the actual skill of a musician is perceivable from vision of movement was examined. In Experiment 1, musicians and non-musicians rated performances by novice, intermediate and expert clarinettists from point-light animations of their movements, sound recordings, or both. Performances by clarinettists of more advanced skill level were rated significantly higher from vision of movements, although this effect was stronger when sound was also presented. In Experiment 2, movements and sound from the novice and expert clarinettists' performances were switched for half the presentations, and were matched for the rest. Ratings of novice music were significantly higher when presented with expert movements, although the opposite was not found for expert sound presented with novice movements. No perceptual effect of raters' own level of musicianship was found in either experiment. These results suggest that expertise is perceivable from vision of musicians' body movements, although perception of skill from sound is dominant. The results from Experiment 2 further indicate a cross-modal effect of vision and audition on the perception of musical expertise. © 2012 Elsevier B.V.

Perception of patterns of musical beat distribution in phonological developmental dyslexia:significant longitudinal relations with word reading and reading comprehension

In a recent study, we reported that the accurate perception of beat structure in music ('perception of musical meter') accounted for over 40% of the variance in single word reading in children with and without dyslexia (Huss et al., 2011). Performance in the musical task was most strongly associated with the auditory processing of rise time, even though beat structure was varied by manipulating the duration of the musical notes.