A flexible bio-inspired hierarchical model for analyzing musical timbre

A flexible and multipurpose bio-inspired hierarchical model for analyzing musical timbre is presented in this paper. Inspired by findings in the fields of neuroscience, computational neuroscience, and psychoacoustics, not only does the model extract spectral and temporal characteristics of a signal, but it also analyzes amplitude modulations on different timescales. It uses a cochlear filter bank to resolve the spectral components of a sound, lateral inhibition to enhance spectral resolution, and a modulation filter bank to extract the global temporal envelope and roughness of the sound from amplitude modulations. The model was evaluated in three applications. First, it was used to simulate subjective data from two roughness experiments. Second, it was used for musical instrument classification using the k-NN algorithm and a Bayesian network. Third, it was applied to find the features that characterize sounds whose timbres were labeled in an audiovisual experiment. The successful application of the proposed model in these diverse tasks revealed its potential in capturing timbral information.

Human-Computer interaction methodologies applied in the evaluation of haptic digital musical instruments

Recent developments in interactive technologies have seen major changes in the manner in which artists, performers, and creative individuals interact with digital music technology; this is due to the increasing variety of interactive technologies that are readily available today. Digital Musical Instruments (DMIs) present musicians with performance challenges that are unique to this form of computer music. One of the most significant deviations from conventional acoustic musical instruments is the level of physical feedback conveyed by the instrument to the user. Currently, new interfaces for musical expression are not designed to be as physically communicative as acoustic instruments. Specifically, DMIs are often void of haptic feedback and therefore lack the ability to impart important performance information to the user. Moreover, there currently is no standardised way to measure the effect of this lack of physical feedback. Best practice would expect that there should be a set of methods to effectively, repeatedly, and quantifiably evaluate the functionality, usability, and user experience of DMIs. Earlier theoretical and technological applications of haptics have tried to address device performance issues associated with the lack of feedback in DMI designs and it has been argued that the level of haptic feedback presented to a user can significantly affect the user’s overall emotive feeling towards a musical device. The outcome of the investigations contained within this thesis are intended to inform new haptic interface.

El ejercicio y el desempeño musical: doce semanas de actividad física mejoran la capacidad de ejecución de un instrumento musical en estudiantes de conservatorio

Con el fin de evaluar los cambios que produce un programa de actividad física en la percepción que tienen los músicos sobre su capacidad de ejecución de un instrumento musical, se realizó una intervención con un programa de actividad física basado en la técnica Pilates, durante 12 semanas, en el Conservatorio de Música de la Universidad Nacional de Colombia, con estudiantes del programa de música instrumental. Se midieron parámetros de la aptitud física (capacidad cardiorrespiratoria, fuerza, flexibilidad y composición corporal) y de la percepción de la capacidad de ejecución (fatiga muscular, nivel de esfuerzo, dolor y fluidez) antes y después de la intervención. Los resultados arrojaron cambios positivos en la aptitud física logrando un aumento significativo en la flexibilidad y resistencia de los miembros inferiores en 14 participantes (70% de la muestra), y en la percepción de la capacidad de ejecución instrumental con el retraso en la aparición de la fatiga muscular mientras se está ejecutando el instrumento (30 minutos en promedio). Esto permite a los músicos abordar un repertorio extenso con menor fatiga, minimizando el riesgo de lesión o alteraciones musculo-esqueléticas que influyan directamente en su desempeño técnico y artístico.

Towards a definition of the performing AudioVisualist

The evolution of the laptop computer as a musical instrument in the 1990s provided a tool for empowering the solo musician and divergent approaches to the application of this technology in performance remain consistently debated.  The increasing ubiquity of digital media combined with the power of current generation notebook technology has provided the perfect platform to realise integrated audio-visual toolsets that respond to musical controllers and provide mixed-media results.  Despite emerging practitioners increasingly availing themselves to the musical affordances of this technology, theoretical discussion in the field ignores the various approaches a solo musician might take in developing integrated media works for performance.   In an increasingly crowded niche there is a clear compulsion to consider expanded modes of performance, yet lacking any formal framework these integrations can easily alienate an audience, distract from performance and lead to criticisms of novelty for novelty's sake. 

The HarmonyGrid : music, space and performance in grid music systems

This research explores music in space, as experienced through performing and music-making with interactive systems. It explores how musical parameters may be presented spatially and displayed visually with a view to their exploration by a musician during performance. Spatial arrangements of musical components, especially pitches and harmonies, have been widely studied in the literature, but the current capabilities of interactive systems allow the improvisational exploration of these musical spaces as part of a performance practice. This research focuses on quantised spatial organisation of musical parameters that can be categorised as grid music systems (GMSs), and interactive music systems based on them. The research explores and surveys existing and historical uses of GMSs, and develops and demonstrates the use of a novel grid music system designed for whole body interaction. Grid music systems provide plotting of spatialised input to construct patterned music on a two-dimensional grid layout. GMSs are navigated to construct a sequence of parametric steps, for example a series of pitches, rhythmic values, a chord sequence, or terraced dynamic steps. While they are conceptually simple when only controlling one musical dimension, grid systems may be layered to enable complex and satisfying musical results. These systems have proved a viable, effective, accessible and engaging means of music-making for the general user as well as the musician. GMSs have been widely used in electronic and digital music technologies, where they have generally been applied to small portable devices and software systems such as step sequencers and drum machines. This research shows that by scaling up a grid music system, music-making and musical improvisation are enhanced, gaining several advantages: (1) Full body location becomes the spatial input to the grid. The system becomes a partially immersive one in four related ways: spatially, graphically, sonically and musically. (2) Detection of body location by tracking enables hands-free operation, thereby allowing the playing of a musical instrument in addition to “playing” the grid system. (3) Visual information regarding musical parameters may be enhanced so that the performer may fully engage with existing spatial knowledge of musical materials. The result is that existing spatial knowledge is overlaid on, and combined with, music-space. Music-space is a new concept produced by the research, and is similar to notions of other musical spaces including soundscape, acoustic space, Smalley's “circumspace” and “immersive space” (2007, 48-52), and Lotis's “ambiophony” (2003), but is rather more textural and “alive”—and therefore very conducive to interaction. Music-space is that space occupied by music, set within normal space, which may be perceived by a person located within, or moving around in that space. Music-space has a perceivable “texture” made of tensions and relaxations, and contains spatial patterns of these formed by musical elements such as notes, harmonies, and sounds, changing over time. The music may be performed by live musicians, created electronically, or be prerecorded. Large-scale GMSs have the capability not only to interactively display musical information as music representative space, but to allow music-space to co-exist with it. Moving around the grid, the performer may interact in real time with musical materials in music-space, as they form over squares or move in paths. Additionally he/she may sense the textural matrix of the music-space while being immersed in surround sound covering the grid. The HarmonyGrid is a new computer-based interactive performance system developed during this research that provides a generative music-making system intended to accompany, or play along with, an improvising musician. This large-scale GMS employs full-body motion tracking over a projected grid. Playing with the system creates an enhanced performance employing live interactive music, along with graphical and spatial activity. Although one other experimental system provides certain aspects of immersive music-making, currently only the HarmonyGrid provides an environment to explore and experience music-space in a GMS.

Leisure activities and obesity in adolescence : a follow-up study among twins

The aims of this dissertation were 1) to investigate associations of weight status of adolescents with leisure activities, and computer and cell phone use, and 2) to investigate environmental and genetic influences on body mass index (BMI) during adolescence. Finnish twins born in 1983–1987 responded to postal questionnaires at the ages of 11-12 (5184 participants), 14 (4643 participants), and 17 years (4168 participants). Information was obtained on weight and height, leisure activities including television viewing, video viewing, computer games, listening to music, board games, musical instrument playing, reading, arts, crafts, socializing, clubs, sports, and outdoor activities, as well as computer and cell phone use. Activity patterns were studied using latent class analysis. The relationship between leisure activities and weight status was investigated using logistic and linear regression. Genetic and environmental effects on BMI were studied using twin modeling. Of individual leisure activities, sports were associated with decreased overweight risk among boys in both cross-sectional and longitudinal analyses, but among girls only cross-sectionally. Many sedentary leisure activities, such as video viewing (boys/girls), arts (boys), listening to music (boys), crafts (girls), and board games (girls), had positive associations with being overweight. Computer use was associated with a higher prevalence of overweight in cross-sectional analyses. However, musical instrument playing, commonly considered as a sedentary activity, was associated with a decreased overweight risk among boys. Four patterns of leisure activities were found: ‘Active and sociable’, ‘Active but less sociable’, ‘Passive but sociable’, and ‘Passive and solitary’. The prevalence of overweight was generally highest among the ‘Passive and solitary’ adolescents. Overall, leisure activity patterns did not predict overweight risk later in adolescence. An exception were 14-year-old ‘Passive and solitary’ girls who had the greatest risk of becoming overweight by 17 years of age. Heritability of BMI was high (0.58-0.83). Common environmental factors shared by family-members affected the BMI at 11-12 and 14 years but their effect had disappeared by 17 years of age. Additive genetic factors explained 90-96% of the BMI stability across adolescence. Genetic correlations across adolescence were high, which suggests similar genetic effects on BMI throughout adolescence, while unique environmental effects on BMI appeared to vary. These findings suggest that family-based interventions hold promise for obesity prevention into early and middle adolescence, but that later in adolescence obesity prevention should focus on individuals. A useful target could be adolescents' leisure time, and our findings highlight the importance of versatility in leisure activities.

Vapaa-ajan vieton luonnonvarojen kulutus

This thesis studies the use of natural resources for leisure time activities. The method used is Material Input per Service Unit (MIPS method). Leisure time has an increasing effect on the material flows of households, and that way has a bigger pressure on the environment. The most popular way of spending spare time in Finland is to watch TV and to listen to music or radio. Regardless of these, this thesis takes a closer look at boating, playing a musical instrument and visiting a theatre and tries to quantify their material flows. MIPS calculations of this thesis are case-studies and do not tell the whole truth about the hobbies. The aim was to have an overview about the magnitude of the activities. In the boating calculations, inside the system boundaries there are the boat itself, transport of the boat, outboard motor, gasoline consumption of the outboard motor, travelling to and from the harbour, and the harbour infrastructure. Calculations of playing a music instrument consider the instrument itself, music school and its maintenance, and travelling to the school. In the case of theatre the included things are theatre house and its maintenance, decor and costumes of the plays, transport of the decor, and travelling of the audience. The results of this thesis suggest that the biggest material flow of boating comes from travelling to and from the harbour and from the harbour infrastructure. The gasoline consumption of the outboard motor also makes a difference. One hour of boating with a rowing boat consumes 1 kg of abiotic materials. Boating with an outboard motorboat consumes astonishing 113 kg of abiotic resources. Visiting a music lesson for one hour consumes 9 kg of abiotic resources when travelling there by bus. One hour in a theatre play consumes 17 kg of abiotic materials when travelling by bus. Transport has a significant role on the resource consumption of leisure time activities.

Interpreting the input admittance of violins and guitars

The most widespread vibration measurement on musical instrument bodies is of the point mobility at the bridge. Analysis of such measurements is presented, with a view to assessing what range of information could feasibly be extracted from the corpus of data. Analysis approaches include (1) pole-residue extraction; (2) damping trend analysis based on time decay information; (3) statistical estimates based on SEA power-balance and variance theory. Comparative results are shown for some key quantities. Damping trends with frequency are shown to have unexpectedly different forms for violins and for guitars. Linear averaging to estimate the "direct field" component gives a simple and clear visualisation of any local resonance behaviour near the bridge, such as the "bridge hill", and reveals some violins that show a double hill, while viols show only weak hills, and guitars none at all. © S. Hirzel Verlag · EAA.

Haptic Sensation and Instrumental Transgression

Anyone who has ever played a musical instrument will certify the development of a particular type of relationship between the instrument and the performer. This relationship goes beyond a convenient coupling that is optimized for sound production. Every musical instrument defines ways in which to be touched, felt, activated. Music performance is dependent on bodily involvement that goes beyond the auditory and the sense of hearing. This article investigates the role of haptic sensation in the context of the performer-instrument relationship and draws on the writings of Georges Bataille to illuminate a discussion of the erotic in performance.

Cutaneous Grooves: Composing for the Sense of Touch

What if the traditional relationship between touch and music was essentially turned upside down, making the tactile sensation the aesthetic end? This paper presents a novel coupling of haptics technology and music, introducing the notion of tactile composition or aesthetic composition for the sense of touch. A system that facilitates the composition and perception of intricate, musically structured spatio-temporal patterns of vibration on the surface of the body is described. Relevant work from disciplines including sensory substitution, electronic musical instrument design, simulation design, entertainment technology, and visual music is considered. The psychophysical parameter space for our sense of touch is summarized and the building blocks of a compositional language for touch are explored. A series of concerts held for the skin and ears is described, as well as some of the lessons learned along the way. In conclusion, some potential evolutionary branches of tactile composition are posited.

Design Methodology for Real-Time FPGA-Based Sound Synthesis

Explicit finite difference (FD) schemes can realise highly realistic physical models of musical instruments but are computationally complex. A design methodology is presented for the creation of FPGA-based micro-architectures for FD schemes which can be applied to a range of applications with varying computational requirements, excitation and output patterns and boundary conditions. It has been applied to membrane and plate-based sound producing models, resulting in faster than real-time performance on a Xilinx XC2VP50 device which is 10 to 35 times faster than general purpose and DSP processors. The models have developed in such a way to allow a wide range of interaction (by a musician) thereby leading to the possibility of creating a highly realistic digital musical instrument.

Touching at a Distance: Resistance, Tactility, Proxemics and the Development of a Hybrid Virtual/Physical Performance System

This is a paper about resistance and affordance as they relate to music-making in the most extended sense, and perhaps about empathy if this is understood as a capacity to ‘read’ the resistances and affordances of objects, bodies, people and environments. It proceeds from a set of broad working assumptions which inform one individual’s musical practice, via a description a musical-instrument making project which is a hybrid of physical and virtual elements and is designed to test those assumptions, to a speculative finale in which it is suggested that musicking might, in some circumstances, be regarded in itself as a form of resistance. It moves from the intimate and personal, through what might be regarded as local concerns to more global observation, prefiguring the structure of the performance system it describes: the Virtual-Physical Feedback flute

Ambiguous Devices

Developed and performed in collaboration with Tom Davis (Bournemouth University), the work explores notions of presence and absence, technologically mediated communication and audience perception through the staging of ambiguous but repeatable performative interactions taking place on a co-located but distributed musical instrument. Public performances and installations include: CCRMA, Stanford University (2012); NIME conference, University of Michigan (2012); SARC, Queen's University Belfast (2013); INTIME symposium, Coventry University (2013); RE-NEW digital arts festival, Copenhagen (2013).

Categorizations of physical gesture in piano teaching: A preliminary enquiry

The significance of the “physicality” involved in learning to play a musical instrument and the essential role of teachers are areas in need of research. This article explores the role of gesture within teacher–student communicative interaction in one-to-one piano lessons. Three teachers were required to teach a pre-selected repertoire of two contrasting pieces to three students studying piano grade 1. The data was collected by video recordings of piano lessons and analysis based on the type and frequency of gestures employed by teachers in association to teaching behaviours specifying where gestures fit under (or evade) predefined classifications. Spontaneous co-musical gestures were observed in the process of piano tuition emerging with similar general communicative purposes as spontaneous co-verbal gestures and were essential for the process of musical communication between teachers and students. Observed frequencies of categorized gestures varied significantly between different teaching behaviours and between the three teachers. Parallels established between co-verbal and co-musical spontaneous gestures lead to an argument for extension of McNeill’s (2005) ideas of imagery–language–dialectic to imagery–music–dialectic with relevant implications for piano pedagogy and fields of study invested in musical communication.

A Spherical Array Approach for Simulation of Binaural Impulse Responses using the Finite Difference Time Domain Method

The finite difference time domain (FDTD) method has direct applications in musical instrument modeling, simulation of environmental acoustics, room acoustics and sound reproduction paradigms, all of which benefit from auralization. However, rendering binaural impulse responses from simulated
data is not straightforward to accomplish as the calculated pressure at FDTD grid nodes does not contain any directional information. This paper addresses this issue by introducing a spherical array to capture sound pressure on a finite difference grid, and decomposing it into a plane-wave density
function. Binaural impulse responses are then constructed in the spherical harmonics domain by combining the decomposed grid data with free field head-related transfer functions. The effects of designing a spherical array in a Cartesian grid are studied, and emphasis is given to the relationships
between array sampling and the spatial and spectral design parameters of several finite-difference
schemes.