Access to meaningful relationships through virtual instruments and ensembles

This workshop focuses upon research about the qualities of community in music and of music in community facilitated by technologically supported relationships. Generative media systems present an opportunity for users to leverage computational systems to form new relationships through interactive and collaborative experiences. Generative music and art are a relatively new phenomenon that use procedural invention as a creative technique to produce music and visual media. Early systems have demonstrated the potential to provide access to collaborative ensemble experiences for users with little formal musical or artistic expertise. This workshop examines the relational affordances of these systems evidenced by selected field data drawn from the Network Jamming Project. These generative performance systems enable access to unique ensembles with very little musical knowledge or skill and offer the possibility of interactive relationships with artists and musical knowledge through collaborative performance. In this workshop we will focus on data that highlights how these simulated experiences might lead to understandings that may be of social benefit. Conference participants will be invited to jam in real time using virtual interfaces and to evaluate purposively selected video artifacts that demonstrate different kinds of interactive relationship with artists, peers, and community and that enrich the sense of expressive self. Theoretical insights about meaningful engagement drawn from the longitudinal and cross cultural experiences will underpin the discussion and practical presentation.

Challenges of Malaysian developers in creating good interfaces for interactive courseware

There are many reasons why interface design for interactive courseware fails to support quality of learning experiences. The causes such as the level of interactivity, the availability of the interfaces to interact with the end users and a lack of deep knowledge about the role of interface design by the designers in the development process are most acknowledged. Related to this, as a creator for the interactive courseware, generally the developers expect the resources that they produced are effective, accurate and robust. However, rarely do the developers have the opportunity to create good interfaces with the emphasis on time consuming, money and skill. Thus, some challenges faces by them in the interface design development can’t be underestimated as well. Therefore, their perspective of the interactive courseware is important to ensure the material and also the features of the interactive courseware can facilitate teaching and learning activity. Within this context in mind, this paper highlights the challenges that faces by the Malaysian developer from the ten face to face interviewed data gathered. It discusses from the Malaysian developer perspectives that involved in the development of interface design for interactive courseware for the Smart School Project. Particularly, in creating such a great interfaces, the highlights challenges will present within the constraints of time, curriculum demand, and competencies of the development team.

The HarmonyGrid : music, space and performance in grid music systems

This research explores music in space, as experienced through performing and music-making with interactive systems. It explores how musical parameters may be presented spatially and displayed visually with a view to their exploration by a musician during performance. Spatial arrangements of musical components, especially pitches and harmonies, have been widely studied in the literature, but the current capabilities of interactive systems allow the improvisational exploration of these musical spaces as part of a performance practice. This research focuses on quantised spatial organisation of musical parameters that can be categorised as grid music systems (GMSs), and interactive music systems based on them. The research explores and surveys existing and historical uses of GMSs, and develops and demonstrates the use of a novel grid music system designed for whole body interaction. Grid music systems provide plotting of spatialised input to construct patterned music on a two-dimensional grid layout. GMSs are navigated to construct a sequence of parametric steps, for example a series of pitches, rhythmic values, a chord sequence, or terraced dynamic steps. While they are conceptually simple when only controlling one musical dimension, grid systems may be layered to enable complex and satisfying musical results. These systems have proved a viable, effective, accessible and engaging means of music-making for the general user as well as the musician. GMSs have been widely used in electronic and digital music technologies, where they have generally been applied to small portable devices and software systems such as step sequencers and drum machines. This research shows that by scaling up a grid music system, music-making and musical improvisation are enhanced, gaining several advantages: (1) Full body location becomes the spatial input to the grid. The system becomes a partially immersive one in four related ways: spatially, graphically, sonically and musically. (2) Detection of body location by tracking enables hands-free operation, thereby allowing the playing of a musical instrument in addition to “playing” the grid system. (3) Visual information regarding musical parameters may be enhanced so that the performer may fully engage with existing spatial knowledge of musical materials. The result is that existing spatial knowledge is overlaid on, and combined with, music-space. Music-space is a new concept produced by the research, and is similar to notions of other musical spaces including soundscape, acoustic space, Smalley's “circumspace” and “immersive space” (2007, 48-52), and Lotis's “ambiophony” (2003), but is rather more textural and “alive”—and therefore very conducive to interaction. Music-space is that space occupied by music, set within normal space, which may be perceived by a person located within, or moving around in that space. Music-space has a perceivable “texture” made of tensions and relaxations, and contains spatial patterns of these formed by musical elements such as notes, harmonies, and sounds, changing over time. The music may be performed by live musicians, created electronically, or be prerecorded. Large-scale GMSs have the capability not only to interactively display musical information as music representative space, but to allow music-space to co-exist with it. Moving around the grid, the performer may interact in real time with musical materials in music-space, as they form over squares or move in paths. Additionally he/she may sense the textural matrix of the music-space while being immersed in surround sound covering the grid. The HarmonyGrid is a new computer-based interactive performance system developed during this research that provides a generative music-making system intended to accompany, or play along with, an improvising musician. This large-scale GMS employs full-body motion tracking over a projected grid. Playing with the system creates an enhanced performance employing live interactive music, along with graphical and spatial activity. Although one other experimental system provides certain aspects of immersive music-making, currently only the HarmonyGrid provides an environment to explore and experience music-space in a GMS.

Meaningful engagement with music composition

Musical value cannot be experienced without direct knowledge of music, and engagement with the interactive elements of materials, expressive character and structure. Through these channels something is communicated, something is transmitted, some residue of ‘meaning’ is left with us. When a work of art stirs us it is more than simply sensory stimulation or some kind of emotional indulgence. We are gaining knowledge and expanding our experience... contributing to knowledge of ourselves and of the world.

Music selection using a touch screen interface : effect of auditory and visual feedback on driving and usability

This study aimed to examine the effects on driving, usability and subjective workload of performing music selection tasks using a touch screen interface. Additionally, to explore whether the provision of visual and/or auditory feedback offers any performance and usability benefits. Thirty participants performed music selection tasks with a touch screen interface while driving. The interface provided four forms of feedback: no feedback, auditory feedback, visual feedback, and a combination of auditory and visual feedback. Performance on the music selection tasks significantly increased subjective workload and degraded performance on a range of driving measures including lane keeping variation and number of lane excursions. The provision of any form of feedback on the touch screen interface did not significantly affect driving performance, usability or subjective workload, but was preferred by users over no feedback. Overall, the results suggest that touch screens may not be a suitable input device for navigating scrollable lists.

Experience design and interactive software in music education research

This paper examines the integration of computing technologies into music education research in a way informed by constructivism. In particular, this paper focuses on an approach established by Jeanne Bamberger, which the author also employs, that integrates software design, pedagogical exploration, and the building of music education theory. In this tradition, researchers design software and associated activities to facilitate the interactive manipulation of musical structures and ideas. In short, this approach focuses on designing experiences and tools that support musical thinking and doing. In comparing the work of Jean Bamberger with that of the author, this paper highlights and discusses issues of significance and identifies lessons for future research.

Improvisation in interactive music systems

This project investigates machine listening and improvisation in interactive music systems with the goal of improvising musically appropriate accompaniment to an audio stream in real-time. The input audio may be from a live musical ensemble, or playback of a recording for use by a DJ. I present a collection of robust techniques for machine listening in the context of Western popular dance music genres, and strategies of improvisation to allow for intuitive and musically salient interaction in live performance. The findings are embodied in a computational agent – the Jambot – capable of real-time musical improvisation in an ensemble setting. Conceptually the agent’s functionality is split into three domains: reception, analysis and generation. The project has resulted in novel techniques for addressing a range of issues in each of these domains. In the reception domain I present a novel suite of onset detection algorithms for real-time detection and classification of percussive onsets. This suite achieves reasonable discrimination between the kick, snare and hi-hat attacks of a standard drum-kit, with sufficiently low-latency to allow perceptually simultaneous triggering of accompaniment notes. The onset detection algorithms are designed to operate in the context of complex polyphonic audio. In the analysis domain I present novel beat-tracking and metre-induction algorithms that operate in real-time and are responsive to change in a live setting. I also present a novel analytic model of rhythm, based on musically salient features. This model informs the generation process, affording intuitive parametric control and allowing for the creation of a broad range of interesting rhythms. In the generation domain I present a novel improvisatory architecture drawing on theories of music perception, which provides a mechanism for the real-time generation of complementary accompaniment in an ensemble setting. All of these innovations have been combined into a computational agent – the Jambot, which is capable of producing improvised percussive musical accompaniment to an audio stream in real-time. I situate the architectural philosophy of the Jambot within contemporary debate regarding the nature of cognition and artificial intelligence, and argue for an approach to algorithmic improvisation that privileges the minimisation of cognitive dissonance in human-computer interaction. This thesis contains extensive written discussions of the Jambot and its component algorithms, along with some comparative analyses of aspects of its operation and aesthetic evaluations of its output. The accompanying CD contains the Jambot software, along with video documentation of experiments and performances conducted during the project.

Exploration and reflection in interactive art : glass pond

Glass Pond is an interactive artwork designed to engender exploration and reflection through an intuitive, tangible interface and a simulation agent. It is being developed using iterative methods. A study has been conducted with the aim of illuminating user experience, interface, design, and performance issues.The paper describes the study methodology and process of data analysis including coding schemes for cognitive states and movements. Analysis reveals that exploration and reflection occurred as well as composing behaviours (unexpected). Results also show that participants interacted to varying degrees. Design discussion includes the artwork's (novel) interface and configuration.

Emergence in interactive art

This thesis is concerned with creating and evaluating interactive art systems that facilitate emergent participant experiences. For the purposes of this research, interactive art is the computer based arts involving physical participation from the audience, while emergence is when a new form or concept appears that was not directly implied by the context from which it arose. This emergent ‘whole’ is more than a simple sum of its parts. The research aims to develop understanding of the nature of emergent experiences that might arise during participant interaction with interactive art systems. It also aims to understand the design issues surrounding the creation of these systems. The approach used is Practice-based, integrating practice, evaluation and theoretical research. Practice used methods from Reflection-in-action and Iterative design to create two interactive art systems: Glass Pond and +-now. Creation of +-now resulted in a novel method for instantiating emergent shapes. Both art works were also evaluated in exploratory studies. In addition, a main study with 30 participants was conducted on participant interaction with +-now. These sessions were video recorded and participants were interviewed about their experience. Recordings were transcribed and analysed using Grounded theory methods. Emergent participant experiences were identified and classified using a taxonomy of emergence in interactive art. This taxonomy draws on theoretical research. The outcomes of this Practice-based research are summarised as follows. Two interactive art systems, where the second work clearly facilitates emergent interaction, were created. Their creation involved the development of a novel method for instantiating emergent shapes and it informed aesthetic and design issues surrounding interactive art systems for emergence. A taxonomy of emergence in interactive art was also created. Other outcomes are the evaluation findings about participant experiences, including different types of emergence experienced and the coding schemes produced during data analysis.

Prediction and proactivity in real-time interactive music systems

We advocate for the use of predictive techniques in interactive computer music systems. We suggest that the inclusion of prediction can assist in the design of proactive rather than reactive computational performance partners. We summarize the significant role prediction plays in human musical decisions, and the only modest use of prediction in interactive music systems to date. After describing how we are working toward employing predictive processes in our own metacreation software we reflect on future extensions to these approaches.

Building sensitising terms to understand free-play in open-ended interactive art environments

In this paper we introduce and discuss the nature of free-play in the context of three open-ended interactive art installation works. We observe the interaction work of situated free-play of the participants in these environments and, building on precedent work, devise a set of sensitising terms derived both from the literature and from what we observe from participants interacting there. These sensitising terms act as guides and are designed to be used by those who experience, evaluate or report on open-ended interactive art. That is, we propose these terms as a common-ground language to be used by participants communicating while in the art work to describe their experience, by researchers in the various stages of research process (observation, coding activity, analysis, reporting, and publication), and by inter-disciplinary researchers working across the fields of HCI and art. This work builds a foundation for understanding the relationship between free-play, open-ended environments, and interactive installations and contributes sensitising terms useful for the HCI community for discussion and analysis of open-ended interactive art works.

Evoking gesture in interactive art

In this paper, we describe an interactive artwork that uses large body gestures as its primary interactive mode. The artist intends the work to provoke active reflection in the audience by way of gesture and content. The technology is not the focus, rather the aim is to provoke memory, to elicit feelings of connective human experiences in a required-to-participate audience. We find the work provokes a diverse and contradictory set of responses. The methods used to understand this include qualitative methods common to evaluating interactive art works, as well as in-depth discussions with the artist herself. This paper is relevant to the Human - Centered Computing track because in all stages of the design of the work - as well as the evaluation - the focus is on the human aspect; the computing is designed to enable all-too-human responses.

'Music eScape': mHealth application

Research Background Young people’s avid use of mobile technologies in daily life has led to an increase in the design and research on mHealth (mobile health) interventions targeting young people. ‘Music eScape’ is a mobile based mood regulation app that uses an innovative approach to promoting young people’s wellbeing using music. Research Question The design, research, development and evaluation of ‘Music eScape’ addressed a number of research questions from across the fields of Psychology and Interactive and Visual Design. The specific design research question addressed was: How can interaction and visual design be utilized to promote and enable young people to effectively regulate their mood using music and how can the new design further promote their experience of empowerment, control and agency over actively directing their mood journey? Research Contribution Innovation and New Knowledge Through its unique visual interface design and interactivity, the application presents a novel approach to promoting young people’s wellbeing using music and a specific function that allows users to ‘draw’ their mood journey in order to generate a playlist. The mobile app is the first to contain a function that enables users to plan their mood journey and exercise a sense of agency, intentional choice and control over the mood shift and by extension, their wellbeing. The feature ‘drawing’ interface was designed by Oksana Zelenko using participatory design research and Russell’s circumplex model of affect (1980) to inform the key visual design concept and underpinning interaction design. Research Significance The significance of the design research component within the larger interdisciplinary practices that have informed ‘Music eScape’ (e.g. field of psychology, reported through journal articles and other related outcomes), is the unique visual and interactive presentation of participant data and music therapy research within the app interface and interaction design which improves and increases young people’s engagement with the health messages it contains. The industry quality standard is further demonstrated by the launch on Apple iTunes. This demonstrates the application meets the high professional requirements for national release and meets international standards. The app also creates a new benchmark for the quality of health apps on the market as it marks the industry release of a trialled evidence-based mHealth intervention co-designed with young people.

Interactive music formats : will audiences interact?

The purpose of this research was to conduct a pilot study of a prototype interactive music release format which sought to investigate the readiness of audiences to interact with an interactive alternative to a fixed recorded work. A prototype music interface was created for testing. The prototype was then tested on a sample of users to understand what factors might be critical to audience engagement. The research further investigated the potential implications of the interactive release format on musicians' creative process.

Mediated music makers : Constructing author images in popular music

The subject of the thesis is the mediated construction of author images in popular music. In the study, the construction of images is treated as a process in which artists, the media and the members of the audience participate. The notions of presented, mediated and compiled author images are used in explaining the mediation process and the various authorial roles of the agents involved. In order to explore the issue more closely, I analyse the author images of a group of popular music artists representing the genres of rock, pop and electronic dance music. The analysed material consists mostly of written media texts through which the artists authorial roles and creative responsibilities are discussed. Theoretically speaking, the starting points for the examination lie in cultural studies and discourse analysis. Even though author images may be conceived as intertextual constructions, the artist is usually presented as a recognizable figure whose purpose is to give the music its public face. This study does not, then, deal with musical authors as such, but rather with their public images and mediated constructions. Because of the author-based functioning of popular music culture and the idea of the artist s individual creative power, the collective and social processes involved in the making of popular music are often superseded by the belief in a single, originating authorship. In addition to the collective practices of music making, the roles of the media and the marketing machinery complicate attempts to clarify the sharing of authorial contributions. As the case studies demonstrate, the differences between the examined author images are connected with a number of themes ranging from issues of auteurism and stardom to the use of masked imagery and the blending of authorial voices. Also the emergence of new music technologies has affected not only the ways in which music is made, but also how the artist s authorial status and artistic identity is understood. In the study at hand, the author images of auteurs, stars, DJs and sampling artists are discussed alongside such varied topics as collective authorship, evaluative hierarchies, visual promotion and generic conventions. Taken altogether, the examined case studies shed light on the functioning of popular music culture and the ways in which musical authorship is (re)defined.