Interactive music formats : will audiences interact?

The purpose of this research was to conduct a pilot study of a prototype interactive music release format which sought to investigate the readiness of audiences to interact with an interactive alternative to a fixed recorded work. A prototype music interface was created for testing. The prototype was then tested on a sample of users to understand what factors might be critical to audience engagement. The research further investigated the potential implications of the interactive release format on musicians' creative process.

Improvisation in interactive music systems

This project investigates machine listening and improvisation in interactive music systems with the goal of improvising musically appropriate accompaniment to an audio stream in real-time. The input audio may be from a live musical ensemble, or playback of a recording for use by a DJ. I present a collection of robust techniques for machine listening in the context of Western popular dance music genres, and strategies of improvisation to allow for intuitive and musically salient interaction in live performance. The findings are embodied in a computational agent – the Jambot – capable of real-time musical improvisation in an ensemble setting. Conceptually the agent’s functionality is split into three domains: reception, analysis and generation. The project has resulted in novel techniques for addressing a range of issues in each of these domains. In the reception domain I present a novel suite of onset detection algorithms for real-time detection and classification of percussive onsets. This suite achieves reasonable discrimination between the kick, snare and hi-hat attacks of a standard drum-kit, with sufficiently low-latency to allow perceptually simultaneous triggering of accompaniment notes. The onset detection algorithms are designed to operate in the context of complex polyphonic audio. In the analysis domain I present novel beat-tracking and metre-induction algorithms that operate in real-time and are responsive to change in a live setting. I also present a novel analytic model of rhythm, based on musically salient features. This model informs the generation process, affording intuitive parametric control and allowing for the creation of a broad range of interesting rhythms. In the generation domain I present a novel improvisatory architecture drawing on theories of music perception, which provides a mechanism for the real-time generation of complementary accompaniment in an ensemble setting. All of these innovations have been combined into a computational agent – the Jambot, which is capable of producing improvised percussive musical accompaniment to an audio stream in real-time. I situate the architectural philosophy of the Jambot within contemporary debate regarding the nature of cognition and artificial intelligence, and argue for an approach to algorithmic improvisation that privileges the minimisation of cognitive dissonance in human-computer interaction. This thesis contains extensive written discussions of the Jambot and its component algorithms, along with some comparative analyses of aspects of its operation and aesthetic evaluations of its output. The accompanying CD contains the Jambot software, along with video documentation of experiments and performances conducted during the project.

Prediction and proactivity in real-time interactive music systems

We advocate for the use of predictive techniques in interactive computer music systems. We suggest that the inclusion of prediction can assist in the design of proactive rather than reactive computational performance partners. We summarize the significant role prediction plays in human musical decisions, and the only modest use of prediction in interactive music systems to date. After describing how we are working toward employing predictive processes in our own metacreation software we reflect on future extensions to these approaches.

Interactive music visualization: implementation, realization and evaluation

This thesis describes all process of the development of music visualization, starting with the implementation, followed by realization and then evaluation. The main goal is to have to knowledge of how the audience live performance experience can be enhanced through music visualization. With music visualization is possible to give a better understanding about the music feelings constructing an intensive atmosphere in the live music performance, which enhances the connection between the live music and the audience through visuals. These visuals have to be related to the live music, furthermore has to quickly respond to live music changes and introduce novelty into the visuals. The mapping between music and visuals is the focus of this project, in order to improve the relationship between the live performance and the spectators. The implementation of music visualization is based on the translation of music into graphic visualizations, therefore at the beginning the project was based on the existent works. Later on, it was decided to introduce new ways of conveying music into visuals. Several attempts were made in order to discover the most efficient mapping between music and visualization so people can fully connect with the performance. Throughout this project, those attempts resulted in several music visualizations created for four live music performances, afterwards it was produced an online survey to evaluate those live performances with music visualization. In the end, all conclusions are presented based on the results of the online survey, and also is explained which music elements should be depicted in the visuals, plus how those visuals should respond to the selected music elements.

Resilience by design: A participatory approach to designing an interactive digital application for promoting children's resilience

The promotion of resilience (the capacity of an individual or community to bounce back and recover from adversity) has become an important area of public health. In recent years it has expanded into the digital domain, and many online applications have been developed to promote children's resilience. In this study, it is argued that the majority of existing applications are limited because they take a didactic approach, and conceive of interaction as providing navigational choices. Because they simply provide information about resilience or replicate offline, scenario-based strategies, the understanding of resilience they provide is confined to a few, predetermined factors. In this study I propose a new, experiential approach to promoting resilience digitally. I define resilience as an emergent, situated and context-specific phenomenon. Using a Participatory Design model in combination with a salutogenic (strength-based) health methodology, this project has involved approximately 50 children as co-designers and co-researchers over two years. The children have contributed to the design of a new set of interactive resilience tools, which facilitate resilience promotion through dialogic and experiential learning. The major outcomes of this study include a new methodology for developing digital resilience tools, a new set of tools that have been developed and evaluated in collaboration with children and a set of design principles to guide future development. Beyond these initial and tangible outcomes, this study has also established that the benefits of introducing Participatory Design into a health promoting model rests primarily in the change of the role of children from "users" of technology and education to co-designers, where they assume a leadership role in both designing the tools and in directing their resilience learning.

The Odrysian Kingdom – a Concept of an Interactive Multimedia Application for Educational Purposes

The presentation of cultural heritage is difficult comprehensive and constantly updated topic. Researchers often focus more on the different techniques to digitize artifacts of cultural heritage. This work focuses on the overall shape and structure of future multimedia application whose specificity is determined by the topic - Odrysian kingdom. Below is presented a concept for structure and content-based information available for individual kings from Odryssae dynasty. Special attention is paid to the presentation of preserved artifacts associated with the reign of specific rulers. The main concept of multimedia application dedicated to the Odrysian kingdom, it is to be used in teaching programs related to cultural heritage and history of antiquity in universities. The aim of designers is that it can be modified easy for use in museums also.

Generative Music editorial

An issue on generative music in Contemporary Music Review allows space to explore many of these controversies, and to explore the rich algorithmic scene in contemporary practice, as well as the diverse origins and manifestations of such a culture. A roster of interesting exponents from both academic and arts practice backgrounds are involved, matching the broad spectrum of current work. Contributed articles range from generative algorithms in live systems, from live coding to interactive music systems to computer games, through algorithmic modelling of longer-term form, evolutionary algorithms, to interfaces between modalities and mediums, in algorithmic choreography. A retrospective on the intensive experimentation into algorithmic music and sound synthesis at the Institute of Sonology in the 1960s and 70s creates a complementary strand, as well as an open paper on the issues raised by open source, as opposed to proprietary, software and operating systems, with consequences in the creation and archiving of algorithmic work.

Performing with grid music systems

Grid music systems provide discrete geometric methods for simplified music-making by providing spatialised input to construct patterned music on a 2D matrix layout. While they are conceptually simple, grid systems may be layered to enable complex and satisfying musical results. Grid music systems have been applied to a range of systems from small portable devices up to larger systems. In this paper we will discuss the use of grid music systems in general and present an overview of the HarmonyGrid system we have developed as a new interactive performance system. We discuss a range of issues related to the design and use of larger-scale grid- based interactive performance systems such as the HarmonyGrid.

The HarmonyGrid : music, space and performance in grid music systems

This research explores music in space, as experienced through performing and music-making with interactive systems. It explores how musical parameters may be presented spatially and displayed visually with a view to their exploration by a musician during performance. Spatial arrangements of musical components, especially pitches and harmonies, have been widely studied in the literature, but the current capabilities of interactive systems allow the improvisational exploration of these musical spaces as part of a performance practice. This research focuses on quantised spatial organisation of musical parameters that can be categorised as grid music systems (GMSs), and interactive music systems based on them. The research explores and surveys existing and historical uses of GMSs, and develops and demonstrates the use of a novel grid music system designed for whole body interaction. Grid music systems provide plotting of spatialised input to construct patterned music on a two-dimensional grid layout. GMSs are navigated to construct a sequence of parametric steps, for example a series of pitches, rhythmic values, a chord sequence, or terraced dynamic steps. While they are conceptually simple when only controlling one musical dimension, grid systems may be layered to enable complex and satisfying musical results. These systems have proved a viable, effective, accessible and engaging means of music-making for the general user as well as the musician. GMSs have been widely used in electronic and digital music technologies, where they have generally been applied to small portable devices and software systems such as step sequencers and drum machines. This research shows that by scaling up a grid music system, music-making and musical improvisation are enhanced, gaining several advantages: (1) Full body location becomes the spatial input to the grid. The system becomes a partially immersive one in four related ways: spatially, graphically, sonically and musically. (2) Detection of body location by tracking enables hands-free operation, thereby allowing the playing of a musical instrument in addition to “playing” the grid system. (3) Visual information regarding musical parameters may be enhanced so that the performer may fully engage with existing spatial knowledge of musical materials. The result is that existing spatial knowledge is overlaid on, and combined with, music-space. Music-space is a new concept produced by the research, and is similar to notions of other musical spaces including soundscape, acoustic space, Smalley's “circumspace” and “immersive space” (2007, 48-52), and Lotis's “ambiophony” (2003), but is rather more textural and “alive”—and therefore very conducive to interaction. Music-space is that space occupied by music, set within normal space, which may be perceived by a person located within, or moving around in that space. Music-space has a perceivable “texture” made of tensions and relaxations, and contains spatial patterns of these formed by musical elements such as notes, harmonies, and sounds, changing over time. The music may be performed by live musicians, created electronically, or be prerecorded. Large-scale GMSs have the capability not only to interactively display musical information as music representative space, but to allow music-space to co-exist with it. Moving around the grid, the performer may interact in real time with musical materials in music-space, as they form over squares or move in paths. Additionally he/she may sense the textural matrix of the music-space while being immersed in surround sound covering the grid. The HarmonyGrid is a new computer-based interactive performance system developed during this research that provides a generative music-making system intended to accompany, or play along with, an improvising musician. This large-scale GMS employs full-body motion tracking over a projected grid. Playing with the system creates an enhanced performance employing live interactive music, along with graphical and spatial activity. Although one other experimental system provides certain aspects of immersive music-making, currently only the HarmonyGrid provides an environment to explore and experience music-space in a GMS.

eBILITY : from tool use to partnerships

In this article we identify how computational automation achieved through programming has enabled a new class of music technologies with generative music capabilities. These generative systems can have a degree of music making autonomy that impacts on our relationships with them; we suggest that this coincides with a shift in the music-equipment relationship from tool use to a partnership. This partnership relationship can occur when we use technologies that display qualities of agency. It raises questions about the kinds of skills and knowledge that are necessary to interact musically in such a partnership. These are qualities of musicianship we call eBility. In this paper we seek to define what eBility might consist of and how consideration of it might effect music education practice. The 'e' in eBility refers not only to the electronic nature of computing systems but also to the ethical, enabling, experiential and educational dimensions of the creative relationship with technologies with agency. We hope to initiate a discussion around differentiating what we term representational technologies from those with agency and begin to uncover the implications of these ideas for music educators in schools and communities. We hope also to elucidate the emergent theory and practice that has enabled the development of strategies for optimising this kind of eBility where the tool becomes partner. The identification of musical technologies with agency adds to the authors’ list of metaphors for technology use in music education that previously included tool, medium and instrument. We illustrate these ideas with examples and with data from our work with the jam2jam interactive music system. In this discussion we will outline our experiences with jam2jam as an example of a technology with agency and describe the aspects of eBility that interaction with it promotes.

Practices of interactive courseware at Malaysia primary smart school : usage standing and causes attributing

Malaysian teachers are now using interactive courseware in their teaching practices as part of teaching activity in classroom, to teach some subjects such as languages, mathematics, and science. Towards this, there is some debate going on the effectiveness of interactive courseware used in classroom. This paper reports data from interviews conducted with school principals and teachers from six selected primary Smart Schools across Malaysia.The interview conducted is looked on their current school practices and challenges in the implementation of interactive courseware in the classroom. The interview covers several aspects of their school facilities, the causes attributed and their concerns. However, the findings revealed that these six schools generally need more supports in the physical and technical form, in order to utilize the interactive courseware. Initially, the current support from the Malaysian Ministry of Education in physical form has proved that not very helpful. Thus, the respondents suggested that supports should be both: physical and technical. Therefore, this preliminary findings could be used as a pointer to the Malaysian government and other stakeholders to the improvement of interface design for future interactive courseware.

The adventure of the refrain : composing with improvised music

This practice-based inquiry investigates the process of composing notated scores using improvised solos by saxophonists John Butcher and Anthony Braxton. To compose with these improvised sources, I developed a new method of analysis and through this method I developed new compositional techniques in applying these materials into a score. This method of analysis and composition utilizes the conceptual language of Gilles Deleuze and Felix Guattari found in A Thousand Plateaus. The conceptual language of Deleuze and Guattari, in particular the terms assemblage, refrain and deterritorialization are discussed in depth to give a context for the philosophical origins and also to explain how the language is used in reference to improvised music and the compositional process. The project seeks to elucidate the conceptual language through the creative practice and in turn for the creative practice to clarify the use of the conceptual terminology. The outcomes of the research resulted in four notated works being composed. Firstly, Gravity, for soloist and ensemble based on the improvisational language of John Butcher and secondly a series of 3 studies titled Transbraxton Studies for solo instruments based on the improvisational-compositional language of Anthony Braxton. The implications of this research include the application of the analysis method to a number of musical contexts including: to be used in the process of composing with improvised music; in the study of style and authorship in solo improvisation; as a way of analyzing group improvisation; in the analysis of textural music including electronic music; and in the analysis of music from different cultures—particularly cultures where improvisation and per formative aspects to the music are significant to the overall meaning of the work. The compositional technique that was developed has further applications in terms of an expressive method of composing with non-metered improvised materials and one that merges well with the transcription method developed of notating pitch and sounds to a timeline. It is hoped that this research can open further lines of enquiry into the application of the conceptual ideas of Deleuze and Guattari to the analysis of more forms of music.

The design and development of music eScape: A new iPhone mood management music App

The physical, emotional, educational and social developmental challenges of adolescence can be associated with high levels of emotional vulnerability. Thus, the development of effective emotion-regulation strategies is crucial during this time period. Young people commonly use music to identify, express and regulate their emotions. Modern mobile technology provides an engaging, easily accessible means of assisting young people through music. A systematic contextual review identified 20 iPhone applications addressing emotions through music and two independent raters, using the Mobile App Rating Scale (MARS), evaluated the quality of the apps. Their characteristics, key features and overall quality will be presented. Three participatory design workshops (N=13, 6 males, 7 females; age 15-25) were conducted to explore young people’s use of music to enhance wellbeing. Young people were also asked to trial existing mood and music apps and to conceptualise their ultimate mood targeting music application. A thematic analysis of the participatory design workshops content identified the following music affect-regulation strategies: relationship building, modifying cognitions, modifying emotions, and immersing in emotions. The application of the key learnings from the mobile app review and participatory design workshops and the design and development of the music eScape app were presented and implications for future research was discussed.