BitBox!: A case study interface for teaching real-time adaptive music composition for video games

Real-time adaptive music is now well-established as a popular medium, largely through its use in video game soundtracks. Commercial packages, such as fmod, make freely available the underlying technical methods for use in educational contexts, making adaptive music technologies accessible to students. Writing adaptive music, however, presents a significant learning challenge, not least because it requires a different mode of thought, and tutor and learner may have few mutual points of connection in discovering and understanding the musical drivers, relationships and structures in these works. This article discusses the creation of ‘BitBox!’, a gestural music interface designed to deconstruct and explain the component elements of adaptive composition through interactive play. The interface was displayed at the Dare Protoplay games exposition in Dundee in August 2014. The initial proof-of- concept study proved successful, suggesting possible refinements in design and a broader range of applications.

Agency and animation: the performance of interactive game characters

As an animator and practice-based researcher with a background in games development, I am interested in technological change in the video game medium, with a focus on the tools and technologies that drive game character animation and interactive story. In particular, I am concerned with the issue of ‘user agency’, or the ability of the end user to affect story development—a key quality of the gaming experience and essential to the aesthetics of gaming, which is defined in large measure by its interactive elements. In this paper I consider the unique qualities of the video game1 as an artistic medium and the impact that these qualities have on the production of animated virtual character performances. I discuss the somewhat oppositional nature of animated character performances found in games from recent years, which range from inactive to active—in other words, low to high agency. Where procedural techniques (based on coded rules of movement) are used to model dynamic character performances, the user has the ability to interactively affect characters in real-time within the larger sphere of the game. This game play creates a high degree of user agency. However, it lacks the aesthetic nuances of the more crafted sections of games: the short cut-scenes, or narrative interludes where entire acted performances are mapped onto game characters (often via performance capture)2 and constructed into relatively cinematic representations. While visually spectacular, cut-scenes involve minimal interactivity, so user agency is low. Contemporary games typically float between these two distinct methods of animation, from a focus on user agency and dynamically responsive animation to a focus on animated character performance in sections where the user is a passive participant. We tend to think of the majority of action in games as taking place via playable figures: an avatar or central character that represents a player. However, there is another realm of characters that also partake in actions ranging from significant to incidental: non-playable characters, or NPCs, which populate action sequences where game play takes place as well as cut scenes that unfold without much or any interaction on the part of the player. NPCs are the equivalent to supporting roles, bit characters, or extras in the world of cinema. Minor NPCs may simply be background characters or enemies to defeat, but many NPCs are crucial to the overall game story. It is my argument that, thus far, no game has successfully utilized the full potential of these characters to contribute toward development of interactive, high performance action. In particular, a type of NPC that I have identified as ‘pivotal’3—those constituting the supporting cast of a video game—are essential to the telling of a game story, particularly in genres that focus on story and characters: adventure games, action games, and role-playing games. A game story can be defined as the entirety of the narrative, told through non-interactive cut-scenes as well a interactive sections of play, and development of more complex stories in games clearly impacts the animation of NPCs. I argue that NPCs in games must be capable of acting with emotion throughout a game—in the cutscenes, which are tightly controlled, but also in sections of game play, where player agency can potentially alter the story in real-time. When the animated performance of NPCs and user agency are not continuous throughout the game, the implication is that game stories may be primarily told through short movies within games, making it more difficult to define video games animation as a distinct artistic medium.

The game jam movement: disruption, performance and artwork

This paper explores the current conventions and intentions of the game jam - contemporary events that encourage the rapid, collaborative creation of game design prototypes. Game jams are often renowned for their capacity to encourage creativity and the development of alternative, innovative game designs. However, there is a growing necessity for game jams to continue to challenge traditional development practices through evolving new formats and perspectives to maintain the game jam as a disruptive, refreshing aspect of game development culture. As in other creative jam style events, a game jam is not only a process but also, an outcome. Through a discussion of the literature this paper establishes a theoretical basis with which to analyse game jams as disruptive, performative processes that result in original creative artefacts. In support of this, case study analysis of Development Cultures: a series of workshops that centred on innovation and new forms of practice through play, chance, and experimentation, is presented. The findings indicate that game jams can be considered as processes that inspire creativity within a community and that the resulting performances can be considered as a form of creative artefact, thus parallels can be drawn between game jams and performative and interactive art.

Creative communities: shaping process through performance and play

This paper studies the use of play as a method to unlock creativity and innovation within a community of practice (a group of individuals who share a common interest and who see value in interaction to enhance their understanding). An analysis of communities of practice and the value of play informs evaluation of two case studies exploring the development of communities of practice, one within the discipline of videogames and one which bridges performing arts and videogames. The case studies provide qualitative data from which the potential of play as a method to inspire creativity and support the development of a potential community of practice is recognised. Establishing trust, disruption of process through play and reflection are key steps proposed in a ‘context provider’s framework’ for individuals or organisations to utilise in the design of activities to support creative process and innovation within a potential community of practice.

Postdigital play and the aesthetics of recruitment

This paper analyses reconfigurations of play in newly emergent material and digital configurations of game design. It extends recent work examining dimensions of hybridity in playful products by turning attention to interfaces, practices and spaces, rather than devices. We argue that the concept of hybrid play relies on predefining clear and distinct entities that then enter into hybrid situations. Drawing on concepts of the ‘interface’ and ‘postdigital’, we argue the distribution of computing devices creates difficulties for such presuppositions. Instead, we propose an ‘aesthetic of recruitment’ that is adequate to the new openness of social and technical play.

Playing with sound and gesture in digital audio games

This article introduces the genre of a digital audio game and discusses selected play interaction solutions implemented in the Audio Game Hub, a prototype designed and evaluated in the years 2014 and 2015 at the Gamification Lab at Leuphana University Lüneburg.1 The Audio Game Hub constitutes a set of familiar playful activities (aiming at a target, reflex-based reacting to sound signals, labyrinth exploration) and casual games (e.g. Tetris, Memory) adapted to the digital medium and converted into the audio sphere, where the player is guided predominantly or solely by sound. The authors will discuss the design questions raised at early stages of the project, and confront them with the results of user experience testing performed on two groups of sighted and one group of visually impaired gamers.

Pivoting the player: a framework for player character research in offline computer role-playing games

This article introduces a theoretical framework for the analysis of the player character (PC) in offline computer role-playing games (cRPGs). It derives from the assumption that the character constitutes the focal point of the game, around which all the other elements revolve. This underlying observation became the foundation of the Player Character Grid and its constituent Pivot Player Character Model, a conceptual framework illustrating the experience of gameplay as perceived through the PC’s eyes. Although video game characters have been scrutinised from many different perspectives, a systematic framework has not been introduced yet. This study aims to fill that void by proposing a model replicable across the cRPG genre. It has been largely inspired by Anne Ubersfeld’s semiological dramatic character research implemented in Reading Theatre I (1999) and is demonstrated with reference to The Witcher (CD Projekt RED 2007).

Pathfinder: a performance-game for the augmented drum-kit

Pathfinder is a performance-game for solo drummer, exploring the synergies between multiple contemporary creative practices. The work navigates between music composition, improvisation, projection/light art and game art. At its heart lies a bespoke electro-acoustic instrument, the augmented drum-kit, used not only to provide the sonic content of the work in real-time, but also as a highly expressive game controller that interacts with an instrument-specific game. The musical instrument offers a much wider range of expressive possibilities, control and tactile feedback in comparison to a traditional general-purpose game controller, and as a result it affords a more diverse and nuanced game play performance. Live electronics, lights, projections and the drum-kit all make up the performance-game’s universe, within which the performer has to explore, adapt, navigate and complete a journey.

Ludification of work or labourisation of play? On work-play interferences.

Paper presented at the 1st International Joint Conference of DiGRA and FDG Dundee, August 1-6, 2016.

Playful laboratories: the significance of games for knowledge production in the digital age

Presented at DiGRA 2015 Diversity of Play, Leuphana University, Lüneburg, Germany, on the 15th of May 2015.

Ludification of work or labourisation of play? On work/play interferences in the digital times.

Presented at Money and Games: 12th annual Game Research Lab Spring seminar, University of Tampere, Finland, on the 19th of April 2016.

Infants, interfaces, and intermediation: digital parenting and the production of 'iPad baby' videos on YouTube

We investigate the ways young children’s use of mobile touchscreen interfaces is both understood and shaped by parents through the production of YouTube videos and discussions in associated comment threads. This analysis expands on, and departs from, theories of parental mediation, which have traditionally been framed through a media effects approach in analyzing how parents regulate their children’s use of broadcast media, such as television, within family life. We move beyond the limitations of an effects framing through more culturally and materially oriented theoretical lenses of mediation, considering the role mobile interfaces now play in the lives of infants through analysis of the ways parents intermediate between domestic spaces and networked publics. We propose the concept of intermediation, which builds on insights from critical interface studies as well as cultural industries literature to help account for these expanded aspects of digital parenting. Here, parents are not simply moderating children’s media use within the home, but instead operating as an intermediary in contributing to online representations and discourses of children’s digital culture. This intermediary role of parents engages with ideological tensions in locating notions of “naturalness:” the iPad’s gestural interface or the child’s digital dexterity.