Talk2Me : engaging interactive installation environments

In this research I explore what elements there may be in common between tangible interactive-technology works that successfully engage their participants. An exploration of existing methods for obtaining useful evaluations for non-use and ambiguous environments forms a part of the discussion.

Building sensitising terms to understand free-play in open-ended interactive art environments

In this paper we introduce and discuss the nature of free-play in the context of three open-ended interactive art installation works. We observe the interaction work of situated free-play of the participants in these environments and, building on precedent work, devise a set of sensitising terms derived both from the literature and from what we observe from participants interacting there. These sensitising terms act as guides and are designed to be used by those who experience, evaluate or report on open-ended interactive art. That is, we propose these terms as a common-ground language to be used by participants communicating while in the art work to describe their experience, by researchers in the various stages of research process (observation, coding activity, analysis, reporting, and publication), and by inter-disciplinary researchers working across the fields of HCI and art. This work builds a foundation for understanding the relationship between free-play, open-ended environments, and interactive installations and contributes sensitising terms useful for the HCI community for discussion and analysis of open-ended interactive art works.

Knowmore (House of Commons)

Knowmore (House of Commons) is a large scale generative interactive installation that incorporates embodied interaction, dynamic image creation, new furniture forms, touch sensitivity, innovative collaborative processes and multichannel generative sound creation. A large circular table spun by hand and a computer-controlled video projection falls on its top, creating an uncanny blend of physical object and virtual media. Participants’ presence around the table and how they touch it is registered, allowing up to five people to collaboratively ‘play’ this deeply immersive audiovisual work. Set within an ecological context, the work subtly asks what kind of resources and knowledges might be necessary to move us past simply knowing what needs to be changed to instead actually embodying that change, whilst hinting at other deeply relational ways of understanding and knowing the world. The work has successfully operated in two high traffic public environments, generating a subtle form of interactivity that allows different people to interact at different paces and speeds and with differing intentions, each contributing towards dramatic public outcomes. The research field involved developing new interaction and engagement strategies for eco-political media arts practice. The context was the creation of improved embodied, performative and improvisational experiences for participants; further informed by ‘Sustainment’ theory. The central question was, what ontological shifts may be necessary to better envision and align our everyday life choices in ways that respect that which is shared by all - 'The Commons'. The methodology was primarily practice-led and in concert with underlying theories. The work’s knowledge contribution was to question how new media interactive experience and embodied interaction might prompt participants to reflect upon the kind of resources and knowledges required to move past simply knowing what needs to be changed to instead actually embodying that change. This was achieved through focusing on the power of embodied learning implied by the works' strongly physical interface (i.e. the spinning of a full size table) in concert with the complex field of layered imagery and sound. The work was commissioned by the State Library of Queensland and Queensland Artworkers Alliance and significantly funded by The Australia Council for the Arts, Arts Queensland, QUT, RMIT Centre for Animation and Interactive Media and industry partners E2E Visuals. After premiering for 3 months at the State Library of Queensland it was curated into the significant ‘Mediations Biennial of Modern Art’ in Poznan, Poland. The work formed the basis of two papers, was reviewed in Realtime (90), was overviewed at Subtle Technologies (2010) in Toronto and shortlisted for ISEA 2011 Istanbul and included in the edited book/catalogue ‘Art in Spite of Economics’, a collaboration between Leonardo/ISAST (MIT Press); Goldsmiths, University of London; ISEA International; and Sabanci University, Istanbul.

Disco Puppy

Disco Puppy is an energetic interactive installation for children (and parents) which stars three larger than life dancing puppies. The work was initially produced for the Ipswich Art Gallery and has since toured nationally.

Sustaining the sustainable? Developing a practice and problem-led new media praxis

In this chapter I introduce an ecological-philosophical approach to artmaking that has guided my work over the past 16 years. I call this ‘Ecosophical praxis’. To illustrate how this infuses and directs my research methodologies, I draw upon a case study called Knowmore (House of Commons), an emerging interactive installation due for first showings in late 2008. This allows me to tease out the complex interrelationships between research and practice within my work, and describe how they comment upon and model these eco-cultural theories. I conclude with my intentions and hopes for the continued emergence of a contemporary eco-political modality of new media praxis that self-reflexively questions how we might re-focus future practices upon ‘sustaining the sustainable’.

The Remnant

An Interactive Installation with holographic 3D projections, satellite imagery, surround sound and intuitive body driven interactivity. Remnant (v.1) was commissioned by the 2010 TreeLine ecoArt event - an initiative of the Sunshine Coast Council and presented at a remnant block of subtropical rainforest called ‘Mary Cairncross Scenic Reserve’ - located 100kms north of Brisbane near the township of Maleny. V2 was later commissioned for KickArts Gallery, Cairns, re-presenting the work in a new open format which allowed audiences to both experience the original power of the work and to also understand the construction of the work's powerful illusory, visual spaces. This art-science project focused upon the idea of remnant landscapes - isolated blocks of forest (or other vegetation types) typically set within a patchwork quilt of surrounding farmed land. Participants peer into a mysterious, long tunnel of imagery whilst navigating entirely through gentle head movements - allowing them to both 'steer' in three dimensions and also 'alight', as a butterfly might, upon a sector of landscape - which in turn reveals an underlying 'landscape of mind'. The work challenges audiences to re-imagine our conceptions of country in ways that will lead us to better reconnect and sustain today’s heavily divided landscapes. The research field involved developing new digital image projection methods, alternate embodied interaction and engagement strategies for eco-political media arts practice. The context was the creation of improved embodied and improvisational experiences for participants, further informed by ‘eco-philosophical’ and sustainment theories. By engaging with deep conceptions of connectivity between apparently disparate elements, the work considered novel strategies for fostering new desires, for understanding and re-thinking the requisite physical and ecological links between ‘things’ that have been historically shattered. The methodology was primarily practice-led and in concert with underlying theories. The work’s knowledge contribution was to question how new media interactive experience and embodied interaction might prompt participants to reflect upon appropriate resources and knowledges required to generate this substantive desire for new approaches to sustainment. This accentuated through the power of learning implied by the works' strongly visual and kinaesthetic interface (i.e. the tunnel of imagery and the head and torso operated navigation). The work was commissioned by the 2010 TreeLine ecoArt event - an initiative of the Sunshine Coast Council and the second version was commissioned by Kickarts Gallery, Cairns, specifically funded by a national optometrist chain. It was also funded in development by Arts Queensland and reviewed in Realtime.

Finitude (Mallee:Time)

An interactive installation with full body interface, digital projection, multi-touch sensitive screen surfaces, interactive 3D gaming software, motorised dioramas, 4.1 spatial sound & new furniture forms - investigating the cultural dimensions of sustainability through the lens of 'time'. “Time is change, time is finitude. Humans are a finite species. Every decision we make today brings that end closer, or alternatively pushes it further away. Nothing can be neutral”. Tony Fry DETAILS: Finitude (Mallee:Time) is a major new media/sculptural hybrid work premiered in 2011 in version 1 at the Ka-rama Motel for the Mildura Palimpsest #8 ('Collaborators and Saboteurs'). Each participant/viewer lies comfortably on their back on the double bed of Room 22. Directly above them, supported by a wooden structure, not unlike a house frame, is a semi-transparent Perspex screen that displays projected 3D imagery and is simultaneously sensitive to the lightest of finger touches. Depending upon the ever changing qualities of the projected image on this screen the participant can see through its surface to a series of physical dioramas suspended above, lit by subtle LED spotlighting. This diorama consists of a slowly rotating series of physical environments, which also include several animatronic components, allowing the realtime composition of whimsical ‘landscapes’ of both 'real' and 'virtual' media. Through subtle, non-didactic touch-sensitive interactivity the participant then has influence over both the 3D graphic imagery, the physical movements of the diorama and the 4.1 immersive soundscape, creating an uncanny blend of physical and virtual media. Five speakers positioned around the room deliver a rich interactive soundscape that responds both audibly and physically to interactions. VERSION 1, CONTEXT/THEORY: Finitude (Mallee: Time) is Version 1 of a series of presentations during 2012-14. This version has been inspired through a series of recent visits and residencies in the SW Victoria Mallee country. Further drawing on recent writings by post colonial author Paul Carter, the work is envisaged as an evolving ‘personal topography’ of place-discovery. By contrasting and melding readily available generalisations of the Mallee regions’ rational surfaces, climatic maps and ecological systems with what Carter calls “a fine capillary system of interconnected words, places, memories and sensations” generated through my own idiosyncratic research processes, Finitude (Mallee Time) invokes a “dark writing” of place through outside eyes - an approach that avoids concentration upon what 'everyone else knows', to instead imagine and develop a sense how things might be. This basis in re-imagining and re-invention becomes the vehicle for the work’s more fundamental intention - as a meditative re-imagination of 'time' (and region) as finite resources: Towards this end, every object, process and idea in the work is re-thought as having its own ‘time component’ or ‘residue’ that becomes deposited into our 'collective future'. Thought this way Finitude (Mallee Time) suggests the poverty of predominant images of time as ‘mechanism’ to instead envisage time as a plastic cyclical medium that we can each choose to ‘give to’ or ‘take away from’ our future. Put another way - time has become finitude.

Grounded Media: Expanding the Scope of Ecological Art Practices Within New Media Arts Culture

‘Grounded Media’ is a form of art practice focused around the understanding that our ecological crisis is also a cultural crisis, perpetuated by our sense of separation from the material and immaterial ecologies upon which we depend. This misunderstanding of relationships manifests not only as environmental breakdown, but also in the hemorrhaging of our social fabric. ‘Grounded Media’ is consistent with an approach to media art making that I name ‘ecosophical’ and ‘praxis-led’ – which seeks through a range of strategies, to draw attention to the integrity, diversity and efficacy of the biophysical, social and electronic environments of which we are an integral part. It undertakes this through particular choices of location, interaction design,participative strategies and performative direction. This form of working emerged out of the production of two major projects, Grounded Light [8] and Shifting Intimacies [9] and is evident in a recent prototypical wearable art project called In_Step [6]. The following analysis and reflections will assist in promoting new, sustainable roles for media artists who are similarly interested in attuning their practices.

Finitude (V03)

WHAT: An interactive installation with full body interface, digital projection, multi-touch sensitive screen surfaces, interactive 3D gaming software, motorised dioramas, 4.1 spatial sound & new furniture forms - investigating the cultural dimensions of sustainability through the lens of 'time'. “Time is change, time is finitude. Humans are a finite species. Every decision we make today brings that end closer, or alternatively pushes it further away. Nothing can be neutral”. Tony Fry DETAILS: Each participant/viewer lies comfortably on their back. Directly above them is a semi-transparent Perspex screen that displays projected 3D imagery and is simultaneously sensitive to the lightest of finger touches. Depending upon the ever changing qualities of the projected image on this screen the participant can see through its surface to a series of physical dioramas suspended above, lit by subtle LED spotlighting. This diorama consists of a slowly rotating series of physical environments, which also include several animatronic components, allowing the realtime composition of whimsical ‘landscapes’ of both 'real' and 'virtual' media. Through subtle, non-didactic touch-sensitive interactivity the participant then has influence over both the 3D graphic imagery, the physical movements of the diorama and the 4 channel immersive soundscape, creating an uncanny blend of physical and virtual media. Five speakers positioned around the room deliver a rich interactive soundscape that responds both audibly and physically to interactions.

Talk2Me : the art of augmenting conversations

This paper describes an interactive installation work set in a large dome space. The installation is an audio and physical re-rendition of an interactive writing work. In the original work, the user interacted via keyboard and screen while online. This rendition of the work retains the online interaction, but also places the interaction within a physical space, where the main 'conversation' takes place by the participant-audience speaking through microphones and listening through headphones. The work now also includes voice and SMS input, using speech-to-text and text-to-speech conversion technologies, and audio and displayed text for output. These additions allow the participant-audience to co-author the work while they participate in audible conversation with keyword-triggering characters (bots). Communication in the space can be person-to-computer via microphone, keyboard, and phone; person-to-person via machine and within the physical space; computer-to- computer; and computer-to-person via audio and projected text.

Artefact and iteration : circular entanglements

In this workshop proposal I discuss a case study physical computing environment named Talk2Me. This work was exhibited in February 2006 at The Block, Brisbane as an interactive installation in the early stages of its development. The major artefact in this work is a 10 metre wide X 3 metre high light-permeable white dome. There are other technologies and artefacts contained within the dome that make up this interactive environment. The dome artefact has impacted heavily on the design process, including shaping the types of interactions involved, the kinds of technologies employed, and the choice of other artefacts. In this workshop paper, I chart some of the various iterations Talk2Me has undergone in the design process.

Intimate Transactions Stage 4

A telepresence-based interactive installation allowing people at three sites (The National Art Museum of China, Beijing; The Imperial City Art Museum, Beijing; CalPoly University, California, USA) to interact simultaneously using only their bodies. Each participant used a physical interface called a ‘Bodyshelf’ and wore a sound vibration transmission device called a ‘haptic pendant’ around their necks. By gently moving their bodies and engaging through this ‘smart furniture’, they instigated ‘intimate transactions’, which influenced an evolving computationally-generated ‘world’ created from digital imagery, multichannel sound and tactile feedback. Intimate Transactions (Version 4) was the culmination of a long-term interdisciplinary research project developed in four distinct stages. It was launched in in 2008 and subsequently acquired on invitation by Professor Peter Weibel for the ZKM Media Art History Museum Karlsruhe in 2012.

The Instabooth

The InstaBooth is a portable demountable interactive installation for situated community engagement. Its aim is to give a voice to communities who can share their thoughts and ideas in an unstructured and playful way that combines digital technology with tangible materials. It is constructed from standard CNC-cut plywood stock and plans for its construction are available for others to download and use. Its modular design accommodates a range of bespoke interactive technologies, both analogue and digital, designed to facilitate the engagement process by providing means to present different materials and offer different ways to collect feedback. The appearance and interactions of the booth are designed to appeal to different demographics and foster an interactive discussion about a range of different topics such as change management, policy development, and urban planning.

Emergence in interactive art

This thesis is concerned with creating and evaluating interactive art systems that facilitate emergent participant experiences. For the purposes of this research, interactive art is the computer based arts involving physical participation from the audience, while emergence is when a new form or concept appears that was not directly implied by the context from which it arose. This emergent ‘whole’ is more than a simple sum of its parts. The research aims to develop understanding of the nature of emergent experiences that might arise during participant interaction with interactive art systems. It also aims to understand the design issues surrounding the creation of these systems. The approach used is Practice-based, integrating practice, evaluation and theoretical research. Practice used methods from Reflection-in-action and Iterative design to create two interactive art systems: Glass Pond and +-now. Creation of +-now resulted in a novel method for instantiating emergent shapes. Both art works were also evaluated in exploratory studies. In addition, a main study with 30 participants was conducted on participant interaction with +-now. These sessions were video recorded and participants were interviewed about their experience. Recordings were transcribed and analysed using Grounded theory methods. Emergent participant experiences were identified and classified using a taxonomy of emergence in interactive art. This taxonomy draws on theoretical research. The outcomes of this Practice-based research are summarised as follows. Two interactive art systems, where the second work clearly facilitates emergent interaction, were created. Their creation involved the development of a novel method for instantiating emergent shapes and it informed aesthetic and design issues surrounding interactive art systems for emergence. A taxonomy of emergence in interactive art was also created. Other outcomes are the evaluation findings about participant experiences, including different types of emergence experienced and the coding schemes produced during data analysis.

The ECOS Project. An interactive simulation to promote an ECO-friendly universe.

Custom designed for display on the Cube Installation situated in the new Science and Engineering Centre (SEC) at QUT, the ECOS project is a playful interface that uses real-time weather data to simulate how a five-star energy building operates in climates all over the world. In collaboration with the SEC building managers, the ECOS Project incorporates energy consumption and generation data of the building into an interactive simulation, which is both engaging to users and highly informative, and which invites play and reflection on the roles of green buildings. ECOS focuses on the principle that humans can have both a positive and negative impact on ecosystems with both local and global consequence. The ECOS project draws on the practice of Eco-Visualisation, a term used to encapsulate the important merging of environmental data visualization with the philosophy of sustainability. Holmes (2007) uses the term Eco-Visualisation (EV) to refer to data visualisations that ‘display the real time consumption statistics of key environmental resources for the goal of promoting ecological literacy’. EVs are commonly artifacts of interaction design, information design, interface design and industrial design, but are informed by various intellectual disciplines that have shared interests in sustainability. As a result of surveying a number of projects, Pierce, Odom and Blevis (2008) outline strategies for designing and evaluating effective EVs, including ‘connecting behavior to material impacts of consumption, encouraging playful engagement and exploration with energy, raising public awareness and facilitating discussion, and stimulating critical reflection.’ Consequently, Froehlich (2010) and his colleagues also use the term ‘Eco-feedback technology’ to describe the same field. ‘Green IT’ is another variation which Tomlinson (2010) describes as a ‘field at the juncture of two trends… the growing concern over environmental issues’ and ‘the use of digital tools and techniques for manipulating information.’ The ECOS Project team is guided by these principles, but more importantly, propose an example for how these principles may be achieved. The ECOS Project presents a simplified interface to the very complex domain of thermodynamic and climate modeling. From a mathematical perspective, the simulation can be divided into two models, which interact and compete for balance – the comfort of ECOS’ virtual denizens and the ecological and environmental health of the virtual world. The comfort model is based on the study of psychometrics, and specifically those relating to human comfort. This provides baseline micro-climatic values for what constitutes a comfortable working environment within the QUT SEC buildings. The difference between the ambient outside temperature (as determined by polling the Google Weather API for live weather data) and the internal thermostat of the building (as set by the user) allows us to estimate the energy required to either heat or cool the building. Once the energy requirements can be ascertained, this is then balanced with the ability of the building to produce enough power from green energy sources (solar, wind and gas) to cover its energy requirements. Calculating the relative amount of energy produced by wind and solar can be done by, in the case of solar for example, considering the size of panel and the amount of solar radiation it is receiving at any given time, which in turn can be estimated based on the temperature and conditions returned by the live weather API. Some of these variables can be altered by the user, allowing them to attempt to optimize the health of the building. The variables that can be changed are the budget allocated to green energy sources such as the Solar Panels, Wind Generator and the Air conditioning to control the internal building temperature. These variables influence the energy input and output variables, modeled on the real energy usage statistics drawn from the SEC data provided by the building managers.