Social contraptions and embodied interaction

In this paper we introduce the idea of "social contraptions", which are interactive physical devices employed as designerly explorations of social relations as mediated by physical space and artefacts. We present two independent but related design explorations that were situated in fine art and industrial research contexts. We argue that these contraptions open up for exploration some interaction issues related to the theme of ’Embodied Facilitation'. This is particularly in relation to awareness and coordination between interactants as mediated by the spatial and material configuration of the contraptions. These methods, as well as the insights gained from them can contribute to the development of the emerging field of embodied interaction.

Marker-Based Embodied Interaction for Handheld Augmented Reality Games

This article deals with embodied user interfaces for handheld augmented reality games, which consist of both physical and virtual components. We have developed a number of spatial interaction techniques that optically capture the device's movement and orientation relative to a visual marker. Such physical interactions in 3-D space enable manipulative control of mobile games. In addition to acting as a physical controller that recognizes multiple game-dependent gestures, the mobile device augments the camera view with graphical overlays. We describe three game prototypes that use ubiquitous product packaging and other passive media as backgrounds for handheld augmentation. The prototypes can be realized on widely available off-the-shelf hardware and require only minimal setup and infrastructure support.

A review of locative media, mobile and embodied spatial interaction

Highlights ► Provides a review of the history and development of locative media. ► Outlines different human-computer interaction techniques applied in locative media. ► Discusses how locative media applications have changed interaction affordances in and of physical spaces. ► Discusses practices of people in urban settings that evolved through these new affordances. ► Provides an overview on methods to investigate and elaborate design principles for future locative media.

The embodied hybrid space : designing ubiquitous computing towards an amplification of situated real world experiences

The emergence of mobile and ubiquitous computing technology has created what is often referred to as the hybrid space – a virtual layer of digital information and interaction opportunities that sit on top of and augment the physical environment. Embodied media materialise digital information as observable and sometimes interactive parts of the physical environment. The aim of this work is to explore ways to enhance people’s situated real world experience, and to find out what the role and impact of embodied media in achieving this goal can be. The Edge, an initiative of the State Library of Queensland in Brisbane, Australia, and case study of this thesis, envisions to be a physical place for people to meet, explore, experience, learn and teach each other creative practices in various areas related to digital technology and arts. Guided by an Action Research approach, this work applies Lefebvre’s triad of space (1991) to investigate the Edge as a social space from a conceived, perceived and lived point of view. Based on its creators’ vision and goals on the conceived level, different embodied media are iteratively designed, implemented and evaluated towards shaping and amplifying the Edge’s visitor experience on the perceived and lived level.

Knowmore (House of Commons)

Knowmore (House of Commons) is a large scale generative interactive installation that incorporates embodied interaction, dynamic image creation, new furniture forms, touch sensitivity, innovative collaborative processes and multichannel generative sound creation. A large circular table spun by hand and a computer-controlled video projection falls on its top, creating an uncanny blend of physical object and virtual media. Participants’ presence around the table and how they touch it is registered, allowing up to five people to collaboratively ‘play’ this deeply immersive audiovisual work. Set within an ecological context, the work subtly asks what kind of resources and knowledges might be necessary to move us past simply knowing what needs to be changed to instead actually embodying that change, whilst hinting at other deeply relational ways of understanding and knowing the world. The work has successfully operated in two high traffic public environments, generating a subtle form of interactivity that allows different people to interact at different paces and speeds and with differing intentions, each contributing towards dramatic public outcomes. The research field involved developing new interaction and engagement strategies for eco-political media arts practice. The context was the creation of improved embodied, performative and improvisational experiences for participants; further informed by ‘Sustainment’ theory. The central question was, what ontological shifts may be necessary to better envision and align our everyday life choices in ways that respect that which is shared by all - 'The Commons'. The methodology was primarily practice-led and in concert with underlying theories. The work’s knowledge contribution was to question how new media interactive experience and embodied interaction might prompt participants to reflect upon the kind of resources and knowledges required to move past simply knowing what needs to be changed to instead actually embodying that change. This was achieved through focusing on the power of embodied learning implied by the works' strongly physical interface (i.e. the spinning of a full size table) in concert with the complex field of layered imagery and sound. The work was commissioned by the State Library of Queensland and Queensland Artworkers Alliance and significantly funded by The Australia Council for the Arts, Arts Queensland, QUT, RMIT Centre for Animation and Interactive Media and industry partners E2E Visuals. After premiering for 3 months at the State Library of Queensland it was curated into the significant ‘Mediations Biennial of Modern Art’ in Poznan, Poland. The work formed the basis of two papers, was reviewed in Realtime (90), was overviewed at Subtle Technologies (2010) in Toronto and shortlisted for ISEA 2011 Istanbul and included in the edited book/catalogue ‘Art in Spite of Economics’, a collaboration between Leonardo/ISAST (MIT Press); Goldsmiths, University of London; ISEA International; and Sabanci University, Istanbul.

Identifying with an avatar : a multidisciplinary perspective

Avatars perform a complex range of inter-related functions. They not only allow us to express a digital identity, they facilitate the expression of physical motility and, through non-verbal expression, help to mediate social interaction in networked environments. When well designed, they can contribute to a sense of “presence” (a sense of being there) and a sense of “co-presence” (a sense of being there with others) in digital space. Because of this complexity, the study of avatars can be enriched by theoretical insights from a range of disciplines. This paper considers avatars from the perspectives of critical theory, visual communication, and art theory (on portraiture) to help elucidate the role of avatars as an expression of identity. It goes on to argue that identification with an avatar is also produced through their expression of motility and discusses the benefits of film theory for explaining this process. Conceding the limits of this approach, the paper draws on philosophies of body image, Human Computer Interaction (HCI) theory on embodied interaction, and fields as diverse as dance to explain the sense of identification, immersion, presence and co-presence that avatars can produce.

The Remnant

An Interactive Installation with holographic 3D projections, satellite imagery, surround sound and intuitive body driven interactivity. Remnant (v.1) was commissioned by the 2010 TreeLine ecoArt event - an initiative of the Sunshine Coast Council and presented at a remnant block of subtropical rainforest called ‘Mary Cairncross Scenic Reserve’ - located 100kms north of Brisbane near the township of Maleny. V2 was later commissioned for KickArts Gallery, Cairns, re-presenting the work in a new open format which allowed audiences to both experience the original power of the work and to also understand the construction of the work's powerful illusory, visual spaces. This art-science project focused upon the idea of remnant landscapes - isolated blocks of forest (or other vegetation types) typically set within a patchwork quilt of surrounding farmed land. Participants peer into a mysterious, long tunnel of imagery whilst navigating entirely through gentle head movements - allowing them to both 'steer' in three dimensions and also 'alight', as a butterfly might, upon a sector of landscape - which in turn reveals an underlying 'landscape of mind'. The work challenges audiences to re-imagine our conceptions of country in ways that will lead us to better reconnect and sustain today’s heavily divided landscapes. The research field involved developing new digital image projection methods, alternate embodied interaction and engagement strategies for eco-political media arts practice. The context was the creation of improved embodied and improvisational experiences for participants, further informed by ‘eco-philosophical’ and sustainment theories. By engaging with deep conceptions of connectivity between apparently disparate elements, the work considered novel strategies for fostering new desires, for understanding and re-thinking the requisite physical and ecological links between ‘things’ that have been historically shattered. The methodology was primarily practice-led and in concert with underlying theories. The work’s knowledge contribution was to question how new media interactive experience and embodied interaction might prompt participants to reflect upon appropriate resources and knowledges required to generate this substantive desire for new approaches to sustainment. This accentuated through the power of learning implied by the works' strongly visual and kinaesthetic interface (i.e. the tunnel of imagery and the head and torso operated navigation). The work was commissioned by the 2010 TreeLine ecoArt event - an initiative of the Sunshine Coast Council and the second version was commissioned by Kickarts Gallery, Cairns, specifically funded by a national optometrist chain. It was also funded in development by Arts Queensland and reviewed in Realtime.

Dramatic character development personas to tailor apartment designs for different residential lifestyles

This chapter reports on research work that aims to overcome some limitations of conventional community engagement for urban planning. Adaptive and human-centred design approaches that are well established in human-computer interaction (such as personas and design scenarios) as well as creative writing and dramatic character development methods (such as the Stanislavsky System and the Meisner Technique) are yet largely unexplored in the rather conservative and long-term design context of urban planning. Based on these approaches, we have been trialling a set of performance based workshop activities to gain insights into participants’ desires and requirements that may inform the future design of apartments and apartment buildings in inner city Brisbane. The focus of these workshops is to analyse the behaviour and lifestyle of apartment dwellers and generate residential personas that become boundary objects in the cross-disciplinary discussions of urban design and planning teams. Dramatisation and embodied interaction of use cases form part of the strategies we employed to engage participants and elicit community feedback.

Intelligent physical modelling systems - Why?

“What did you think you were doing?” Was the question posed by the conference organizers to me as the inventor and constructor of the first working Tangible Interfaces over 40 years ago. I think the question was intended to encourage me to talk about the underlying ideas and intentionality rather than describe an endless sequence of electronic bricks and that is what I shall do in this presentation. In the sixties the prevalent idea for a graphics interface was an analogue with sketching which was to somehow be understood by the computer as three dimensional form. I rebelled against this notion for reasons which I will explain in the presentation and instead came up with tangible physical three dimensional intelligent objects. I called these first prototypes “Intelligent Physical Modelling Systems” which is a really dumb name for an obvious concept. I am eternally grateful to Hiroshi Ishii for coining the term “Tangible User Interfaces” - the same idea but with a much smarter name. Another motivator was user involvement in the design process, and that led to the Generator (1979) project with Cedric Price for the world’s first intelligent building capable of organizing itself in response to the appetites of the users. The working model of that project is in MoMA. And the same motivation led to a self builders design kit (1980) for Walter Segal which facilitated self-builders to design their own houses. And indeed as the organizer’s question implied, the motivation and intentionality of these projects developed over the years in step with advancing technology. The speaker will attempt to articulate these changes with medical, psychological and educational examples. Much of this later work indeed stemming from the Media Lab where we are talking. Related topics such as “tangible thinking” and “intelligent teacups” will be introduced and the presentation will end with some speculations for the future. The presentation will be given against a background of images of early prototypes many of which have never been previously published.

An observational study of how objects support engineering design thinking and communication : implications for the design of tangible media

There has been an increasing interest in objects within the HCI field particularly with a view to designing tangible interfaces. However, little is known about how people make sense of objects and how objects support thinking. This paper presents a study of groups of engineers using physical objects to prototype designs, and articulates the roles that physical objects play in supporting their design thinking and communications. The study finds that design thinking is heavily dependent upon physical objects, that designers are active and opportunistic in seeking out physical props and that the interpretation and use of an object depends heavily on the activity. The paper discusses the trade-offs that designers make between speed and accuracy of models, and specificity and generality in choice of representations. Implications for design of tangible interfaces are discussed.

Engaging bodily with video in design

Video is commonly used as a method for recording embodied interaction for purposes of analysis and design and has been proposed as a useful ‘material’ for interaction designers to engage with. But video is not a straight forward reproduction of embodied activity – in themselves video recordings ‘flatten’ the space of embodied interaction, they impose a perspective on unfolding action, and remove the embodied spatial and social context within which embodied interaction unfolds. This does not mean that video is not a useful medium with which to engage as part of a process of investigating and designing for embodied interaction – but crucially, it requires that as people attempting to engage with video, designers own bodies and bodily understandings must be engaged with and brought into play. This paper describes and reflects upon our experiences of engaging with video in two different activities as part of a larger research project investigating the design of gestural interfaces for a dental surgery context.

Learning beyond books : strategies for ambient media to improve libraries and collaboration spaces as interfaces for social learning

With the advent of digital media and online information resources, public libraries as physical destinations for information access are being increasingly challenged. As a response, many libraries follow the trend of removing bookshelves in order to provide more floorspace for social interaction and collaboration. Such spaces follow a Commons 2.0 model: they are designed to support collaborative work and social learning. The acquisition of skills and knowledge is facilitated as a result of being surrounded by and interacting with a community of likeminded others. Based on the results of a case study on a Commons 2.0 library space, this paper describes several issues of collaboration and social learning in public library settings. Acknowledging the significance of the architectural characteristics of the physical space, we discuss opportunities for ambient media to better reflect the social attributes of the library as a place; i.e. amplify the sense of other co-present library visitors and provide opportunities for shared encounters and conversations, which would remain invisible otherwise. We present the design of a user check-in system for improving the library as a physical destination for social learning, sharing, and inspiration for and by the community.

Habituated objects : everyday tangibles that foster the independent living of an elderly woman

The boundaries between 'the digital' and our everyday physical world are dissolving as we develop more physical ways of interacting with computing. This forum presents some of the topics discussed in the colorful multidisciplinary field of tangible and embodied interaction

Intimate Transactions Stage 4

A telepresence-based interactive installation allowing people at three sites (The National Art Museum of China, Beijing; The Imperial City Art Museum, Beijing; CalPoly University, California, USA) to interact simultaneously using only their bodies. Each participant used a physical interface called a ‘Bodyshelf’ and wore a sound vibration transmission device called a ‘haptic pendant’ around their necks. By gently moving their bodies and engaging through this ‘smart furniture’, they instigated ‘intimate transactions’, which influenced an evolving computationally-generated ‘world’ created from digital imagery, multichannel sound and tactile feedback. Intimate Transactions (Version 4) was the culmination of a long-term interdisciplinary research project developed in four distinct stages. It was launched in in 2008 and subsequently acquired on invitation by Professor Peter Weibel for the ZKM Media Art History Museum Karlsruhe in 2012.

Intimate transactions : art, exhibition and interaction within distributed network environments

The broad research questions of the book are: How can successful, interdisciplinary collaboration contribute to research innovation through Practice-led research? What contributes to the design, production and curation of successful new media art? What are the implications of exhibiting it across dual sites for artists, curators and participant audiences? Is it possible to create an 'intimate transaction' between people who are separated by vast distances but joined by interfaces and distributed networks? Centred on a new media work of the same name by the Transmute Collective (led by Keith Armstrong), this book provides insights from multidisciplinary perspectives. Visual, sound and performance artists, furniture designers, spatial architects, technology systems designers, and curators who collaborated in the production of Intimate Transactions discuss their design philosophies, working processes and resolution of this major new media work. Analytical and philosophical essays by international writers complement these writings on production. They consider how new media art, like Intimate Transactions, challenges traditional understandings of art, curatorial installation and exhibition experience because of the need to take into account interaction, the reconfiguration of space, co-presence, performativity and inter-site collaboration.