Cascading appearance-based features for visual voice activity detection

The detection of voice activity is a challenging problem, especially when the level of acoustic noise is high. Most current approaches only utilise the audio signal, making them susceptible to acoustic noise. An obvious approach to overcome this is to use the visual modality. The current state-of-the-art visual feature extraction technique is one that uses a cascade of visual features (i.e. 2D-DCT, feature mean normalisation, interstep LDA). In this paper, we investigate the effectiveness of this technique for the task of visual voice activity detection (VAD), and analyse each stage of the cascade and quantify the relative improvement in performance gained by each successive stage. The experiments were conducted on the CUAVE database and our results highlight that the dynamics of the visual modality can be used to good effect to improve visual voice activity detection performance.

Wide-angle visual feature matching for outdoor localization

Wide-angle images exhibit significant distortion for which existing scale-space detectors such as the scale-invariant feature transform (SIFT) are inappropriate. The required scale-space images for feature detection are correctly obtained through the convolution of the image, mapped to the sphere, with the spherical Gaussian. A new visual key-point detector, based on this principle, is developed and several computational approaches to the convolution are investigated in both the spatial and frequency domain. In particular, a close approximation is developed that has comparable computation time to conventional SIFT but with improved matching performance. Results are presented for monocular wide-angle outdoor image sequences obtained using fisheye and equiangular catadioptric cameras. We evaluate the overall matching performance (recall versus 1-precision) of these methods compared to conventional SIFT. We also demonstrate the use of the technique for variable frame-rate visual odometry and its application to place recognition.

Cherbourg time : young black and deadly art 2003-2007

Art is most often at the margins of community life, seen as a distraction or entertainment only; an individual’s whim. It is generally seen as without a useful role to play in that community. This is a perception of grown-ups; children seem readily to accept an engagement with art making. Our research has shown that when an individual is drawn into a crafted art project where they have an actual involvement with the direction and production of the art work, then they become deeply engaged on multiple levels. This is true of all age groups. Artists skilled in community collaboration are able to produce art of value that transcends the usual judgements of worth. It gives people a licence to unfetter their imagination and then cooperatively be drawn back to a reachable visual solution. If you engage with children in a community, you engage the extended family at some point. The primary methodology was to produce a series of educationally valid projects at the Cherbourg State School that had a resonance into that community, then revisit and refine them where necessary and develop a new series that extended all of the positive aspects of them. This was done over a period of five years. The art made during this time is excellent. The children know it, as do their families, staff at the school, members of the local community and the others who have viewed it in exhibitions in far places like Brisbane and Melbourne. This art and the way it has been made has been acknowledged as useful by the children, teachers and the community, in educational and social terms. The school is a better place to be. This has been acknowledged by the children, teachers and the community The art making of the last five years has become an integral part of the way the school now operates and the influence of that has begun to seep into other parts of the community. Art needs to be taken from the margins and put to work at the centre.

Designing ePortfolios for music postgraduate study : a practice-led inquiry

In this research I have examined how ePortfolios can be designed for Music postgraduate study through a practice led research enquiry. This process involved designing two Web 2.0 ePortfolio systems for a group of five post graduate music research students. The design process revolved around the application of an iterative methodology called Software Develop as Research (SoDaR) that seeks to simultaneously develop design and pedagogy. The approach to designing these ePortfolio systems applied four theoretical protocols to examine the use of digitised artefacts in ePortfolio systems to enable a dynamic and inclusive dialogue around representations of the students work. The research and design process involved an analysis of existing software and literature with a focus upon identifying the affordances of available Web 2.0 software and the applications of these ideas within 21st Century life. The five post graduate music students each posed different needs in relation to the management of digitised artefacts and the communication of their work amongst peers, supervisors and public display. An ePortfolio was developed for each of them that was flexible enough to address their needs within the university setting. However in this first SoDaR iteration data gathering phase I identified aspects of the university context that presented a negative case that impacted upon the design and usage of the ePortfolios and prevented uptake. Whilst the portfolio itself functioned effectively, the university policies and technical requirements prevented serious use. The negative case analysis of the case study found revealed that Access and Control and Implementation, Technical and Policy Constraints protocols where limiting user uptake. From the semistructured interviews carried out as part of this study participant feedback revealed that whilst the participants did not use the ePortfolio system I designed, each student was employing Web 2.0 social networking and storage processes in their lives and research. In the subsequent iterations I then designed a more ‘ideal’ system that could be applied outside of the University context that draws upon the employment of these resources. In conclusion I suggest recommendations about ePortfolio design that considers what the applications of the theoretical protocols reveal about creative arts settings. The transferability of these recommendations are of course dependent upon the reapplication of the theoretical protocols in a new context. To address the mobility of ePortfolio design between Institutions and wider settings I have also designed a prototype for a business card sized USB portal for the artists’ ePortfolio. This research project is not a static one; it stands as an evolving design for a Web 2.0 ePortfolio that seeks to refer to users needs, institutional and professional contexts and the development of software that can be incorporated within the design. What it potentially provides to creative artist is an opportunity to have a dialogue about art with artefacts of the artist products and processes in that discussion.

Music, Mime & Metamorphosis: Interdisciplinary intersections, interactions and influences between music, mime and corporate communication

This doctoral thesis comprises three distinct yet related projects which investigate interdisciplinary practice across: music collaboration; mime performance; and corporate communication. Both the processes and underpinning research of these projects explore, expose and exploit areas where disparate and apparently conflicting fields of professional practice successfully and effectively; intersect, interact, and inform each other - rather than conflict - thereby enhancing each, both individually and collectively. Informed by three decades of professional practice across: music; stage performance; television; corporate communication; design; and tertiary education, the three projects have produced innovative, creative, and commercial viable outcomes, manifest in a variety of media including: music; written text; digital, audio/visual; and internet. In exploring new practice and creating new knowledge, these project outcomes clearly demonstrate the value and effectiveness of reconciling disparate fields of practice through the application of inter-disciplinary creativity and innovation to professional practice.

Dynamic visual features for audio-visual speaker verification

The cascading appearance-based (CAB) feature extraction technique has established itself as the state-of-the-art in extracting dynamic visual speech features for speech recognition. In this paper, we will focus on investigating the effectiveness of this technique for the related speaker verification application. By investigating the speaker verification ability of each stage of the cascade we will demonstrate that the same steps taken to reduce static speaker and environmental information for the visual speech recognition application also provide similar improvements for visual speaker recognition. A further study is conducted comparing synchronous HMM (SHMM) based fusion of CAB visual features and traditional perceptual linear predictive (PLP) acoustic features to show that higher complexity inherit in the SHMM approach does not appear to provide any improvement in the final audio-visual speaker verification system over simpler utterance level score fusion.

Collaborative digital media performance with generative music systems

Music is inherently active and interactive. Like technologies before them, digital systems provide a range of enhanced music performance opportunities. In this paper we outline the educational advantages of ensemble performance in which generative media systems are integrated. As a concrete example, we focus on our work with the jam2jam system which uses generative music processes to enhance collaborative music making. We suggest that our research points toward a new class of activities that maintain the well established benefits of ensemble performance while adding cultural and pedagogical value by leveraging the capabilities and cachet of digital media practices.

Lumia : art/light/motion

Lumia: art/light/motion is an exciting new media exhibition presented by State Library of Queensland in partnership with Queensland-based Kuuki collective artists Priscilla Bracks and Gavin Sade. The exhibition explored contemporary life and encourages thought about the future through an extraordinary collection of hand-crafted and interactive electronic creatures and installations. The beautifully crafted new media artworks in Lumia: art/light/motion combine the bespoke with art and technology to create strange but intriguing objects. Lumia invited audiences to play, learn and then ponder the way we live and the environmental and social implications of our choices.

DIScoveringABILITIES : creating interactive performances over the Internet

This article presents a case study that shows how a creative music educator uses the internet to enable participatory performance.

Lumia : Art | light | motion. The Art of Kuuki.

The State Library of Queensland is delighted to present Lumia: art/light/motion, a culmination of many years of collaboration by the Kuuki collective led by Priscilla Bracks and Gavin Sade. This extraordinary exhibition not only showcases the unique talent of these Queenslanders, it also opens up a world of future possibilities while re-presenting the past and present. These contemporary new media installations sit comfortably within the walls of the library as they are the distinctive products of inquisitive and philosophical minds. In a sense the exhibition highlights the longevity and purposefulness of a cultural learning institution, through the non-traditional use of data, information, research and collection interpretation. The exhibition simultaneously articulates one of our key objectives – to progress the state’s digital agenda. Two academic essays have been commissioned for this joint Kuuki and State Library of Queensland publication. The first is by artist and writer Paul Brown, who has specialised in art, science and technology since the late 1960s and in computational and generative art since the mid 1970s. Brown investigates the history of new media, which is celebrating its 60th anniversary, and clearly places Sade and Bracks at the forefront of this genre nationally. The second essay is by arts writer Linda Carroli, who has delved deeply into the thoughts and processes of the artists to bring to light the complex workings of the artists’ minds. The publication also features an interview Carroli conducted with the artists. This exhibition is playful, informative and contemplative. The audience is invited to play, and consequently to ponder the way we live and the environmental and social implications of our choices. The exhibition tempts us to travel deep into the Antarctic, plunge into the Great Barrier Reef, be swamped by an orchestra of crickets, enter the Charmed world and travel back in time to a Victorian parlour where you can interact with a ‘new-world’ lyrebird and consider a brave new world where our only link to the animal world is with robotic representations. In essence this exhibition is about ideas and knowledge and what better institution than the State Library of Queensland to partner such a project?. State Library is committed to preserving culture, exploring new media and creating new content as a lasting legacy of Queensland for all Queenslanders.

Computing in performing arts education and practice

The performing arts have traditionally made limited use of and showed limited acceptance of computing technology. There are cognitive, physical, environmental, and social influences on the use of computers in performing arts. This paper will examine those influences on the practice of computers in the performing arts and their implications for education in those areas. These implications for the learning environment include infrastructure, interface design, industrial design, and software functionality. Although many of the issues raised in this paper are common to all visual and performing arts, there are significant differences between them which require abstraction of the concepts presented in this paper beyond the more practical focus intended. In particular there are differences in the ways humans are involved in the presentation of a work, and the transitory verses static nature of time in art products.

Intimate transactions : art, exhibition and interaction within distributed network environments

The broad research questions of the book are: How can successful, interdisciplinary collaboration contribute to research innovation through Practice-led research? What contributes to the design, production and curation of successful new media art? What are the implications of exhibiting it across dual sites for artists, curators and participant audiences? Is it possible to create an 'intimate transaction' between people who are separated by vast distances but joined by interfaces and distributed networks? Centred on a new media work of the same name by the Transmute Collective (led by Keith Armstrong), this book provides insights from multidisciplinary perspectives. Visual, sound and performance artists, furniture designers, spatial architects, technology systems designers, and curators who collaborated in the production of Intimate Transactions discuss their design philosophies, working processes and resolution of this major new media work. Analytical and philosophical essays by international writers complement these writings on production. They consider how new media art, like Intimate Transactions, challenges traditional understandings of art, curatorial installation and exhibition experience because of the need to take into account interaction, the reconfiguration of space, co-presence, performativity and inter-site collaboration.