Face recognition across pose on video using eigen light-fields

We propose an approach to employ eigen light-fields for face recognition across pose on video. Faces of a subject are collected from video frames and combined based on the pose to obtain a set of probe light-fields. These probe data are then projected to the principal subspace of the eigen light-fields within which the classification takes place. We modify the original light-field projection and found that it is more robust in the proposed system. Evaluation on VidTIMIT dataset has demonstrated that the eigen light-fields method is able to take advantage of multiple observations contained in the video.

Designing and evaluating mobile multimedia user experiences in public urban places : Making sense of the field

The majority of the world’s population now lives in cities (United Nations, 2008) resulting in an urban densification requiring people to live in closer proximity and share urban infrastructure such as streets, public transport, and parks within cities. However, “physical closeness does not mean social closeness” (Wellman, 2001, p. 234). Whereas it is a common practice to greet and chat with people you cross paths with in smaller villages, urban life is mainly anonymous and does not automatically come with a sense of community per se. Wellman (2001, p. 228) defines community “as networks of interpersonal ties that provide sociability, support, information, a sense of belonging and social identity.” While on the move or during leisure time, urban dwellers use their interactive information communication technology (ICT) devices to connect to their spatially distributed community while in an anonymous space. Putnam (1995) argues that available technology privatises and individualises the leisure time of urban dwellers. Furthermore, ICT is sometimes used to build a “cocoon” while in public to avoid direct contact with collocated people (Mainwaring et al., 2005; Bassoli et al., 2007; Crawford, 2008). Instead of using ICT devices to seclude oneself from the surrounding urban environment and the collocated people within, such devices could also be utilised to engage urban dwellers more with the urban environment and the urban dwellers within. Urban sociologists found that “what attracts people most, it would appear, is other people” (Whyte, 1980, p. 19) and “people and human activity are the greatest object of attention and interest” (Gehl, 1987, p. 31). On the other hand, sociologist Erving Goffman describes the concept of civil inattention, acknowledging strangers’ presence while in public but not interacting with them (Goffman, 1966). With this in mind, it appears that there is a contradiction between how people are using ICT in urban public places and for what reasons and how people use public urban places and how they behave and react to other collocated people. On the other hand there is an opportunity to employ ICT to create and influence experiences of people collocated in public urban places. The widespread use of location aware mobile devices equipped with Internet access is creating networked localities, a digital layer of geo-coded information on top of the physical world (Gordon & de Souza e Silva, 2011). Foursquare.com is an example of a location based 118 Mobile Multimedia – User and Technology Perspectives social network (LBSN) that enables urban dwellers to virtually check-in into places at which they are physically present in an urban space. Users compete over ‘mayorships’ of places with Foursquare friends as well as strangers and can share recommendations about the space. The research field of Urban Informatics is interested in these kinds of digital urban multimedia augmentations and how such augmentations, mediated through technology, can create or influence the UX of public urban places. “Urban informatics is the study, design, and practice of urban experiences across different urban contexts that are created by new opportunities of real-time, ubiquitous technology and the augmentation that mediates the physical and digital layers of people networks and urban infrastructures” (Foth et al., 2011, p. 4). One possibility to augment the urban space is to enable citizens to digitally interact with spaces and urban dwellers collocated in the past, present, and future. “Adding digital layer to the existing physical and social layers could facilitate new forms of interaction that reshape urban life” (Kjeldskov & Paay, 2006, p. 60). This methodological chapter investigates how the design of UX through such digital placebased mobile multimedia augmentations can be guided and evaluated. First, we describe three different applications that aim to create and influence the urban UX through mobile mediated interactions. Based on a review of literature, we describe how our integrated framework for designing and evaluating urban informatics experiences has been constructed. We conclude the chapter with a reflective discussion on the proposed framework.

In the Beyond

In this video, phrases sourced from social networking websites are reformatted into a slowly rotating mandala form. As the text changes colour, an abstract, atmospheric soundtrack develops. “In the Beyond” examines how we construct and communicate notions of identity. It mixes language gleaned from social networks, and fuses them with a mandala form. Drawing on Zygmunt Bauman’s theoretical work on “liquid modernity”, this work questions how and where we find space for contemplation and reflection in a contemporary context increasingly defined by temporary social bonds and consumer choices.

Swimming but drowning

This single-channel digital video work documents a performative event, the disintegrative process as a drawing-object is progressively submerged within water. This work forms part of a series of video works examining ephemerality within art and the iconoclastic impulse. This is informed by Gustav Metzger’s conceptualisation of auto-destructive art (1959) as that which contains within itself an agent, which automatically leads to its destruction. Here, the ontological moment of the work corresponds to its demise, investigating a symbiotic relationship between creation and destruction. It addresses the phenomenology and epistemology of destruction in art when presented by the artist, as both a process and an ethical position. The work was included in the group show 'The Construct', curated by Kris Carlon for Artworkers' Residency and Exhibition Project 2006 at the State Library of Queensland.

Developing and enhancing independent learning skills : using video tutorials as a means of helping students help themselves

Purpose – The purpose of this paper is to examine the use of short video tutorials in a post-graduate accounting subject, as a means of helping students develop and enhance independent learning skills. Design/methodology/approach – In total, five short (approximately five to 10 minutes) video tutorials were introduced in an effort to shift the reliance for learning from the lecturer to the student. Data on students’ usage of online video tutorials, and comments by students in university questionnaires were collated over three semesters from 2008 to 2009. Interviews with students were then conducted in late 2009 to more comprehensively evaluate the use and perceived benefits of video tutorials. Findings – Findings reveal preliminary but positive outcomes in terms of both more efficient and effective teaching and learning. Research limitations/implications – The shift towards more independent learning through the use of video tutorials has positive implications for educators, employers, and professional accounting bodies; each of whom has identified the need for this skill in accounting graduates. Practical implications – The use of video tutorials has the potential for more rewarding teaching and more effective learning. Originality/value – This study is one of the first to examine the use and benefits of video tutorials as a means of developing independent learning skills in accountancy students – addressing a key concern within the profession.

Using cost-effective multimedia to create engaging learning experiences in law and other disciplines

This is the final report of an Australian Learning and Teaching Council Teaching Fellowship which addressed the needs of two separate groups of learners: (1) final year law students studying ethics and (2) law academics and other interested educators in higher education wishing to use information and communication technologies (ICT) to create engaging learning environments for their students but lacking the capacity to do so. The Fellowship resulted in final year law students being infused with an improved appreciation of ethical practice than they receive from traditional lecture/tutorial means by the development of an integrated program of blended learning including an online program entitled "Entry into Valhalla". This "ethics capstone‟ utilises multimedia produced using cost effective resources (including the "Second Life" virtual environment) to create engaging, contextualised learning experiences. The Fellowship also constructed the knowledge of producing cost-effective multimedia projects in other law academics and other educators in higher education by staff development activities comprising workshops, conference presentations and an interactive website using the "Entry into Valhalla" program as a case study exemplar.

Everything

Everything (2008) is a looped 3 channel digital video (extracted from a 3D computer animation) that appropriates a range of media including photography, drawing, painting, and pre-shot video. The work departs from traditional time-based video which is generally based on a recording of an external event. Instead, “Everything” constructs an event and space more like a painting or drawing might. The works combines constructed events (including space, combinations of objects, and aesthetic relationship of forms) with pre-recorded video footage and pre-made paintings and drawings. The result is a montage of objects, images – both still and moving – and abstracted ‘painterly’ gestures. This technique creates a complex temporal displacement. 'Past' refers to pre-recorded media such as painting and photography, and 'future' refers to a possible virtual space not in the present, that these objects may occupy together. Through this simultaneity between the real and the virtual, the work comments on a disembodied sense of space and time, while also puncturing the virtual with a sense of materiality through the tactility of drawing and painting forms and processes. In so doing, te work challenges the perspectival Cartesian space synonymous with the virtual. In this work the disembodied wandering virtual eye is met with an uncanny combination of scenes, where scale and the relationships between objects are disrupted and changed. Everything is one of the first international examples of 3D animation technology being utilised in contemporary art. The work won the inaugural $75,000 Premier of Queensland National New Media Art Award and was subsequently acquired by the Queensland Art Gallery. The work has been exhibited and reviewed nationally and internationally.

Relics

Relics is a single-channel video derived from a 3D computer animation that combines a range of media including photography, drawing, painting, and pre-shot video. It is constructed around a series of pictorial stills which become interlinked by the more traditionally filmic processes of panning, zooming and crane shots. In keeping with these ideas, the work revolves around a series of static architectural forms within the strangely menacing enclosure of a geodesic dome. These clinical aspects of the work are complemented by a series of elements that evoke fluidity : fireworks, mirrored biomorphic forms and oscillating projections. The visual dimension of the work is complemented by a soundtrack of rainforest bird calls. Through its ambiguous combination of recorded and virtual imagery, Relics explores the indeterminate boundaries between real and virtual space. On the one hand, it represents actual events and spaces drawn from the artist studio and image archive; on the other it represents the highly idealised spaces of drawing and 3D animation. In this work the disembodied wandering virtual eye is met with an uncanny combination of scenes, where scale and the relationships between objects are disrupted and changed. Through this simultaneity between the real and the virtual, the work conveys a disembodied sense of space and time that carries a powerful sense of affect. Relics was among the first international examples of 3D animation technology in contemporary art. It was originally exhibited in the artist’s solo show, ‘Places That Don’t Exist’ (2007, George Petelin Gallery, Gold Coast) and went on to be included in the group shows ‘d/Art 07/Screen: The Post Cinema Experience’ (2007, Chauvel Cinema, Sydney) , ‘Experimenta Utopia Now: International Biennial of Media Art’ (2010, Arts Centre, Melbourne and national touring venues) and ‘Move on Asia’ (2009, Alternative space Loop, Seoul and Para-site Art Space, Hong Kong) and was broadcast on Souvenirs from Earth (Video Art Cable Channel, Germany and France). The work was analysed in catalogue texts for ‘Places That Don’t Exist’ (2007), ‘d/Art 07’ (2007) and ‘Experimenta Utopia Now’ (2010) and the’ Souvenirs from Earth’ website.

The landing

'The Landing was an exhibition of paintings held at Gallery Barry Keldoulis in Sydney in 2008. The exhibition comprised 7 paintings and a 3-channel video work. The show built upon the artist's interest in drawing parallels between, while simultaneously disrupting the hermetic integrity of both the painted and virtual surface. In this exhibition this was drawn out by the intentionally scratchy and blanched qualities of the painted surface contrasted against the careful delineations and gradation of their subject matter which was drawn exclusively from virtual 3D animated spaces. Conversely, the video work collapsed a series of picture-perfect objects into a dense and incoherent whole. These tensions and slippages act as manifestations of the indeterminacies that frame our subjectivity more broadly - as artist or viewer. As Barry Keldoulis writes: 'Alwast’s practice engages the construction of ‘reality’ in both the digital and painterly worlds. His seamless stitching together of the various modes of virtual reality... is in this exhibition contrasted with his paintings, which show a fondness for what many now see as the quaint naivety of the medium, and the foibles of humanity the painted surface exudes when compared to the clinical exactitude of the virtual world.'(2008, http://www.gbk.com.au/artists/peter-alwast/the-landing) Works from 'The Landing' were included in the exhibition 'Temperature 2: New Queensland Art' (2009, curated by Frank McBride) at the Museum of Brisbane.

Measuring bitrate and quality trade-off in a fast region-of-interest based video coding

Prevailing video adaptation solutions change the quality of the video uniformly throughout the whole frame in the bitrate adjustment process; while region-of-interest (ROI)-based solutions selectively retains the quality in the areas of the frame where the viewers are more likely to pay more attention to. ROI-based coding can improve perceptual quality and viewer satisfaction while trading off some bandwidth. However, there has been no comprehensive study to measure the bitrate vs. perceptual quality trade-off so far. The paper proposes an ROI detection scheme for videos, which is characterized with low computational complexity and robustness, and measures the bitrate vs. quality trade-off for ROI-based encoding using a state-of-the-art H.264/AVC encoder to justify the viability of this type of encoding method. The results from the subjective quality test reveal that ROI-based encoding achieves a significant perceptual quality improvement over the encoding with uniform quality at the cost of slightly more bits. Based on the bitrate measurements and subjective quality assessments, the bitrate and the perceptual quality estimation models for non-scalable ROI-based video coding (AVC) are developed, which are found to be similar to the models for scalable video coding (SVC).

Engaging bodily with video in design

Video is commonly used as a method for recording embodied interaction for purposes of analysis and design and has been proposed as a useful ‘material’ for interaction designers to engage with. But video is not a straight forward reproduction of embodied activity – in themselves video recordings ‘flatten’ the space of embodied interaction, they impose a perspective on unfolding action, and remove the embodied spatial and social context within which embodied interaction unfolds. This does not mean that video is not a useful medium with which to engage as part of a process of investigating and designing for embodied interaction – but crucially, it requires that as people attempting to engage with video, designers own bodies and bodily understandings must be engaged with and brought into play. This paper describes and reflects upon our experiences of engaging with video in two different activities as part of a larger research project investigating the design of gestural interfaces for a dental surgery context.

Probabilistic matching of image sets for video-based face recognition

We address the problem of face recognition on video by employing the recently proposed probabilistic linear discrimi-nant analysis (PLDA). The PLDA has been shown to be robust against pose and expression in image-based face recognition. In this research, the method is extended and applied to video where image set to image set matching is performed. We investigate two approaches of computing similarities between image sets using the PLDA: the closest pair approach and the holistic sets approach. To better model face appearances in video, we also propose the heteroscedastic version of the PLDA which learns the within-class covariance of each individual separately. Our experi-ments on the VidTIMIT and Honda datasets show that the combination of the heteroscedastic PLDA and the closest pair approach achieves the best performance.

The diagnostic accuracy of telegeriatrics for the diagnosis of dementia via video conferencing

Introduction The suitability of video conferencing (VC) technology for clinical purposes relevant to geriatric medicine is still being established. This project aimed to determine the validity of the diagnosis of dementia via VC. Methods This was a multisite, noninferiority, prospective cohort study. Patients, aged 50 years and older, referred by their primary care physician for cognitive assessment, were assessed at 4 memory disorder clinics. All patients were assessed independently by 2 specialist physicians. They were allocated one face-to-face (FTF) assessment (Reference standard – usual clinical practice) and an additional assessment (either usual FTF assessment or a VC assessment) on the same day. Each specialist physician had access to the patient chart and the results of a battery of standardized cognitive assessments administered FTF by the clinic nurse. Percentage agreement (P0) and the weighted kappa statistic with linear weight (Kw) were used to assess inter-rater reliability across the 2 study groups on the diagnosis of dementia (cognition normal, impaired, or demented). Results The 205 patients were allocated to group: Videoconference (n = 100) or Standard practice (n = 105); 106 were men. The average age was 76 (SD 9, 51–95) and the average Standardized Mini-Mental State Examination Score was 23.9 (SD 4.7, 9–30). Agreement for the Videoconference group (P0= 0.71; Kw = 0.52; P < .0001) and agreement for the Standard Practice group (P0= 0.70; Kw = 0.50; P < .0001) were both statistically significant (P < .05). The summary kappa statistic of 0.51 (P = .84) indicated that VC was not inferior to FTF assessment. Conclusions Previous studies have shown that preliminary standardized assessment tools can be reliably administered and scored via VC. This study focused on the geriatric assessment component of the interview (interpretation of standardized assessments, taking a history and formulating a diagnosis by medical specialist) and identified high levels of agreement for diagnosing dementia. A model of service incorporating either local or remote administered standardized assessments, and remote specialist assessment, is a reliable process for enabling the diagnosis of dementia for isolated older adults.

Adaptive bilateral filtering using saliency map for deblocking low bit rate videos

This paper proposes a novel approach to video deblocking which performs perceptually adaptive bilateral filtering by considering color, intensity, and motion features in a holistic manner. The method is based on bilateral filter which is an effective smoothing filter that preserves edges. The bilateral filter parameters are adaptive and avoid over-blurring of texture regions and at the same time eliminate blocking artefacts in the smooth region and areas of slow motion content. This is achieved by using a saliency map to control the strength of the filter for each individual point in the image based on its perceptual importance. The experimental results demonstrate that the proposed algorithm is effective in deblocking highly compressed video sequences and to avoid over-blurring of edges and textures in salient regions of image.

Improvisation in interactive music systems

This project investigates machine listening and improvisation in interactive music systems with the goal of improvising musically appropriate accompaniment to an audio stream in real-time. The input audio may be from a live musical ensemble, or playback of a recording for use by a DJ. I present a collection of robust techniques for machine listening in the context of Western popular dance music genres, and strategies of improvisation to allow for intuitive and musically salient interaction in live performance. The findings are embodied in a computational agent – the Jambot – capable of real-time musical improvisation in an ensemble setting. Conceptually the agent’s functionality is split into three domains: reception, analysis and generation. The project has resulted in novel techniques for addressing a range of issues in each of these domains. In the reception domain I present a novel suite of onset detection algorithms for real-time detection and classification of percussive onsets. This suite achieves reasonable discrimination between the kick, snare and hi-hat attacks of a standard drum-kit, with sufficiently low-latency to allow perceptually simultaneous triggering of accompaniment notes. The onset detection algorithms are designed to operate in the context of complex polyphonic audio. In the analysis domain I present novel beat-tracking and metre-induction algorithms that operate in real-time and are responsive to change in a live setting. I also present a novel analytic model of rhythm, based on musically salient features. This model informs the generation process, affording intuitive parametric control and allowing for the creation of a broad range of interesting rhythms. In the generation domain I present a novel improvisatory architecture drawing on theories of music perception, which provides a mechanism for the real-time generation of complementary accompaniment in an ensemble setting. All of these innovations have been combined into a computational agent – the Jambot, which is capable of producing improvised percussive musical accompaniment to an audio stream in real-time. I situate the architectural philosophy of the Jambot within contemporary debate regarding the nature of cognition and artificial intelligence, and argue for an approach to algorithmic improvisation that privileges the minimisation of cognitive dissonance in human-computer interaction. This thesis contains extensive written discussions of the Jambot and its component algorithms, along with some comparative analyses of aspects of its operation and aesthetic evaluations of its output. The accompanying CD contains the Jambot software, along with video documentation of experiments and performances conducted during the project.