Robust Multimodal Person Identification With Limited Training Data

This paper presents a novel method of audio-visual feature-level fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there are limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature representation and a modified cosine similarity are introduced to combine and compare bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal dataset created from the SPIDRE speaker recognition database and AR face recognition database with variable noise corruption of speech and occlusion in the face images. The system's speaker identification performance on the SPIDRE database, and facial identification performance on the AR database, is comparable with the literature. Combining both modalities using the new method of multimodal fusion leads to significantly improved accuracy over the unimodal systems, even when both modalities have been corrupted. The new method also shows improved identification accuracy compared with the bimodal systems based on multicondition model training or missing-feature decoding alone.

Audiovisual citation: a project update

Existing referencing systems frequently prove inadequate for the citation of moving image and sound media such as vidcasts, streaming television, sound files, un-catalogued archive footage, amateur content hosted online or
non-broadcast radio recordings. Back in 2009 and 2010 a British working group funded by Higher Education Funding Council for England (HEFCE) and co-ordinated by the British Universities Film and Video Council investigated this problem. This report documents the early stages of the project.

Robust bimodal person identification using face and speech with limited training data and corruption of both modalities

This paper presents a novel method of audio-visual fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there is a limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new representation and a modified cosine similarity are introduced for combining and comparing bimodal features with limited training data as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal data set created from the SPIDRE and AR databases with variable noise corruption of speech and occlusion in the face images. The new method has demonstrated improved recognition accuracy.

Cidade Museu

Through the concept of sonic resonance, the project Cidade Museu – Museum City explores five derelict or transitional spaces in the city of Viseu. The activation and capture of these spaces develops an audio- visual memory that reflects architectures, stories and experiences, while creating a sense of place through sounds and images.

The project brings together musicians with a background in contemporary music, electroacoustic music and improvisation and a visual artist focusing on photography and video.

Each member of the collective explores the selected spaces in order to activate them with the help of their respective instruments and through sound projection in an iterative process in which the source of activation gradually gives way to the characteristics of each space, their resonances and acoustic characteristics. The museum city (a nickname for the city of Viseu), in this performance, exposes the contrast between the grandeur and multi-faceted architecture of Viseu’s Cathedral with spaces that spread throughout the city waiting for a new future.

The performance in the Cathedral (Sé) is characterised by a trio ensemble, an eight channel sound system and video projecting audio recordings and images made in each of the five spaces. The audience is invited to explore the relations between the various buildings and their stories while being immersed in their resonances and visual projections.

The performance explores the following spaces in Viseu: the old Orfeão (music hall), an old wine cellar, a mansion home to the national road services, a house with its grounds in Rua Silva Gaio and an old slaughterhouse.

Picturing Probation: Exploring the utility of visual methods in comparative research

A previous review of research on the practice of offender supervision identified the predominant use of interview-based methodologies and limited use of other research approaches (Robinson and Svensson, 2013). It also found that most research has tended to be locally focussed (i.e. limited to one jurisdiction) with very few comparative studies. This article reports on the application of a visual method in a small-scale comparative study. Practitioners in five European countries participated and took photographs of the places and spaces where offender supervision occurs. The aims of the study were two-fold: firstly to explore the utility of a visual approach in a comparative context; and secondly to provide an initial visual account of the environment in which offender supervision takes place. In this article we address the first of these aims. We describe the application of the method in some depth before addressing its strengths and weaknesses. We conclude that visual methods provide a useful tool for capturing data about the environments in which offender supervision takes place and potentially provide a basis for more normative explorations about the practices of offender supervision in comparative contexts.

Economic cost of age-related macular degeneration: a review of recent research

As the population of most developed countries ages so the prevalence of diseases such as age-related macular degeneration (AMD) are likely to increase. To facilitate planning and informed debate regarding making provisions for this disease it is important that we have a clear understanding of the economic impact of visual impairment associated with AMD. In this paper we assess the state of current knowledge based on a review of published evidence in scientific journals. Based on our assessment of the evidence we argue that the paucity of research studies on the subject and wide variation in estimates produced from the few studies available make it difficult to assess with confidence the likely average direct cost-of-illness associated with AMD. We further argue that significant gaps in our understanding of the costs of AMD (particularly in respect of indirect costs) also exist. Current research should be augmented by more comprehensive studies.

Generalisation between opposing visuomotor rotations when each is associated with visual targets and movements of different amplitude

Previous studies have attempted to identify sources of contextual information which can facilitate dual adaptation to two variants of a novel environment, which are normally prone to interference. The type of contextual information previously used can be grouped into two broad categories: that which is arbitrary to the motor system, such as a colour cue, and that which is based on an internal property of the motor system, such as a change in movement effector. The experiments reported here examined whether associating visuomotor rotations to visual targets and movements of different amplitude would serve as an appropriate source of contextual information to enable dual adaptation. The results indicated that visual target and movement amplitude is not a suitable source of contextual information to enable dual adaptation in our task. Interference was observed in groups who were exposed to opposing visuomotor rotations, or a visuomotor rotation and no rotation, both when the onset of the visuomotor rotations was sudden, or occurred gradually over the course of training. Furthermore, the pattern of interference indicated that the inability to dual adapt was a result of the generalisation of learning between the two visuomotor mappings associated with each of the visual target and movement amplitudes. (C) 2008 Elsevier B.V. All rights reserved.

The contribution of visual feedback to visuomotor adaptation: How much and when?

We investigated the role of visual feedback in adapting to novel visuomotor environments. Participants produced isometric elbow torques to move a cursor towards visual targets. Following trials with no rotation, participants adapted to a 60 degrees rotation of the visual feedback before returning to the non-rotated condition. Participants received continuous visual feedback (CF) of cursor position during task execution or post-trial visual feedback (PF). With training, reductions of the angular deviations of the cursor path occurred to a similar extent and at a similar rate for CF and PF groups. However, upon re-exposure to the non-rotated environment only CF participants exhibited post-training aftereffects, manifested as increased angular deviation of the cursor path, with respect to the pre-rotation trials. These aftereffects occurred despite colour cues permitting identification of the change in environment. The results show that concurrent feedback permits automatic recalibration of the visuomotor mapping while post-trial feedback permits performance improvement via a cognitive strategy. (C) 2008 Elsevier B.V. All rights reserved.

Visual feedback alters the variations in corticospinal excitability that arise from rhythmic movements of the opposite limb

Augmented visual feedback can have a profound bearing on the stability of bimanual coordination. Indeed, this has been used to render tractable the study of patterns of coordination that cannot otherwise be produced in a stable fashion. In previous investigations (Carson et al. 1999), we have shown that rhythmic movements, brought about by the contraction of muscles on one side of the body, lead to phase-locked changes in the excitability of homologous motor pathways of the opposite limb. The present study was conducted to assess whether these changes are influenced by the presence of visual feedback of the moving limb. Eight participants performed rhythmic flexion-extension movements of the left wrist to the beat of a metronome (1.5 Hz). In 50% of trials, visual feedback of wrist displacement was provided in relation to a target amplitude, defined by the mean movement amplitude generated during the immediately preceding no feedback trial. Motor potentials (MEPs) were evoked in the quiescent muscles of the right limb by magnetic stimulation of the left motor cortex. Consistent with our previous observations, MEP amplitudes were modulated during the movement cycle of the opposite limb. The extent of this modulation was, however, smaller in the presence of visual feedback of the moving limb (FCR omega(2) =0.41; ECR omega(2)=0.29) than in trials in which there was no visual feedback (FCR omega(2)=0.51; ECR omega(2)=0.48). In addition, the relationship between the level of FCR activation and the excitability of the homologous corticospinal pathway of the opposite limb was sensitive to the vision condition; the degree of correlation between the two variables was larger when there was no visual feedback of the moving limb. The results of the present study support the view that increases in the stability of bimanual coordination brought about by augmented feedback may be mediated by changes in the crossed modulation of excitability in homologous motor pathways.