An approach to statistical lip modelling for speaker identification via chromatic feature extraction

This paper presents a novel technique for the tracking of moving lips for the purpose of speaker identification. In our system, a model of the lip contour is formed directly from chromatic information in the lip region. Iterative refinement of contour point estimates is not required. Colour features are extracted from the lips via concatenated profiles taken around the lip contour. Reduction of order in lip features is obtained via principal component analysis (PCA) followed by linear discriminant analysis (LDA). Statistical speaker models are built from the lip features based on the Gaussian mixture model (GMM). Identification experiments performed on the M2VTS¹ database, show encouraging results

Effect of rod-cone interactions on mesopic visual performance mediated by chromatic and luminance pathways

We studied the effect of rod–cone interactions on mesopic visual reaction time (RT). Rod and cone photoreceptor excitations were independently controlled using a four-primary photostimulator. It was observed that (1) lateral rod–cone interactions increase the cone-mediated RTs; (2) the rod–cone interactions are strongest when rod sensitivity is maximal in a dark surround, but weaker with increased rod activity in a light surround; and (3) the presence of a dark surround nonselectively increased the mean and variability of chromatic (+L-M, S-cone) and luminance (L+M+S) RTs independent of the level of rod activity. The results demonstrate that lateral rod–cone interactions must be considered when deriving mesopic luminous efficiency using RT.

What is that place? Observations of the impact of environment colour through photographic analysis

Isolating the impact of a colour, or a combination of colours, is extremely difficult to achieve because it is difficult to remove other environmental elements such as sound, odours, light, and occasion from the experience of being in a place. In order to ascertain the impact of colour on how we interpret the world in day to day situations, the current study records participant responses to achromatic scenes of the built environment prior to viewing the same scene in colour. A number of environments were photographed in colour or copied from design books; and copies of the images saved as both colour and black/grey/white. An overview of the study will be introduced by firstly providing examples of studies which have linked colour to meaning and emotions. For example, yellow is said to be connected to happiness1 ; or red evokes feelings of anger2 or passion. A link between colour and the way we understand and/or feel is established however, there is a further need for knowledge of colour in context. In response to this need, the current achromatic/chromatic environmental study will be described and discussed in light of the findings. Finally, suggestions for future research are posed. Based on previous research the authors hypothesised that a shift in environmental perception by participants would occur. It was found that the impact of colour includes a shift in perception of aspects such as its atmosphere and youthfulness. Through studio-class discussions it was also noted that the predicted age of the place, the function, and in association, the potential users when colour was added (or deleted) were often challenged. It is posited that the ability of a designer (for example, interior designer, architect, or landscape architect) to design for a particular target group—user and/or clients will be enhanced through more targeted studies relating colour in situ. The importance of noting the perceptual shift for the participants in our study, who were young designers, is the realisation that colour potentially holds the power to impact on the identity of an architectural form, an interior space, and/or particular elements such as doorways, furniture settings, and the like.

Limits of spherical blur determined with an adaptive optics mirror

We extended an earlier study (Vision Research, 45, 1967–1974, 2005) in which we investigated limits at which induced blur of letter targets becomes noticeable, troublesome and objectionable. Here we used a deformable adaptive optics mirror to vary spherical defocus for conditions of a white background with correction of astigmatism; a white background with reduction of all aberrations other than defocus; and a monochromatic background with reduction of all aberrations other than defocus. We used seven cyclopleged subjects, lines of three high-contrast letters as targets, 3–6 mm artificial pupils, and 0.1–0.6 logMAR letter sizes. Subjects used a method of adjustment to control the defocus component of the mirror to set the 'just noticeable', 'just troublesome' and 'just objectionable' defocus levels. For the white-no adaptive optics condition combined with 0.1 logMAR letter size, mean 'noticeable' blur limits were ±0.30, ±0.24 and ±0.23 D at 3, 4 and 6 mm pupils, respectively. White-adaptive optics and monochromatic-adaptive optics conditions reduced blur limits by 8% and 20%, respectively. Increasing pupil size from 3–6 mm decreased blur limits by 29%, and increasing letter size increased blur limits by 79%. Ratios of troublesome to noticeable, and of objectionable to noticeable, blur limits were 1.9 and 2.7 times, respectively. The study shows that the deformable mirror can be used to vary defocus in vision experiments. Overall, the results of noticeable, troublesome and objectionable blur agreed well with those of the previous study. Attempting to reduce higher-order aberrations or chromatic aberrations, reduced blur limits to only a small extent.

Change in colour appearance of small defocused lights

Small long wavelength lights (≤ 1’ arc) change colour appearance with positive defocus, appearing yellow or white. I investigated influences of longitudinal chromatic aberration and monochromatic aberrations on colour appearance of small narrow band lights. Seven cyclopleged participants viewed a small light (1’ arc diameter, λmax range 510 - 628 nm) centred within a 4.6’ black annulus and surrounded by a uniform white field under photopic light levels. An optical trombone varied focus. Participants were required to vary the focus by moving the optical trombone in either positive or negative direction and report when they noticed a change in appearance of the defocused narrow band light. Longitudinal chromatic aberration was controlled using a Powell achromatizing lens and its doublet and triplet components that neutralized, doubled and reversed the eye’s chromatic aberration, respectively. Changes in colour appearance for a 628 nm light occurred without any lens at +0.5 ± 0.2D defocus and with the doublet at +0.6 ± 0.2 D. The achromatizing lens did not affect appearance and the phenomenon was evident with the triplet for negative defocus (-0.5 ± 0.3 D). Adaptive optics correction of astigmatism and higher order monochromatic aberration did not affect magnitude significantly. Colour changes occurred despite a range of participant L/M cone ratios. Direction of change in colour appearance was reversed for short compared to long wavelengths. We conclude that longitudinal chromatic aberrations, but not monochromatic aberrations, are involved in changing appearance of small lights with defocus. Additional neuronal mechanisms that may contribute to the colour changes are considered.

Effect of optical aberrations on the colour appearance of small lights

Purpose: Small red lights (one minute of arc or less) change colour appearance with positive defocus. We investigated the influence of longitudinal chromatic aberration and monochromatic aberrations on the colour appearance of small narrow band lights. Methods: Seven cyclopleged, trichromatic observers viewed a small light (one minute of arc, λmax = 510, 532, 550, 589, 620, 628 nm, approximately 19 per cent Weber contrast) centred within a black annulus (4.5 minutes of arc) and surrounded by a uniform white field (2,170 cd/m2). Pupil size was four millimetres. An optical trombone varied focus. Longitudinal chromatic aberration was controlled with a two component Powell achromatising lens that neutralises the eye’s chromatic aberration; a doublet that doubles and a triplet that reverses the eye’s chromatic aberration. Astigmatism and higher order monochromatic aberrations were corrected using adaptive optics. Results: Observers reported a change in appearance of the small red light (628 nm) without the Powell lens at +0.49 ± 0.21 D defocus and with the doublet at +0.62 ± 0.16 D. Appearance did not alter with the Powell lens, and five of seven observers reported the phenomenon with the triplet for negative defocus (-0.80 ± 0.47 D). Correction of aberrations did not significantly affect the magnitude at which the appearance of the red light changed (+0.44 ± 0.18 D without correction; +0.46 ± 0.16 D with correction). The change in colour appearance with defocus extended to other wavelengths (λmax = 510 to 620 nm), with directions of effects being reversed for short wavelengths relative to long wavelengths. Conclusions: Longitudinal chromatic aberrations but not monochromatic aberrations are involved in changing the appearance of small lights with defocus.

Effect of optical aberrations on the color appearance of small defocused lights

We investigated influences of optics and surround area on color appearance of defocused, small narrow band photopic lights (1’ arc diameter, λmax 510 - 628 nm) centered within a black annulus and surrounded by a white field. Participants included seven normal trichromats with L- or M-cone biased ratios. We controlled chromatic aberration with elements of a Powell achromatizing lens and corrected higher-order aberrations with an adaptive-optics system. Longitudinal chromatic aberrations, but not monochromatic aberrations, are involved in changing appearance of small lights with defocus. Surround field structure is important because color changes were not observed when lights were presented on a uniform white surround.

Thomas Young's contribution to visual optics : the Bakerian lecture "On the mechanism of the eye"

Thomas Young (1773-1829) carried out major pioneering work in many different subjects. In 1800 he gave the Bakerian Lecture of the Royal Society on the topic of the “mechanism of the eye”: this was published in the following year (Young, 1801). Young used his own design of optometer to measure refraction and accommodation, and discovered his own astigmatism. He considered the different possible origins of accommodation and confirmed that it was due to change in shape of the lens rather than to change in shape of the cornea or an increase in axial length. However, the paper also dealt with many other aspects of visual and ophthalmic optics, such as biometric parameters, peripheral refraction, longitudinal chromatic aberration, depth-of-focus and instrument myopia. These aspects of the paper have previously received little attention. We now give detailed consideration to these and other less-familiar features of Young’s work and conclude that his studies remain relevant to many of the topics which currently engage visual scientists.

The HarmonyGrid : music, space and performance in grid music systems

This research explores music in space, as experienced through performing and music-making with interactive systems. It explores how musical parameters may be presented spatially and displayed visually with a view to their exploration by a musician during performance. Spatial arrangements of musical components, especially pitches and harmonies, have been widely studied in the literature, but the current capabilities of interactive systems allow the improvisational exploration of these musical spaces as part of a performance practice. This research focuses on quantised spatial organisation of musical parameters that can be categorised as grid music systems (GMSs), and interactive music systems based on them. The research explores and surveys existing and historical uses of GMSs, and develops and demonstrates the use of a novel grid music system designed for whole body interaction. Grid music systems provide plotting of spatialised input to construct patterned music on a two-dimensional grid layout. GMSs are navigated to construct a sequence of parametric steps, for example a series of pitches, rhythmic values, a chord sequence, or terraced dynamic steps. While they are conceptually simple when only controlling one musical dimension, grid systems may be layered to enable complex and satisfying musical results. These systems have proved a viable, effective, accessible and engaging means of music-making for the general user as well as the musician. GMSs have been widely used in electronic and digital music technologies, where they have generally been applied to small portable devices and software systems such as step sequencers and drum machines. This research shows that by scaling up a grid music system, music-making and musical improvisation are enhanced, gaining several advantages: (1) Full body location becomes the spatial input to the grid. The system becomes a partially immersive one in four related ways: spatially, graphically, sonically and musically. (2) Detection of body location by tracking enables hands-free operation, thereby allowing the playing of a musical instrument in addition to “playing” the grid system. (3) Visual information regarding musical parameters may be enhanced so that the performer may fully engage with existing spatial knowledge of musical materials. The result is that existing spatial knowledge is overlaid on, and combined with, music-space. Music-space is a new concept produced by the research, and is similar to notions of other musical spaces including soundscape, acoustic space, Smalley's “circumspace” and “immersive space” (2007, 48-52), and Lotis's “ambiophony” (2003), but is rather more textural and “alive”—and therefore very conducive to interaction. Music-space is that space occupied by music, set within normal space, which may be perceived by a person located within, or moving around in that space. Music-space has a perceivable “texture” made of tensions and relaxations, and contains spatial patterns of these formed by musical elements such as notes, harmonies, and sounds, changing over time. The music may be performed by live musicians, created electronically, or be prerecorded. Large-scale GMSs have the capability not only to interactively display musical information as music representative space, but to allow music-space to co-exist with it. Moving around the grid, the performer may interact in real time with musical materials in music-space, as they form over squares or move in paths. Additionally he/she may sense the textural matrix of the music-space while being immersed in surround sound covering the grid. The HarmonyGrid is a new computer-based interactive performance system developed during this research that provides a generative music-making system intended to accompany, or play along with, an improvising musician. This large-scale GMS employs full-body motion tracking over a projected grid. Playing with the system creates an enhanced performance employing live interactive music, along with graphical and spatial activity. Although one other experimental system provides certain aspects of immersive music-making, currently only the HarmonyGrid provides an environment to explore and experience music-space in a GMS.

Dynamics of rod and cone photoreceptor interactions under mesopic light levels

Visual adaptation regulates contrast sensitivity during dynamically changing light conditions (Crawford, 1947; Hecht, Haig & Chase, 1937). These adaptation dynamics are unknown under dim (mesopic) light levels when the rod (R) and long (L), medium (M) and short (S) wavelength cone photoreceptor classes contribute to vision via interactions in shared non-opponent Magnocellular (MC), chromatically opponent Parvocellular (PC) and Koniocellular (KC) visual pathways (Dacey, 2000). This study investigated the time-course of adaptation and post-receptoral pathways mediating receptor specific rod and cone interactions under mesopic illumination. A four-primary photostimulator (Pokorny, Smithson & Quinlan, 2004) was used to independently control the activity of the four photoreceptor classes and their post-receptoral visual athways in human observers. In the first experiment, the contrast sensitivity and time-course of visual adaptation under mesopic illumination were measured for receptoral (L, S, R) and post-receptoral (LMS, LMSR, L-M) stimuli. An incremental (Rapid-ON) sawtooth conditioning pulse biased detection to ON-cells within the visual pathways and sensitivity was assayed relative to pulse onset using a briefly presented incremental probe that did not alter adaptation. Cone.Cone interactions with luminance stimuli (L cone, LMS, LMSR) reduced sensitivity by 15% and the time course of recovery was 25± 5ms-1 (μ ± SEM). PC mediated (+L-M) chromatic stimuli sensitivity loss was less (8%) than for luminance and recovery was slower (μ = 2.95 ± 0.05 ms-1), with KC mediated (S cone) chromatic stimuli showing a high sensitivity loss (38%) and the slowest recovery time (1.6 ± 0.2 ms-1). Rod-Rod interactions increased sensitivity by 20% and the time course of recovery was 0.7 ± 0.2 ms-1 (μ ± SD). Compared to these interaction types, Rod-Cone interactions reduced sensitivity to a lesser degree (5%) and showed the fastest recovery (μ = 43 ± 7 ms-1). In the second experiment, rod contribution to the magnocellular, parvocellular and koniocellular post-receptoral pathways under mesopic illumination was determined as a function of incremental stimulus duration and waveform (rectangular; sawtooth) using a rod colour match procedure (Cao, Pokorny & Smith, 2005; Cao, Pokorny, Smith & Zele, 2008a). For a 30% rod increment, a cone match required a decrease in [L/(L+M)] and an increase in [L+M] and [S/(L+M)], giving a greenish-blue and brighter appearance for probe durations of 75 ms or longer. Probe durations less than 75 ms showed an increase in [L+M] and no change in chromaticity [L/(L+M) or S/(L+M)], uggesting mediation by the MC pathway only for short duration rod stimuli. s We advance previous studies by determining the time-course and nature of photoreceptor specific retinal interactions in the three post-receptoral pathways under mesopic illumination. In the first experiment, the time-course of adaptation for ON cell processing was determined, revealing opponent cell facilitation in chromatic PC and KC pathways. The Rod-Rod and Rod-Cone data identify previously unknown interaction types that act to maintain contrast sensitivity during dynamically changing light conditions and improve the speed of light adaptation under mesopic light levels. The second experiment determined the degree of rod contribution to the inferred post-eceptoral pathways as a function of the temporal properties of the rod signal. r The understanding of the mechanisms underlying interactions between photoreceptors under mesopic illumination has implications for the study of retinal disease. Visual function has been shown to be reduced in persons with age-related maculopathy (ARM) risk genotypes prior to clinical signs of the disease (Feigl, Cao, Morris & Zele, 2011) and disturbances in rod-mediated adaptation have been shown in early phases of ARM (Dimitrov, Guymer, Zele, Anderson & Vingrys, 2008; Feigl, Brown, Lovie-Kitchin & Swann, 2005). Also, the understanding of retinal networks controlling vision enables the development of international lighting standards to optimise visual performance nder dim light levels (e.g. work-place environments, transportation).

Slow horizon

“Slow Horizon” is comprised of six lenticular panels hung in an even, horizontal sequence. As the viewer moves in front of the work, each panel alternates subtly between two vertical colour gradients. From left to right, the panels move through yellow, orange, magenta and violet to ‘midnight blue’. Together, the coloured panels comprise an abstract horizon line that references the changing nature of light at sunset. The scale, movement and chromatic qualities of the panels also allude to the formal characteristics of the screen technologies that pervade contemporary visual culture. “Slow Horizon” contributes to studies in the field of contemporary art. It is particularly concerned with the relationships between abstraction, colour, signification and perception. Since early Modernity, debates concerning representation and the formal qualities of the picture plane have been fundamental to art practice and theory. These debates have often dovetailed with questions of art’s capacity to generate shifts in thought and perception. Practitioners such as Ellsworth Kelly, James Turrell and Ed Ruscha have variously used block and blended colour to engage in these formal, symbolic and perceptual potentials of colour. Using a practice-led research methodology, “Slow Horizon” furthers this creative inquiry. By conflating the reductive visual logics of abstraction and minimalism with the iconic, romantic evocations of sunset imagery, it questions not only the contemporary relationship between abstraction and image-making, but also art’s ability to create moments of stillness and contemplation in a context significantly shaped by screen technologies. “Slow Horizon” has been exhibited internationally as part of “Supermassive” at LA Louver Gallery, Venice, California in 2013. The exhibition was reviewed in The Los Angeles Times.