The relative contributions of colour and luminance signals towards the visuomotor localisation of targets in human peripheral vision

We sought to determine the extent to which colour (and luminance) signals contribute towards the visuomotor localization of targets. To do so we exploited the movement-related illusory displacement a small stationary window undergoes when it has a continuously moving carrier grating behind it. We used drifting (1.0-4.2 Hz) red/green-modulated isoluminant gratings or yellow/black luminance-modulated gratings as carriers, each curtailed in space by a stationary, two-dimensional window. After each trial, the perceived location of the window was recorded with reference to an on-screen ruler (perceptual task) or the on-screen touch of a ballistic pointing movement made without visual feedback (visuomotor task). Our results showed that the perceptual displacement measures were similar for each stimulus type and weakly dependent on stimulus drift rate. However, while the visuomotor displacement measures were similar for each stimulus type at low drift rates (<4 Hz), they were significantly larger for luminance than colour stimuli at high drift rates (>4 Hz). We show that the latter cannot be attributed to differences in perceived speed between stimulus types. We assume, therefore, that our visuomotor localization judgements were more susceptible to the (carrier) motion of luminance patterns than colour patterns. We suggest that, far from being detrimental, this susceptibility may indicate the operation of mechanisms designed to counter the temporal asynchrony between perceptual experiences and the physical changes in the environment that give rise to them. We propose that perceptual localisation is equally supported by both colour and luminance signals but that visuomotor localisation is predominantly supported by luminance signals. We discuss the neural pathways that may be involved with visuomotor localization. © 2007 Springer-Verlag.

Development of visual evoked responses to tritan,red-green and luminance stimuli in human infants

The principal aim of this work was to investigate the development of the S-cone colour-opponent pathway in human infants aged 4 weeks to 6 months. This was achieved by recording transient visual evoked responses to pattern-onset stimuli along a tritanopic confusion axis (tritan stimuli) at and around the adult isoluminant match. For comparison, visual evoked responses to red-green and luminance-modulated stimuli were recorded from the same infants at the same ages. Evoked responses were also recorded from colour-normal adults for comparison with those of the infants. The transient VEP allowed observation of response morphology as luminance differences were introduced to the chromatic stimuli. In this way, an estimate of isoluminance was possible in infants. Estimated isoluminant points for a group of six infants aged 6 to 10 weeks closely approximated the adult isoluminant match. This finding has implications for the use of photometric isoluminance in infant work, and suggests that photopic spectral sensitivity is similar in infants and adults. Abnormalities of the visual evoked responses to tritan, red-green and luminance-modulated stimuli in an infant with cystic fibrosis are reported. The results suggest abnormal function of the retino-striate visual pathway in this infant, and it is argued that these may be secondary to his illness, although data from more infants with cystic fibrosis are needed to clarify this further. A group of nine healthy infants demonstrated evoked responses to tritan stimuli by 4 to 10 weeks and to red-green stimuli by 6 to 11 weeks post-term age. Responses to luminance-modulated stimuli were present in all nine infants at the earliest age tested, namely 4 weeks post-term. The slightly earlier age of onset of evoked responses to tritan stimuli than for red-green may be explained by the relatively lower cone contrast afforded by red-green stimuli. Latency of the evoked response to both types of chromatic stimuli and to luminance-modulated stimuli decreased with age at a similar rate, suggesting that the visual pathways transmitting luminance and chromatic information mature at similar rates in young infants.

Area summation of first- and second-order modulations of luminance

To extend our understanding of the early visual hierarchy, we investigated the long-range integration of first- and second-order signals in spatial vision. In our first experiment we performed a conventional area summation experiment where we varied the diameter of (a) luminance-modulated (LM) noise and (b) contrastmodulated (CM) noise. Results from the LM condition replicated previous findings with sine-wave gratings in the absence of noise, consistent with long-range integration of signal contrast over space. For CM, the summation function was much shallower than for LM suggesting, at first glance, that the signal integration process was spatially less extensive than for LM. However, an alternative possibility was that the high spatial frequency noise carrier for the CM signal was attenuated by peripheral retina (or cortex), thereby impeding our ability to observe area summation of CM in the conventional way. To test this, we developed the ''Swiss cheese'' stimulus of Meese and Summers (2007) in which signal area can be varied without changing the stimulus diameter, providing some protection against inhomogeneity of the retinal field. Using this technique and a two-component subthreshold summation paradigm we found that (a) CM is spatially integrated over at least five stimulus cycles (possibly more), (b) spatial integration follows square-law signal transduction for both LM and CM and (c) the summing device integrates over spatially-interdigitated LM and CM signals when they are co-oriented, but not when crossoriented. The spatial pooling mechanism that we have identified would be a good candidate component for amodule involved in representing visual textures, including their spatial extent.

Binocular functional architecture for detection of contrast-modulated gratings

Combination of signals from the two eyes is the gateway to stereo vision. To gain insight into binocular signal processing, we studied binocular summation for luminance-modulated gratings (L or LM) and contrast-modulated gratings (CM). We measured 2AFC detection thresholds for a signal grating (0.75 c/deg, 216msec) shown to one eye, both eyes, or both eyes out-of-phase. For LM and CM, the carrier noise was in both eyes, even when the signal was monocular. Mean binocular thresholds for luminance gratings (L) were 5.4dB better than monocular thresholds - close to perfect linear summation (6dB). For LM and CM the binocular advantage was again 5-6dB, even when the carrier noise was uncorrelated, anti-correlated, or at orthogonal orientations in the two eyes. Binocular combination for CM probably arises from summation of envelope responses, and not from summation of these conflicting carrier patterns. Antiphase signals produced no binocular advantage, but thresholds were about 1-3dB higher than monocular ones. This is not consistent with simple linear summation, which should give complete cancellation and unmeasurably high thresholds. We propose a three-channel model in which noisy monocular responses to the envelope are binocularly combined in a contrast-weighted sum, but also remain separately available to perception via a max operator. Vision selects the largest of the three responses. With in-phase gratings the binocular channel dominates, but antiphase gratings cancel in the binocular channel and the monocular channels mediate detection. The small antiphase disadvantage might be explained by a subtle influence of background responses on binocular and monocular detection.

Fourth-root summation of contrast over area:no end in sight when spatially inhomogeneous sensitivity is compensated by a witch's hat

Measurements of area summation for luminance-modulated stimuli are typically confounded by variations in sensitivity across the retina. Recently we conducted a detailed analysis of sensitivity across the visual field (Baldwin et al, 2012) and found it to be well-described by a bilinear “witch’s hat” function: sensitivity declines rapidly over the first 8 cycles or so, more gently thereafter. Here we multiplied luminance-modulated stimuli (4 c/deg gratings and “Swiss cheeses”) by the inverse of the witch’s hat function to compensate for the inhomogeneity. This revealed summation functions that were straight lines (on double log axes) with a slope of -1/4 extending to ≥33 cycles, demonstrating fourth-root summation of contrast over a wider area than has previously been reported for the central retina. Fourth-root summation is typically attributed to probability summation, but recent studies have rejected that interpretation in favour of a noisy energy model that performs local square-law transduction of the signal, adds noise at each location of the target and then sums over signal area. Modelling shows our results to be consistent with a wide field application of such a contrast integrator. We reject a probability summation model, a quadratic model and a matched template model of our results under the assumptions of signal detection theory. We also reject the high threshold theory of contrast detection under the assumption of probability summation over area.

Colour and luminance interactions in the visual perception of motion

We sought to determine the extent to which red–green, colour–opponent mechanisms in the human visual system play a role in the perception of drifting luminance–modulated targets. Contrast sensitivity for the directional discrimination of drifting luminance–modulated (yellow–black) test sinusoids was measured following adaptation to isoluminant red–green sinusoids drifting in either the same or opposite direction. When the test and adapt stimuli drifted in the same direction, large sensitivity losses were evident at all test temporal frequencies employed (1–16 Hz). The magnitude of the loss was independent of temporal frequency. When adapt and test stimuli drifted in opposing directions, large sensitivity losses were evident at lower temporal frequencies (1–4 Hz) and declined with increasing temporal frequency. Control studies showed that this temporal–frequency–dependent effect could not reflect the activity of achromatic units. Our results provide evidence that chromatic mechanisms contribute to the perception of luminance–modulated motion targets drifting at speeds of up to at least 32°s-1. We argue that such mechanisms most probably lie within a parvocellular–dominated cortical visual pathway, sensitive to both chromatic and luminance modulation, but only weakly selective for the direction of stimulus motion.

Nonlinearities in the binocular combination of luminance and contrast

We studied the rules by which visual responses to luminous targets are combined across the two eyes. Previous work has found very different forms of binocular combination for targets defined by increments and by decrements of luminance, with decrement data implying a severe nonlinearity before binocular combination. We ask whether this difference is due to the luminance of the target, the luminance of the background, or the sign of the luminance excursion. We estimated the pre-binocular nonlinearity (power exponent) by fitting a computational model to ocular equibrightness matches. The severity of the nonlinearity had a monotonic dependence on the signed difference between target and background luminance. For dual targets, in which there was both a luminance increment and a luminance decrement (e.g. contrast), perception was governed largely by the decrement. The asymmetry in the nonlinearities derived from the subjective matching data made a clear prediction for visual performance: there should be more binocular summation for detecting luminance increments than for detecting luminance decrements. This prediction was confirmed by the results of a subsequent experiment. We discuss the relation between these results and luminance nonlinearities such as a logarithmic transform, as well as the involvement of contemporary model architectures of binocular vision.

Depth propagation and surface construction in 3-D vision

In stereo vision, regions with ambiguous or unspecified disparity can acquire perceived depth from unambiguous regions. This has been called stereo capture, depth interpolation or surface completion. We studied some striking induced depth effects suggesting that depth interpolation and surface completion are distinct stages of visual processing. An inducing texture (2-D Gaussian noise) had sinusoidal modulation of disparity, creating a smooth horizontal corrugation. The central region of this surface was replaced by various test patterns whose perceived corrugation was measured. When the test image was horizontal 1-D noise, shown to one eye or to both eyes without disparity, it appeared corrugated in much the same way as the disparity-modulated (DM) flanking regions. But when the test image was 2-D noise, or vertical 1-D noise, little or no depth was induced. This suggests that horizontal orientation was a key factor. For a horizontal sine-wave luminance grating, strong depth was induced, but for a square-wave grating, depth was induced only when its edges were aligned with the peaks and troughs of the DM flanking surface. These and related results suggest that disparity (or local depth) propagates along horizontal 1-D features, and then a 3-D surface is constructed from the depth samples acquired. The shape of the constructed surface can be different from the inducer, and so surface construction appears to operate on the results of a more local depth propagation process.

Discriminating depth in corrugated stereo surfaces: facilitation by a pedestal is explained by removal of uncertainty

With luminance gratings, psychophysical thresholds for detecting a small increase in the contrast of a weak ‘pedestal’ grating are 2–3 times lower than for detection of a grating when the pedestal is absent. This is the ‘dipper effect’ – a reliable improvement whose interpretation remains controversial. Analogies between luminance and depth (disparity) processing have attracted interest in the existence of a ‘disparity dipper’. Are thresholds for disparity modulation (corrugated surfaces), facilitated by the presence of a weak disparity-modulated pedestal? We used a 14-bit greyscale to render small disparities accurately, and measured 2AFC discrimination thresholds for disparity modulation (0.3 or 0.6 c/deg) of a random texture at various pedestal levels. In the first experiment, a clear dipper was found. Thresholds were about 2× lower with weak pedestals than without. But here the phase of modulation (0 or 180 deg) was varied from trial to trial. In a noisy signal-detection framework, this creates uncertainty that is reduced by the pedestal, which thus improves performance. When the uncertainty was eliminated by keeping phase constant within sessions, the dipper effect was weak or absent. Monte Carlo simulations showed that the influence of uncertainty could account well for the results of both experiments. A corollary is that the visual depth response to small disparities is probably linear, with no threshold-like nonlinearity.

Discriminating depth in stereo surfaces: dipper effect found, and lost

Measurement of detection and discrimination thresholds yields information about visual signal processing. For luminance contrast, we are 2 - 3 times more sensitive to a small increase in the contrast of a weak 'pedestal' grating, than when the pedestal is absent. This is the 'dipper effect' - a reliable improvement whose interpretation remains controversial. Analogies between luminance and depth (disparity) processing have attracted interest in the existence of a 'disparity dipper' - are thresholds for disparity, or disparity modulation (corrugated surfaces), facilitated by the presence of a weak pedestal? Lunn and Morgan (1997 Journal of the Optical Society of America A 14 360 - 371) found no dipper for disparity-modulated gratings, but technical limitations (8-bit greyscale) might have prevented the necessary measurement of very small disparity thresholds. We used a true 14-bit greyscale to render small disparities accurately, and measured 2AFC discrimination thresholds for disparity modulation (0.6 cycle deg-1) of a random texture at various pedestal levels. Which interval contained greater modulation of depth? In the first experiment, a clear dipper was found. Thresholds were about 2X1 lower with weak pedestals than without. But here the phase of modulation (0° or 180°) was randomised from trial to trial. In a noisy signal-detection framework, this creates uncertainty that is reduced by the pedestal, thus improving performance. When the uncertainty was eliminated by keeping phase constant within sessions, the dipper effect disappeared, confirming Lunn and Morgan's result. The absence of a dipper, coupled with shallow psychometric slopes, suggests that the visual response to small disparities is essentially linear, with no threshold-like nonlinearity.

Added luminance ramp alters perceived edge blur and contrast:A critical test for derivative-based models of edge coding

In many models of edge analysis in biological vision, the initial stage is a linear 2nd derivative operation. Such models predict that adding a linear luminance ramp to an edge will have no effect on the edge's appearance, since the ramp has no effect on the 2nd derivative. Our experiments did not support this prediction: adding a negative-going ramp to a positive-going edge (or vice-versa) greatly reduced the perceived blur and contrast of the edge. The effects on a fairly sharp edge were accurately predicted by a nonlinear multi-scale model of edge processing [Georgeson, M. A., May, K. A., Freeman, T. C. A., & Hesse, G. S. (in press). From filters to features: Scale-space analysis of edge and blur coding in human vision. Journal of Vision], in which a half-wave rectifier comes after the 1st derivative filter. But we also found that the ramp affected perceived blur more profoundly when the edge blur was large, and this greater effect was not predicted by the existing model. The model's fit to these data was much improved when the simple half-wave rectifier was replaced by a threshold-like transducer [May, K. A. & Georgeson, M. A. (2007). Blurred edges look faint, and faint edges look sharp: The effect of a gradient threshold in a multi-scale edge coding model. Vision Research, 47, 1705-1720.]. This modified model correctly predicted that the interaction between ramp gradient and edge scale would be much larger for blur perception than for contrast perception. In our model, the ramp narrows an internal representation of the gradient profile, leading to a reduction in perceived blur. This in turn reduces perceived contrast because estimated blur plays a role in the model's estimation of contrast. Interestingly, the model predicts that analogous effects should occur when the width of the window containing the edge is made narrower. This has already been confirmed for blur perception; here, we further support the model by showing a similar effect for contrast perception. © 2007 Elsevier Ltd. All rights reserved.

The role of local luminance amplitude in the interpretation of shape-from-shading in textured surfaces

Luminance changes within a scene are ambiguous; they can indicate reflectance changes, shadows, or shading due to surface undulations. How does vision distinguish between these possibilities? When a surface painted with an albedo texture is shaded, the change in local mean luminance (LM) is accompanied by a similar modulation of the local luminance amplitude (AM) of the texture. This relationship does not necessarily hold for reflectance changes or for shading of a relief texture. Here we concentrate on the role of AM in shape-from-shading. Observers were presented with a noise texture onto which sinusoidal LM and AM signals were superimposed, and were asked to indicate which of two marked locations was closer to them. Shape-from-shading was enhanced when LM and AM co-varied (in-phase), and was disrupted when they were out-of-phase. The perceptual differences between cue types (in-phase vs out-of-phase) were enhanced when the two cues were present at different orientations within a single image. Similar results were found with a haptic matching task. We conclude that vision can use AM to disambiguate luminance changes. LM and AM have a positive relationship for rendered, undulating, albedo textures, and we assess the degree to which this relationship holds in natural images. [Supported by EPSRC grants to AJS and MAG].

Local luminance amplitude modulates the interpretation of shape-from-shading in textured surfaces

The pattern of illumination on an undulating surface can be used to infer its 3-D form (shape-from-shading). But the recovery of shape would be invalid if the luminance changes actually arose from changes in reflectance. So how does vision distinguish variation in illumination from variation in reflectance to avoid illusory depth? When a corrugated surface is painted with an albedo texture, the variation in local mean luminance (LM) due to shading is accompanied by a similar modulation in local luminance amplitude (AM). This is not so for reflectance variation, nor for roughly textured surfaces. We used depth mapping and paired comparison methods to show that modulations of local luminance amplitude play a role in the interpretation of shape-from-shading. The shape-from-shading percept was enhanced when LM and AM co-varied (in-phase) and was disrupted when they were out of phase or (to a lesser degree) when AM was absent. The perceptual differences between cue types (in-phase vs out-of-phase) were enhanced when the two cues were present at different orientations within a single image. Our results suggest that when LM and AM co-vary (in-phase) this indicates that the source of variation is illumination (caused by undulations of the surface), rather than surface reflectance. Hence, the congruence of LM and AM is a cue that supports a shape-from-shading interpretation. © 2006 Elsevier Ltd. All rights reserved.

The role of texture in shape from shading: are humans biased towards seeing relief textures?

When a textured surface is modulated in depth and illuminated, the level of illumination varies across the surface, producing coarse-scale luminance modulations (LM) and amplitude modulation (AM) of the fine-scale texture. If the surface has an albedo texture (reflectance variation) then the LM and AM components are always in-phase, but if the surface has a relief texture the phase relation between LM and AM varies with the direction and nature of the illuminant. We showed observers sinusoidal luminance and amplitude modulations of a binary noise texture, in various phase relationships, in a paired-comparisons design. In the first experiment, the combinations under test were presented in different temporal intervals. Observers indicated which interval contained the more depthy stimulus. LM and AM in-phase were seen as more depthy than LM alone which was in turn more depthy than LM and AM in anti-phase, but the differences were weak. In the second experiment the combinations under test were presented in a single interval on opposite obliques of a plaid pattern. Observers were asked to indicate the more depthy oblique. Observers produced the same depth rankings as before, but now the effects were more robust and significant. Intermediate LM/AM phase relationships were also tested: phase differences less than 90 deg were seen as more depthy than LM-only, while those greater than 90 deg were seen as less depthy. We conjecture that the visual system construes phase offsets between LM and AM as indicating relief texture and thus perceives these combinations as depthy even when their phase relationship is other than zero. However, when different LM/AM pairs are combined in a plaid, the signals on the obliques are unlikely to indicate corrugations of the same texture, and in this case the out-of-phase pairing is seen as flat. [Supported by the Engineering and Physical Sciences Research Council (EPSRC)].

The role of texture amplitude in shape from shading

When a textured surface is modulated in depth and illuminated, parts of the surface receive different levels of illumination; the resulting variations in luminance can be used to infer the shape of the depth modulations-shape from shading. The changes in illumination also produce changes in the amplitude of the texture, although local contrast remains constant. We investigated the role of texture amplitude in supporting shape from shading. If a luminance plaid is added to a binary noise texture (LM), shape from shading produces perception of corrugations in two directions. If the amplitude of the noise is also modulated (AM) such that it is in-phase with one of the luminance sinusoids and out-of-phase with the other, the resulting surface is seen as corrugated in only one directionöthat supported by the in-phase pairing. We confirmed this subjective report experimentally, using a depth-mapping technique. Further, we asked naïve observers to indicate the direction of corrugations in plaids made up of various combinations of LM and AM. LM+AM was seen as having most depth, then LM-only, then LM-AM, and then AM-only. Our results suggest that while LM is required to see depth from shading, its phase relative to any AM component is also important.