Visual motion perception.

The primate visual motion system performs numerous functions essential for survival in a dynamic visual world. Prominent among these functions is the ability to recover and represent the trajectories of objects in a form that facilitates behavioral responses to those movements. The first step toward this goal, which consists of detecting the displacement of retinal image features, has been studied for many years in both psychophysical and neurobiological experiments. Evidence indicates that achievement of this step is computationally straightforward and occurs at the earliest cortical stage. The second step involves the selective integration of retinal motion signals according to the object of origin. Realization of this step is computationally demanding, as the solution is formally underconstrained. It must rely--by definition--upon utilization of retinal cues that are indicative of the spatial relationships within and between objects in the visual scene. Psychophysical experiments have documented this dependence and suggested mechanisms by which it may be achieved. Neurophysiological experiments have provided evidence for a neural substrate that may underlie this selective motion signal integration. Together they paint a coherent portrait of the means by which retinal image motion gives rise to our perceptual experience of moving objects.

Neural systems underlying learning and representation of global motion

We demonstrate performance-related changes in cortical and cerebellar activity. The largest learning-dependent changes were observed in the anterior lateral cerebellum, where the extent and intensity of activation correlated inversely with psychophysical performance. After learning had occurred (a few minutes), the cerebellar activation almost disappeared; however, it was restored when the subjects were presented with a novel, untrained direction of motion for which psychophysical performance also reverted to chance level. Similar reductions in the extent and intensity of brain activations in relation to learning occurred in the superior colliculus, anterior cingulate, and parts of the extrastriate cortex. The motion direction-sensitive middle temporal visual complex was a notable exception, where there was an expansion of the cortical territory activated by the trained stimulus. Together, these results indicate that the learning and representation of visual motion discrimination are mediated by different, but probably interacting, neuronal subsystems.

Computational models of cortical visual processing.

The visual responses of neurons in the cerebral cortex were first adequately characterized in the 1960s by D. H. Hubel and T. N. Wiesel [(1962) J. Physiol. (London) 160, 106-154; (1968) J. Physiol. (London) 195, 215-243] using qualitative analyses based on simple geometric visual targets. Over the past 30 years, it has become common to consider the properties of these neurons by attempting to make formal descriptions of these transformations they execute on the visual image. Most such models have their roots in linear-systems approaches pioneered in the retina by C. Enroth-Cugell and J. R. Robson [(1966) J. Physiol. (London) 187, 517-552], but it is clear that purely linear models of cortical neurons are inadequate. We present two related models: one designed to account for the responses of simple cells in primary visual cortex (V1) and one designed to account for the responses of pattern direction selective cells in MT (or V5), an extrastriate visual area thought to be involved in the analysis of visual motion. These models share a common structure that operates in the same way on different kinds of input, and instantiate the widely held view that computational strategies are similar throughout the cerebral cortex. Implementations of these models for Macintosh microcomputers are available and can be used to explore the models' properties.

Inputs to directionally selective simple cells in macaque striate cortex

It is clear that the initial analysis of visual motion takes place in the striate cortex, where directionally selective cells are found that respond to local motion in one direction but not in the opposite direction. Widely accepted motion models postulate as inputs to directional units two or more cells whose spatio-temporal receptive fields (RFs) are approximately 90° out of phase (quadrature) in space and in time. Simple cells in macaque striate cortex differ in their spatial phases, but evidence is lacking for the varying time delays required for two inputs to be in temporal quadrature. We examined the space-time RF structure of cells in macaque striate cortex and found two subpopulations of (nondirectional) simple cells, some that show strongly biphasic temporal responses, and others that are weakly biphasic if at all. The temporal impulse responses of these two classes of cells are very close to 90° apart, with the strongly biphasic cells having a shorter latency than the weakly biphasic cells. A principal component analysis of the spatio-temporal RFs of directionally selective simple cells shows that their RFs could be produced by a linear combination of two components; these two components correspond closely in their respective latencies and biphasic characters to those of strongly biphasic and weakly biphasic nondirectional simple cells, respectively. This finding suggests that the motion system might acquire the requisite temporal quadrature by combining inputs from these two classes of nondirectional cells (or from their respective lateral geniculate inputs, which appear to be from magno and parvo lateral geniculate cells, respectively).

Shift in speed selectivity of visual cortical neurons: A neural basis of perceived motion contrast

The perceived speed of motion in one part of the visual field is influenced by the speed of motion in its surrounding fields. Little is known about the cellular mechanisms causing this phenomenon. Recordings from mammalian visual cortex revealed that speed preference of the cortical cells could be changed by displaying a contrast speed in the field surrounding the cell’s classical receptive field. The neuron’s selectivity shifted to prefer faster speed if the contextual surround motion was set at a relatively lower speed, and vice versa. These specific center–surround interactions may underlie the perceptual enhancement of speed contrast between adjacent fields.

Perceptual learning in motion discrimination that generalizes across motion directions

When human subjects discriminate motion directions of two visual stimuli, their discrimination improves with practice. This improved performance has been found to be specific to the practiced directions and does not transfer to new motion directions. Indeed, such stimulus-specific learning has become a trademark finding in almost all perceptual learning studies and has been used to infer the loci of learning in the brain. For example, learning in motion discrimination has been inferred to occur in the visual area MT (medial temporal cortex) of primates, where neurons are selectively tuned to motion directions. However, such motion discrimination task is extremely difficult, as is typical of most perceptual learning tasks. When the difficulty is moderately reduced, learning transfers to new motion directions. This result challenges the idea of using simple visual stimuli to infer the locus of learning in low-level visual processes and suggests that higher-level processing is essential even in “simple” perceptual learning tasks.

A wholly empirical explanation of perceived motion

Because the retinal activity generated by a moving object cannot specify which of an infinite number of possible physical displacements underlies the stimulus, its real-world cause is necessarily uncertain. How, then, do observers respond successfully to sequences of images whose provenance is ambiguous? Here we explore the hypothesis that the visual system solves this problem by a probabilistic strategy in which perceived motion is generated entirely according to the relative frequency of occurrence of the physical sources of the stimulus. The merits of this concept were tested by comparing the directions and speeds of moving lines reported by subjects to the values determined by the probability distribution of all the possible physical displacements underlying the stimulus. The velocities reported by observers in a variety of stimulus contexts can be accounted for in this way.

Event-related brain potentials in the study of visual selective attention

Event-related brain potentials (ERPs) provide high-resolution measures of the time course of neuronal activity patterns associated with perceptual and cognitive processes. New techniques for ERP source analysis and comparisons with data from blood-flow neuroimaging studies enable improved localization of cortical activity during visual selective attention. ERP modulations during spatial attention point toward a mechanism of gain control over information flow in extrastriate visual cortical pathways, starting about 80 ms after stimulus onset. Paying attention to nonspatial features such as color, motion, or shape is manifested by qualitatively different ERP patterns in multiple cortical areas that begin with latencies of 100–150 ms. The processing of nonspatial features seems to be contingent upon the prior selection of location, consistent with early selection theories of attention and with the hypothesis that spatial attention is “special.”

The representation of the ipsilateral visual field in human cerebral cortex

Previous studies of cortical retinotopy focused on influences from the contralateral visual field, because ascending inputs to cortex are known to be crossed. Here, functional magnetic resonance imaging was used to demonstrate and analyze an ipsilateral representation in human visual cortex. Moving stimuli, in a range of ipsilateral visual field locations, revealed activity: (i) along the vertical meridian in retinotopic (presumably lower-tier) areas; and (ii) in two large branches anterior to that, in presumptive higher-tier areas. One branch shares the anterior vertical meridian representation in human V3A, extending superiorly toward parietal cortex. The second branch runs antero-posteriorly along lateral visual cortex, overlying motion-selective area MT. Ipsilateral stimuli sparing the region around the vertical meridian representation also produced signal reductions (perhaps reflecting neural inhibition) in areas showing contralaterally driven retinotopy. Systematic sampling across a range of ipsilateral visual field extents revealed significant increases in ipsilateral activation in V3A and V4v, compared with immediately posterior areas V3 and VP. Finally, comparisons between ipsilateral stimuli of different types but equal retinotopic extent showed clear stimulus specificity, consistent with earlier suggestions of a functional segregation of motion vs. form processing in parietal vs. temporal cortex, respectively.

Psychophysical evidence for fast region-based segmentation processes in motion and color.

Theories of image segmentation suggest that the human visual system may use two distinct processes to segregate figure from background: a local process that uses local feature contrasts to mark borders of coherent regions and a global process that groups similar features over a larger spatial scale. We performed psychophysical experiments to determine whether and to what extent the global similarity process contributes to image segmentation by motion and color. Our results show that for color, as well as for motion, segmentation occurs first by an integrative process on a coarse spatial scale, demonstrating that for both modalities the global process is faster than one based on local feature contrasts. Segmentation by motion builds up over time, whereas segmentation by color does not, indicating a fundamental difference between the modalities. Our data suggest that segmentation by motion proceeds first via a cooperative linking over space of local motion signals, generating almost immediate perceptual coherence even of physically incoherent signals. This global segmentation process occurs faster than the detection of absolute motion, providing further evidence for the existence of two motion processes with distinct dynamic properties.

Motion perception: seeing and deciding.

The primate visual system offers unprecedented opportunities for investigating the neural basis of cognition. Even the simplest visual discrimination task requires processing of sensory signals, formation of a decision, and orchestration of a motor response. With our extensive knowledge of the primate visual and oculomotor systems as a base, it is now possible to investigate the neural basis of simple visual decisions that link sensation to action. Here we describe an initial study of neural responses in the lateral intraparietal area (LIP) of the cerebral cortex while alert monkeys discriminated the direction of motion in a visual display. A subset of LIP neurons carried high-level signals that may comprise a neural correlate of the decision process in our task. These signals are neither sensory nor motor in the strictest sense; rather they appear to reflect integration of sensory signals toward a decision appropriate for guiding movement. If this ultimately proves to be the case, several fascinating issues in cognitive neuroscience will be brought under rigorous physiological scrutiny.

Spatial heterogeneity of inhibitory surrounds in the middle temporal visual area.

A recurrent theme in the organization of vertebrate visual cortex is that of receptive fields with an associated "silent" opponency component. In the middle temporal area (area MT), a cortical visual area involved in the analysis of retinal motion in primates, this opponency appears in the form of a region outside the classical receptive field (CRF) that in itself gives no response but suppresses responses to motion evoked within the CRF. This antagonistic motion surround has been described as very large and symmetrically arrayed around the CRF. On the basis of this view, the primary function of the surround has long been thought to consist of simple figure-ground segregation based on movement. We have made use of small stimulus patches to map the form and extent of the surround and find evidence that the surround inhibition of many MT cells is in fact confined to restricted regions on one side or on opposite sides of the CRF. Such regions endow MT cells with the ability to make local-to-local motion comparisons, capable of extracting more complex features from the visual environment, and as such, may be better viewed as intrinsic parts of the receptive field, rather than as separate entities responsible for local-to-global comparisons.

Parameters affecting conscious versus unconscious visual discrimination with damage to the visual cortex (V1).

When the visual (striate) cortex (V1) is damaged in human subjects, cortical blindness results in the contralateral visual half field. Nevertheless, under some experimental conditions, subjects demonstrate a capacity to make visual discriminations in the blind hemifield (blindsight), even though they have no phenomenal experience of seeing. This capacity must, therefore, be mediated by parallel projections to other brain areas. It is also the case that some subjects have conscious residual vision in response to fast moving stimuli or sudden changes in light flux level presented to the blind hemifield, characterized by a contentless kind of awareness, a feeling of something happening, albeit not normal seeing. The relationship between these two modes of discrimination has never been studied systematically. We examine, in the same experiment, both the unconscious discrimination and the conscious visual awareness of moving stimuli in a subject with unilateral damage to V1. The results demonstrate an excellent capacity to discriminate motion direction and orientation in the absence of acknowledged perceptual awareness. Discrimination of the stimulus parameters for acknowledged awareness apparently follows a different functional relationship with respect to stimulus speed, displacement, and stimulus contrast. As performance in the two modes can be quantitatively matched, the findings suggest that it should be possible to image brain activity and to identify the active areas involved in the same subject performing the same discrimination task, both with and without conscious awareness, and hence to determine whether any structures contribute uniquely to conscious perception.

Spatial organization of retinal information about the direction of image motion.

The visual stimuli that elicit neural activity differ for different retinal ganglion cells and these cells have been categorized by the visual information that they transmit. If specific visual information is conveyed exclusively or primarily by a particular set of ganglion cells, one might expect the cells to be organized spatially so that their sampling of information from the visual field is complete but not redundant. In other words, the laterally spreading dendrites of the ganglion cells should completely cover the retinal plane without gaps or significant overlap. The first evidence for this sort of arrangement, which has been called a tiling or tessellation, was for the two types of "alpha" ganglion cells in cat retina. Other reports of tiling by ganglion cells have been made subsequently. We have found evidence of a particularly rigorous tiling for the four types of ganglion cells in rabbit retina that convey information about the direction of retinal image motion (the ON-OFF direction-selective cells). Although individual cells in the four groups are morphologically indistinguishable, they are organized as four overlaid tilings, each tiling consisting of like-type cells that respond preferentially to a particular direction of retinal image motion. These observations lend support to the hypothesis that tiling is a general feature of the organization of information outflow from the retina and clearly implicate mechanisms for recognition of like-type cells and establishment of mutually acceptable territories during retinal development.