Image recognition: Visual grouping, recognition, and learning

Vision extracts useful information from images. Reconstructing the three-dimensional structure of our environment and recognizing the objects that populate it are among the most important functions of our visual system. Computer vision researchers study the computational principles of vision and aim at designing algorithms that reproduce these functions. Vision is difficult: the same scene may give rise to very different images depending on illumination and viewpoint. Typically, an astronomical number of hypotheses exist that in principle have to be analyzed to infer a correct scene description. Moreover, image information might be extracted at different levels of spatial and logical resolution dependent on the image processing task. Knowledge of the world allows the visual system to limit the amount of ambiguity and to greatly simplify visual computations. We discuss how simple properties of the world are captured by the Gestalt rules of grouping, how the visual system may learn and organize models of objects for recognition, and how one may control the complexity of the description that the visual system computes.

Neural systems underlying learning and representation of global motion

We demonstrate performance-related changes in cortical and cerebellar activity. The largest learning-dependent changes were observed in the anterior lateral cerebellum, where the extent and intensity of activation correlated inversely with psychophysical performance. After learning had occurred (a few minutes), the cerebellar activation almost disappeared; however, it was restored when the subjects were presented with a novel, untrained direction of motion for which psychophysical performance also reverted to chance level. Similar reductions in the extent and intensity of brain activations in relation to learning occurred in the superior colliculus, anterior cingulate, and parts of the extrastriate cortex. The motion direction-sensitive middle temporal visual complex was a notable exception, where there was an expansion of the cortical territory activated by the trained stimulus. Together, these results indicate that the learning and representation of visual motion discrimination are mediated by different, but probably interacting, neuronal subsystems.

Perceptual learning in motion discrimination that generalizes across motion directions

When human subjects discriminate motion directions of two visual stimuli, their discrimination improves with practice. This improved performance has been found to be specific to the practiced directions and does not transfer to new motion directions. Indeed, such stimulus-specific learning has become a trademark finding in almost all perceptual learning studies and has been used to infer the loci of learning in the brain. For example, learning in motion discrimination has been inferred to occur in the visual area MT (medial temporal cortex) of primates, where neurons are selectively tuned to motion directions. However, such motion discrimination task is extremely difficult, as is typical of most perceptual learning tasks. When the difficulty is moderately reduced, learning transfers to new motion directions. This result challenges the idea of using simple visual stimuli to infer the locus of learning in low-level visual processes and suggests that higher-level processing is essential even in “simple” perceptual learning tasks.

Perceptual learning reflects external noise filtering and internal noise reduction through channel reweighting

To investigate the nature of plasticity in the adult visual system, perceptual learning was measured in a peripheral orientation discrimination task with systematically varying amounts of external (environmental) noise. The signal contrasts required to achieve threshold were reduced by a factor or two or more after training at all levels of external noise. The strong quantitative regularities revealed by this novel paradigm ruled out changes in multiplicative internal noise, changes in transducer nonlinearites, and simple attentional tradeoffs. Instead, the regularities specify the mechanisms of perceptual learning at the behavioral level as a combination of external noise exclusion and stimulus enhancement via additive internal noise reduction. The findings also constrain the neural architecture of perceptual learning. Plasticity in the weights between basic visual channels and decision is sufficient to account for perceptual learning without requiring the retuning of visual mechanisms.

Dynamics of subjective contour formation in the early visual cortex

To elucidate the roles of visual areas V1 and V2 and their interaction in early perceptual processing, we studied the responses of V1 and V2 neurons to statically displayed Kanizsa figures. We found evidence that V1 neurons respond to illusory contours of the Kanizsa figures. The illusory contour signals in V1 are weaker than in V2, but are significant, particularly in the superficial layers. The population averaged response to illusory contours emerged 100 msec after stimulus onset in the superficial layers of V1, and around 120–190 msec in the deep layers. The illusory contour response in V2 began earlier, occurring at 70 msec in the superficial layers and at 95 msec in the deep layers. The temporal sequence of the events suggests that the computation of illusory contours involves intercortical interaction, and that early perceptual organization is likely to be an interactive process.

Visual habit formation in monkeys with neurotoxic lesions of the ventrocaudal neostriatum

Visual habit formation in monkeys, assessed by concurrent visual discrimination learning with 24-h intertrial intervals (ITI), was found earlier to be impaired by removal of the inferior temporal visual area (TE) but not by removal of either the medial temporal lobe or inferior prefrontal convexity, two of TE's major projection targets. To assess the role in this form of learning of another pair of structures to which TE projects, namely the rostral portion of the tail of the caudate nucleus and the overlying ventrocaudal putamen, we injected a neurotoxin into this neostriatal region of several monkeys and tested them on the 24-h ITI task as well as on a test of visual recognition memory. Compared with unoperated monkeys, the experimental animals were unaffected on the recognition test but showed an impairment on the 24-h ITI task that was highly correlated with the extent of their neostriatal damage. The findings suggest that TE and its projection areas in the ventrocaudal neostriatum form part of a circuit that selectively mediates visual habit formation.

Neural mechanisms for visual memory and their role in attention

Recent studies show that neuronal mechanisms for learning and memory both dynamically modulate and permanently alter the representations of visual stimuli in the adult monkey cortex. Three commonly observed neuronal effects in memory-demanding tasks are repetition suppression, enhancement, and delay activity. In repetition suppression, repeated experience with the same visual stimulus leads to both short- and long-term suppression of neuronal responses in subpopulations of visual neurons. Enhancement works in an opposite fashion, in that neuronal responses are enhanced for objects with learned behavioral relevance. Delay activity is found in tasks in which animals are required to actively hold specific information “on-line” for short periods. Repetition suppression appears to be an intrinsic property of visual cortical areas such as inferior temporal cortex and is thought to be important for perceptual learning and priming. By contrast, enhancement and delay activity may depend on feedback to temporal cortex from prefrontal cortex and are thought to be important for working memory. All of these mnemonic effects on neuronal responses bias the competitive interactions that take place between stimulus representations in the cortex when there is more than one stimulus in the visual field. As a result, memory will often determine the winner of these competitions and, thus, will determine which stimulus is attended.

Traces of learning in the auditory localization pathway

One of the fascinating properties of the central nervous system is its ability to learn: the ability to alter its functional properties adaptively as a consequence of the interactions of an animal with the environment. The auditory localization pathway provides an opportunity to observe such adaptive changes and to study the cellular mechanisms that underlie them. The midbrain localization pathway creates a multimodal map of space that represents the nervous system's associations of auditory cues with locations in visual space. Various manipulations of auditory or visual experience, especially during early life, that change the relationship between auditory cues and locations in space lead to adaptive changes in auditory localization behavior and to corresponding changes in the functional and anatomical properties of this pathway. Traces of this early learning persist into adulthood, enabling adults to reacquire patterns of connectivity that were learned initially during the juvenile period.

Interocular transfer in perceptual learning of a pop-out discrimination task.

The specificity of the improvement in perceptual learning is often used to localize the neuronal changes underlying this type of adult plasticity. We investigated a visual texture discrimination task previously reported to be accomplished preattentively and for which learning-related changes were inferred to occur at a very early level of the visual processing stream. The stimulus was a matrix of lines from which a target popped out, due to an orientation difference between the three target lines and the background lines. The task was to report the global orientation of the target and was performed monocularly. The subjects' performance improved dramatically with training over the course of 2-3 weeks, after which we tested the specificity of the improvement for the eye trained. In all subjects tested, there was complete interocular transfer of the learning effect. The neuronal correlate of this learning are therefore most likely localized in a visual area where input from the two eyes has come together.

Learning and recall of form discriminations during reversible cooling deactivation of ventral-posterior suprasylvian cortex in the cat.

Extrastriate visual cortex of the ventral-posterior suprasylvian gyrus (vPS cortex) of freely behaving cats was reversibly deactivated with cooling to determine its role in performance on a battery of simple or masked two-dimensional pattern discriminations, and three-dimensional object discriminations. Deactivation of vPS cortex by cooling profoundly impaired the ability of the cats to recall the difference between all previously learned pattern and object discriminations. However, the cats' ability to learn or relearn pattern and object discriminations while vPS was deactivated depended upon the nature of the pattern or object and the cats' prior level of exposure to them. During cooling of vPS cortex, the cats could neither learn the novel object discriminations nor relearn a highly familiar masked or partially occluded pattern discrimination, although they could relearn both the highly familiar object and simple pattern discriminations. These cooling-induced deficits resemble those induced by cooling of the topologically equivalent inferotemporal cortex of monkeys and provides evidence that the equivalent regions contribute to visual processing in similar ways.

Receptive field structure in the visual cortex: does selective stimulation induce plasticity?

Sensory areas of adult cerebral cortex can reorganize in response to long-term alterations in patterns of afferent signals. This long-term plasticity is thought to play a crucial role in recovery from injury and in some forms of learning. However, the degree to which sensory representations in primary cortical areas depend on short-term (i.e., minute to minute) stimulus variations remains unclear. A traditional view is that each neuron in the mature cortex has a fixed receptive field structure. An alternative view, with fundamentally different implications for understanding cortical function, is that each cell's receptive field is highly malleable, changing according to the recent history of the sensory environment. Consistent with the latter view, it has been reported that selective stimulation of regions surrounding the receptive field induces a dramatic short-term increase in receptive field size for neurons in the visual cortex [Pettet, M. W. & Gilbert, C. D. (1992) Proc. Natl. Acad. Sci. USA 89, 8366-8370]. In contrast, we report here that there is no change in either the size or the internal structure of the receptive field following several minutes of surround stimulation. However, for some cells, overall responsiveness increases. These results suggest that dynamic alterations of receptive field structure do not underlie short-term plasticity in the mature primary visual cortex. However, some degree of short-term adaptability could be mediated by changes in responsiveness.