Image recognition: Visual grouping, recognition, and learning

Vision extracts useful information from images. Reconstructing the three-dimensional structure of our environment and recognizing the objects that populate it are among the most important functions of our visual system. Computer vision researchers study the computational principles of vision and aim at designing algorithms that reproduce these functions. Vision is difficult: the same scene may give rise to very different images depending on illumination and viewpoint. Typically, an astronomical number of hypotheses exist that in principle have to be analyzed to infer a correct scene description. Moreover, image information might be extracted at different levels of spatial and logical resolution dependent on the image processing task. Knowledge of the world allows the visual system to limit the amount of ambiguity and to greatly simplify visual computations. We discuss how simple properties of the world are captured by the Gestalt rules of grouping, how the visual system may learn and organize models of objects for recognition, and how one may control the complexity of the description that the visual system computes.

On the role of selective attention in visual perception

What is the role of selective attention in visual perception? Before answering this question, it is necessary to differentiate between attentional mechanisms that influence the identification of a stimulus from those that operate after perception is complete. Cognitive neuroscience techniques are particularly well suited to making this distinction because they allow different attentional mechanisms to be isolated in terms of timing and/or neuroanatomy. The present article describes the use of these techniques in differentiating between perceptual and postperceptual attentional mechanisms and then proposes a specific role of attention in visual perception. Specifically, attention is proposed to resolve ambiguities in neural coding that arise when multiple objects are processed simultaneously. Evidence for this hypothesis is provided by two experiments showing that attention—as measured electrophysiologically—is allocated to visual search targets only under conditions that would be expected to lead to ambiguous neural coding.

Faithful representation of similarities among three-dimensional shapes in human vision.

Efficient and reliable classification of visual stimuli requires that their representations reside a low-dimensional and, therefore, computationally manageable feature space. We investigated the ability of the human visual system to derive such representations from the sensory input-a highly nontrivial task, given the million or so dimensions of the visual signal at its entry point to the cortex. In a series of experiments, subjects were presented with sets of parametrically defined shapes; the points in the common high-dimensional parameter space corresponding to the individual shapes formed regular planar (two-dimensional) patterns such as a triangle, a square, etc. We then used multidimensional scaling to arrange the shapes in planar configurations, dictated by their experimentally determined perceived similarities. The resulting configurations closely resembled the original arrangements of the stimuli in the parameter space. This achievement of the human visual system was replicated by a computational model derived from a theory of object representation in the brain, according to which similarities between objects, and not the geometry of each object, need to be faithfully represented.

Image processing via level set curvature flow.

We present a controlled image smoothing and enhancement method based on a curvature flow interpretation of the geometric heat equation. Compared to existing techniques, the model has several distinct advantages. (i) It contains just one enhancement parameter. (ii) The scheme naturally inherits a stopping criterion from the image; continued application of the scheme produces no further change. (iii) The method is one of the fastest possible schemes based on a curvature-controlled approach.

Excitatory–inhibitory network in the visual cortex: Psychophysical evidence

At early stages in visual processing cells respond to local stimuli with specific features such as orientation and spatial frequency. Although the receptive fields of these cells have been thought to be local and independent, recent physiological and psychophysical evidence has accumulated, indicating that the cells participate in a rich network of local connections. Thus, these local processing units can integrate information over much larger parts of the visual field; the pattern of their response to a stimulus apparently depends on the context presented. To explore the pattern of lateral interactions in human visual cortex under different context conditions we used a novel chain lateral masking detection paradigm, in which human observers performed a detection task in the presence of different length chains of high-contrast-flanked Gabor signals. The results indicated a nonmonotonic relation of the detection threshold with the number of flankers. Remote flankers had a stronger effect on target detection when the space between them was filled with other flankers, indicating that the detection threshold is caused by dynamics of large neuronal populations in the neocortex, with a major interplay between excitation and inhibition. We considered a model of the primary visual cortex as a network consisting of excitatory and inhibitory cell populations, with both short- and long-range interactions. The model exhibited a behavior similar to the experimental results throughout a range of parameters. Experimental and modeling results indicated that long-range connections play an important role in visual perception, possibly mediating the effects of context.

Coinciding early activation of the human primary visual cortex and anteromedial cuneus

Proper understanding of processes underlying visual perception requires information on the activation order of distinct brain areas. We measured dynamics of cortical signals with magnetoencephalography while human subjects viewed stimuli at four visual quadrants. The signals were analyzed with minimum current estimates at the individual and group level. Activation emerged 55–70 ms after stimulus onset both in the primary posterior visual areas and in the anteromedial part of the cuneus. Other cortical areas were active after this initial dual activation. Comparison of data between species suggests that the anteromedial cuneus either comprises a homologue of the monkey area V6 or is an area unique to humans. Our results show that visual stimuli activate two cortical areas right from the beginning of the cortical response. The anteromedial cuneus has the temporal position needed to interact with the primary visual cortex V1 and thereby to modify information transferred via V1 to extrastriate cortices.

Segregating the functions of human hippocampus

It is now accepted that hippocampal lesions impair episodic memory. However, the precise functional role of the hippocampus in episodic memory remains elusive. Recent functional imaging data implicate the hippocampus in processing novelty, a finding supported by human in vivo recordings and event-related potential studies. Here we measure hippocampal responses to novelty, using functional MRI (fMRI), during an item-learning paradigm generated from an artificial grammar system. During learning, two distinct types of novelty were periodically introduced: perceptual novelty, pertaining to the physical characteristics of stimuli (in this case visual characteristics), and exemplar novelty, reflecting semantic characteristics of stimuli (in this case grammatical status within a rule system). We demonstrate a left anterior hippocampal response to both types of novelty and adaptation of these responses with stimulus familiarity. By contrast to these novelty effects, we also show bilateral posterior hippocampal responses with increasing exemplar familiarity. These results suggest a functional dissociation within the hippocampus with respect to the relative familiarity of study items. Neural responses in anterior hippocampus index generic novelty, whereas posterior hippocampal responses index familiarity to stimuli that have behavioral relevance (i.e., only exemplar familiarity). These findings add to recent evidence for functional segregation within the human hippocampus during learning.

Binocular visual surface perception.

Binocular disparity, the differential angular separation between pairs of image points in the two eyes, is the well-recognized basis for binocular distance perception. Without denying disparity's role in perceiving depth, we describe two perceptual phenomena, which indicate that a wider view of binocular vision is warranted. First, we show that disparity can play a critical role in two-dimensional perception by determining whether separate image fragments should be grouped as part of a single surface or segregated as parts of separate surfaces. Second, we show that stereoscopic vision is not limited to the registration and interpretation of binocular disparity but that it relies on half-occluded points, visible to one eye and not the other, to determine the layout and transparency of surfaces. Because these half-visible points are coded by neurons carrying eye-of-origin information, we suggest that the perception of these surface properties depends on neural activity available at visual cortical area V1.

Illusory contours activate specific regions in human visual cortex: evidence from functional magnetic resonance imaging.

The neural basis for perceptual grouping operations in the human visual system, including the processes which generate illusory contours, is fundamental to understanding human vision. We have employed functional magnetic resonance imaging to investigate these processes noninvasively. Images were acquired on a GE Signa 1.5T scanner equipped for echo planar imaging with an in-plane resolution of 1.5 x 1.5 mm and slice thicknesses of 3.0 or 5.0 mm. Visual stimuli included nonaligned inducers (pacmen) that created no perceptual contours, similar inducers at the corners of a Kanizsa square that created illusory contours, and a real square formed by continuous contours. Multiple contiguous axial slices were acquired during baseline, visual stimulation, and poststimulation periods. Activated regions were identified by a multistage statistical analysis of the activation for each volume element sampled and were compared across conditions. Specific brain regions were activated in extrastriate cortex when the illusory contours were perceived but not during conditions when the illusory contours were absent. These unique regions were found primarily in the right hemisphere for all four subjects and demonstrate that specific brain regions are activated during the kind of perceptual grouping operations involved in illusory contour perception.

Distinct neural correlates of visual long-term memory for spatial location and object identity: a positron emission tomography study in humans.

The purpose of the present study was to investigate by using positron emission tomography (PET) whether the cortical pathways that are involved in visual perception of spatial location and object identity are also differentially implicated in retrieval of these types of information from episodic long-term memory. Subjects studied a set of displays consisting of three unique representational line drawings arranged in different spatial configurations. Later, while undergoing PET scanning, subjects' memory for spatial location and identity of the objects in the displays was tested and compared to a perceptual baseline task involving the same displays. In comparison to the baseline task, each of the memory tasks activated both the dorsal and the ventral pathways in the right hemisphere but not to an equal extent. There was also activation of the right prefrontal cortex. When PET scans of the memory tasks were compared to each other, areas of activation were very circumscribed and restricted to the right hemisphere: For retrieval of object identity, the area was in the inferior temporal cortex in the region of the fusiform gyrus (area 37), whereas for retrieval of spatial location, it was in the inferior parietal lobule in the region of the supramarginal gyrus (area 40). Thus, our study shows that distinct neural pathways are activated during retrieval of information about spatial location and object identity from long-term memory.

Feature integration in pattern perception

The human visual system is able to effortlessly integrate local features to form our rich perception of patterns, despite the fact that visual information is discretely sampled by the retina and cortex. By using a novel perturbation technique, we show that the mechanisms by which features are integrated into coherent percepts are scale-invariant and nonlinear (phase and contrast polarity independent). They appear to operate by assigning position labels or “place tags” to each feature. Specifically, in the first series of experiments, we show that the positional tolerance of these place tags in foveal, and peripheral vision is about half the separation of the features, suggesting that the neural mechanisms that bind features into forms are quite robust to topographical jitter. In the second series of experiment, we asked how many stimulus samples are required for pattern identification by human and ideal observers. In human foveal vision, only about half the features are needed for reliable pattern interpolation. In this regard, human vision is quite efficient (ratio of ideal to real ≈ 0.75). Peripheral vision, on the other hand is rather inefficient, requiring more features, suggesting that the stimulus may be relatively underrepresented at the stage of feature integration.

Syntax processing by auditory cortical neurons in the FM–FM area of the mustached bat Pteronotus parnellii

Syntax denotes a rule system that allows one to predict the sequencing of communication signals. Despite its significance for both human speech processing and animal acoustic communication, the representation of syntactic structure in the mammalian brain has not been studied electrophysiologically at the single-unit level. In the search for a neuronal correlate for syntax, we used playback of natural and temporally destructured complex species-specific communication calls—so-called composites—while recording extracellularly from neurons in a physiologically well defined area (the FM–FM area) of the mustached bat’s auditory cortex. Even though this area is known to be involved in the processing of target distance information for echolocation, we found that units in the FM–FM area were highly responsive to composites. The finding that neuronal responses were strongly affected by manipulation in the time domain of the natural composite structure lends support to the hypothesis that syntax processing in mammals occurs at least at the level of the nonprimary auditory cortex.

GenMapDB: a database of mapped human BAC clones

GenMapDB (http://genomics.med.upenn.edu/genmapdb) is a repository of human bacterial artificial chromosome (BAC) clones mapped by our laboratory to sequence-tagged site markers. Currently, GenMapDB contains over 3000 mapped clones that span 19 chromosomes, chromosomes 2, 4, 5, 9–22, X and Y. This database provides positional information about human BAC clones from the RPCI-11 human male BAC library. It also contains restriction fragment analysis data and end sequences of the clones. GenMapDB is freely available to the public. The main purpose of GenMapDB is to organize the mapping data and to allow the research community to search for mapped BAC clones that can be used in gene mapping studies and chromosomal mutation analysis projects.

The extraordinarily rapid disappearance of entoptic images.

It has been known for more than 40 years that images fade from perception when they are kept at the same position on the retina by abrogating eye movements. Although aspects of this phenomenon were described earlier, the use of close-fitting contact lenses in the 1950s made possible a series of detailed observations on eye movements and visual continuity. In the intervening decades, many investigators have studied the role of image motion on visual perception. Although several controversies remain, it is clear that images deteriorate and in some cases disappear following stabilization; eye movements are, therefore, essential to sustained exoptic vision. The time course of image degradation has generally been reported to be a few seconds to a minute or more, depending upon the conditions. Here we show that images of entoptic vascular shadows can disappear in less than 80 msec. The rapid vanishing of these images implies an active mechanism of image erasure and creation as the basis of normal visual processing.

Sequence verification of human creatine kinase (43 kDa) isozymes by high-resolution tandem mass spectrometry.

Amino acid sequencing by recombinant DNA technology, although dramatically useful, is subject to base reading errors, is indirect, and is insensitive to posttranslational processing. Mass spectrometry techniques can provide molecular weight data from even relatively large proteins for such cDNA sequences and can serve as a check of an enzyme's purity and sequence integrity. Multiply-charged ions from electrospray ionization can be dissociated to yield structural information by tandem mass spectrometry, providing a second method for gaining additional confidence in primary sequence confirmation. Here, accurate (+/- 1 Da) molecular weight and molecular ion dissociation information for human muscle and brain creatine kinases has been obtained by electrospray ionization coupled with Fourier-transform mass spectrometry to help distinguish which of several published amino acid sequences for both enzymes are correct. The results herein are consistent with one published sequence for each isozyme, and the heterogeneity indicated by isoelectric focusing due to 1-Da deamidation changes. This approach appears generally useful for detailed sequence verification of recombinant proteins.