Image recognition: Visual grouping, recognition, and learning

Vision extracts useful information from images. Reconstructing the three-dimensional structure of our environment and recognizing the objects that populate it are among the most important functions of our visual system. Computer vision researchers study the computational principles of vision and aim at designing algorithms that reproduce these functions. Vision is difficult: the same scene may give rise to very different images depending on illumination and viewpoint. Typically, an astronomical number of hypotheses exist that in principle have to be analyzed to infer a correct scene description. Moreover, image information might be extracted at different levels of spatial and logical resolution dependent on the image processing task. Knowledge of the world allows the visual system to limit the amount of ambiguity and to greatly simplify visual computations. We discuss how simple properties of the world are captured by the Gestalt rules of grouping, how the visual system may learn and organize models of objects for recognition, and how one may control the complexity of the description that the visual system computes.

Illusory contours activate specific regions in human visual cortex: evidence from functional magnetic resonance imaging.

The neural basis for perceptual grouping operations in the human visual system, including the processes which generate illusory contours, is fundamental to understanding human vision. We have employed functional magnetic resonance imaging to investigate these processes noninvasively. Images were acquired on a GE Signa 1.5T scanner equipped for echo planar imaging with an in-plane resolution of 1.5 x 1.5 mm and slice thicknesses of 3.0 or 5.0 mm. Visual stimuli included nonaligned inducers (pacmen) that created no perceptual contours, similar inducers at the corners of a Kanizsa square that created illusory contours, and a real square formed by continuous contours. Multiple contiguous axial slices were acquired during baseline, visual stimulation, and poststimulation periods. Activated regions were identified by a multistage statistical analysis of the activation for each volume element sampled and were compared across conditions. Specific brain regions were activated in extrastriate cortex when the illusory contours were perceived but not during conditions when the illusory contours were absent. These unique regions were found primarily in the right hemisphere for all four subjects and demonstrate that specific brain regions are activated during the kind of perceptual grouping operations involved in illusory contour perception.

Behavioral Differences in the Upper and Lower Visual Hemifields in Shape and Motion Perception.

Perceptual accuracy is known to be influenced by stimuli location within the visual field. In particular, it seems to be enhanced in the lower visual hemifield (VH) for motion and space processing, and in the upper VH for object and face processing. The origins of such asymmetries are attributed to attentional biases across the visual field, and in the functional organization of the visual system. In this article, we tested content-dependent perceptual asymmetries in different regions of the visual field. Twenty-five healthy volunteers participated in this study. They performed three visual tests involving perception of shapes, orientation and motion, in the four quadrants of the visual field. The results of the visual tests showed that perceptual accuracy was better in the lower than in the upper visual field for motion perception, and better in the upper than in the lower visual field for shape perception. Orientation perception did not show any vertical bias. No difference was found when comparing right and left VHs. The functional organization of the visual system seems to indicate that the dorsal and the ventral visual streams, responsible for motion and shape perception, respectively, show a bias for the lower and upper VHs, respectively. Such a bias depends on the content of the visual information.

Visual thalamocortical projections in the flying fox: Parallel pathways to striate and extrastriate areas

We studied thalamic projections to the visual cortex in flying foxes, animals that share neural features believed to resemble those present in the brains of early primates. Neurones labeled by injections of fluorescent tracers in striate and extrastriate cortices were charted relative to the architectural boundaries of thalamic nuclei. Three main findings are reported: First, there are parallel lateral geniculate nucleus (LGN) projections to striate and extrastriate cortices. Second, the pulvinar complex is expansive, and contains multiple subdivisions. Third, across the visual thalamus, the location of cells labeled after visual cortex injections changes systematically, with caudal visual areas receiving their strongest projections from the most lateral thalamic nuclei, and rostral areas receiving strong projections from medial nuclei. We identified three architectural layers in the LGN, and three subdivisions of the pulvinar complex. The outer LGN layer contained the largest cells, and had strong projections to the areas V1, V2 and V3. Neurones in the intermediate LGN layer were intermediate in size, and projected to V1 and, less densely, to V2. The layer nearest to the origin of the optic radiation contained the smallest cells, and projected not only to V1, V2 and V3, but also, weakly, to the occipitotemporal area (OT, which is similar to primate middle temporal area) and the occipitoparietal area (OP, a third tier area located near the dorsal midline). V1, V2 and V3 received strong projections from the lateral and intermediate subdivisions of the pulvinar complex, while OP and OT received their main thalamic input from the intermediate and medial subdivisions of the pulvinar complex. These results suggest parallels with the carnivore visual system, and indicate that the restriction of the projections of the large- and intermediatesized LGN layers to V1, observed in present-day primates, evolved from a more generalized mammalian condition. (C) 2004 IBRO. Published by Elsevier Ltd. All rights reserved.

Animal visual systems and the evolution of color patterns: Sensory processing illuminates signal evolution

Animal color pattern phenotypes evolve rapidly. What influences their evolution? Because color patterns are used in communication, selection for signal efficacy, relative to the intended receiver's visual system, may explain and predict the direction of evolution. We investigated this in bowerbirds, whose color patterns consist of plumage, bower structure, and ornaments and whose visual displays are presented under predictable visual conditions. We used data on avian vision, environmental conditions, color pattern properties, and an estimate of the bowerbird phylogeny to test hypotheses about evolutionary effects of visual processing. Different components of the color pattern evolve differently. Plumage sexual dimorphism increased and then decreased, while overall (plumage plus bower) visual contrast increased. The use of bowers allows relative crypsis of the bird but increased efficacy of the signal as a whole. Ornaments do not elaborate existing plumage features but instead are innovations (new color schemes) that increase signal efficacy. Isolation between species could be facilitated by plumage but not ornaments, because we observed character displacement only in plumage. Bowerbird color pattern evolution is at least partially predictable from the function of the visual system and from knowledge of different functions of different components of the color patterns. This provides clues to how more constrained visual signaling systems may evolve.

Biologically inspired visual landmark processing for simultaneous localization and mapping

This paper illustrates a method for finding useful visual landmarks for performing simultaneous localization and mapping (SLAM). The method is based loosely on biological principles, using layers of filtering and pooling to create learned templates that correspond to different views of the environment. Rather than using a set of landmarks and reporting range and bearing to the landmark, this system maps views to poses. The challenge is to produce a system that produces the same view for small changes in robot pose, but provides different views for larger changes in pose. The method has been developed to interface with the RatSLAM system, a biologically inspired method of SLAM. The paper describes the method of learning and recalling visual landmarks in detail, and shows the performance of the visual system in real robot tests.

Biological polarized light reflectors in stomatopod crustaceans

Body parts that can reflect highly polarized light have been found in several species of stomatopod crustaceans (mantis shrimps). These polarized light reflectors can be grossly divided into two major types. The first type, usually red or pink in color to the human visual system, is located within an animal’s cuticle. Reflectors of the second type, showing iridescent blue, are located beneath the exoskeleton and thus are unaffected by the molt cycle. We used reflection spectropolarimetry and transmission electron microscopy (TEM) to study the reflective properties and the structures that reflect highly polarized light in stomatopods. For the first type of reflector, the degree of polarization usually changes dramatically, from less than 20% to over 70%, with a change in viewing angle. TEM examination indicates that the polarization reflection is generated by multilayer thin-film interference. The second type of reflector, the blue colored ones, reflects highly polarized light to all viewing angles. However, these reflectors show a slight chromatic change with different viewing angles. TEM sections have revealed that streams of oval-shaped vesicles might be responsible for the production of the polarized light reflection. In all the reflectors we have examined so far, the reflected light is always maximally polarized at around 500 nm, which is close to the wavelength best transmitted by sea water. This suggests that the polarized light reflectors found in stomatopods are well adapted to the underwater environment. We also found that most reflectors produce polarized light with a horizontal e-vector. How these polarized light reflectors are used in stomatopod signaling remains unknown.

Colour and luminance interactions in the visual perception of motion

We sought to determine the extent to which red–green, colour–opponent mechanisms in the human visual system play a role in the perception of drifting luminance–modulated targets. Contrast sensitivity for the directional discrimination of drifting luminance–modulated (yellow–black) test sinusoids was measured following adaptation to isoluminant red–green sinusoids drifting in either the same or opposite direction. When the test and adapt stimuli drifted in the same direction, large sensitivity losses were evident at all test temporal frequencies employed (1–16 Hz). The magnitude of the loss was independent of temporal frequency. When adapt and test stimuli drifted in opposing directions, large sensitivity losses were evident at lower temporal frequencies (1–4 Hz) and declined with increasing temporal frequency. Control studies showed that this temporal–frequency–dependent effect could not reflect the activity of achromatic units. Our results provide evidence that chromatic mechanisms contribute to the perception of luminance–modulated motion targets drifting at speeds of up to at least 32°s-1. We argue that such mechanisms most probably lie within a parvocellular–dominated cortical visual pathway, sensitive to both chromatic and luminance modulation, but only weakly selective for the direction of stimulus motion.

Alzheimer's disease Part 2: visual changes

This article describes the pathological changes that have been observed in different parts of the visual system in Alzheimer's disease as well as the visual symptoms which may result from these changes.

The influence of age on the pattern and flash visual evoked field

We have investigated the effect of ageing on the visual system using the relatively new technique of magentoencephalography (MEG). This technique measures the magnetic signals produced by the visual system using a SQUID magnetometer. The magnetic visual evoked field (VEF) was measured over the occipital cortex to pattern and flash stimuli in 86 normal subjects aged 15 - 86 years. Factors that influenced subject defocussing or defixating the stimulus or selective attention were controlled as far as possible. The latency of the major positive component to the pattern reversal stimulus (P100M) increased with age particularly after the age of 55 years while the amplitude of the P100M decreased over the life span. The latency of the major flash component (P2M) increased much more slowly with age, while its amplitude decreased in only a proportion of elderly subjects. Changes in the P100M with age may reflect senile changes in the eye and optic nerve, e.g. senile miosis or degenerative changes in the retina. The P2M may be more susceptible to senile changes in the retina. The data suggest that the spatial frequency channels deteriorate more rapidly with age than the luminance channels and that MEG may be an effective method of studying ageing in the visual system.

Topography of visual evoked magnetic responses to pattern shift, pattern onset and flash stimuli

Different visual stimuli may activate separate channels in the visual system and produce magnetic responses from the human bran which originate from distinct regions of the visual cortex. To test this hypothesis, we have investigated the distribution of visual evoked magnetic responses to three distinct visual stimuli over the occipital region of the scalp with a DC-SQUID second-order gradiometer in an ubshielded environment. Patterned stimuli were presented full field and to the right half field, while a flash stimulus was presented full field only, in five normal subjects. Magnetic responses were recorded from 20 to 42 positions over the occipital scalp. Topographic maps were prepared of the major positive component within the first 150ms to the three stimuli, i.e., the P100m (pattern shift), C11m (pattern onset) and P2m (flash). For the pattern shift stimulus the data suggested the source of the P100m was close to the midline with the current directed towards the medial surface. The data for the pattern onset C11m suggested a source at a similar depth but with the current directed away from the midline towards the lateral surface. The flash P2m appeared to originate closer to the surface of the occipital pole than both the patterned stimuli. Hence the pattern shift (which may represent movement), and the pattern onset C11m (representing contrast and contour) appear to originate in similar areas of brain but to represent different asepcts of cortical processing. By contrast, the flash P2m (representing luminance change) appears to originate in a distinct area of visual cortex closer to the occipital pole.

Magnetoencephalography:technical improvements in image co-registration and studies of visual cortical oscillations

The work presented in this thesis is divided into two distinct sections. In the first, the functional neuroimaging technique of Magnetoencephalography (MEG) is described and a new technique is introduced for accurate combination of MEG and MRI co-ordinate systems. In the second part of this thesis, MEG and the analysis technique of SAM are used to investigate responses of the visual system in the context of functional specialisation within the visual cortex. In chapter one, the sources of MEG signals are described, followed by a brief description of the necessary instrumentation for accurate MEG recordings. This chapter is concluded by introducing the forward and inverse problems of MEG, techniques to solve the inverse problem, and a comparison of MEG with other neuroimaging techniques. Chapter two provides an important contribution to the field of research with MEG. Firstly, it is described how MEG and MRI co-ordinate systems are combined for localisation and visualisation of activated brain regions. A previously used co-registration methods is then described, and a new technique is introduced. In a series of experiments, it is demonstrated that using fixed fiducial points provides a considerable improvement in the accuracy and reliability of co-registration. Chapter three introduces the visual system starting from the retina and ending with the higher visual rates. The functions of the magnocellular and the parvocellular pathways are described and it is shown how the parallel visual pathways remain segregated throughout the visual system. The structural and functional organisation of the visual cortex is then described. Chapter four presents strong evidence in favour of the link between conscious experience and synchronised brain activity. The spatiotemporal responses of the visual cortex are measured in response to specific gratings. It is shown that stimuli that induce visual discomfort and visual illusions share their physical properties with those that induce highly synchronised gamma frequency oscillations in the primary visual cortex. Finally chapter five is concerned with localization of colour in the visual cortex. In this first ever use of Synthetic Aperture Magnetometry to investigate colour processing in the visual cortex, it is shown that in response to isoluminant chromatic gratings, the highest magnitude of cortical activity arise from area V2.

Effects of pixel noise and stimulus structure on visual detection performance

This thesis consisted of two major parts, one determining the masking characteristics of pixel noise and the other investigating the properties of the detection filter employed by the visual system. The theoretical cut-off frequency of white pixel noise can be defined from the size of the noise pixel. The empirical cut-off frequency, i.e. the largest size of noise pixels that mimics the effect of white noise in detection, was determined by measuring contrast energy thresholds for grating stimuli in the presence of spatial noise consisting of noise pixels of various sizes and shapes. The critical i.e. minimum number of noise pixels per grating cycle needed to mimic the effect of white noise in detection was found to decrease with the bandwidth of the stimulus. The shape of the noise pixels did not have any effect on the whiteness of pixel noise as long as there was at least the minimum number of noise pixels in all spatial dimensions. Furthermore, the masking power of white pixel noise is best described when the spectral density is calculated by taking into account all the dimensions of noise pixels, i.e. width, height, and duration, even when there is random luminance only in one of these dimensions. The properties of the detection mechanism employed by the visual system were studied by measuring contrast energy thresholds for complex spatial patterns as a function of area in the presence of white pixel noise. Human detection efficiency was obtained by comparing human performance with an ideal detector. The stimuli consisted of band-pass filtered symbols, uniform and patched gratings, and point stimuli with randomised phase spectra. In agreement with the existing literature, the detection performance was found to decline with the increasing amount of detail and contour in the stimulus. A measure of image complexity was developed and successfully applied to the data. The accuracy of the detection mechanism seems to depend on the spatial structure of the stimulus and the spatial spread of contrast energy.

Neuro-electromagnetic imaging of the human visual cortex

Methods of solving the neuro-electromagnetic inverse problem are examined and developed, with specific reference to the human visual cortex. The anatomy, physiology and function of the human visual system are first reviewed. Mechanisms by which the visual cortex gives rise to external electric and magnetic fields are then discussed, and the forward problem is described mathematically for the case of an isotropic, piecewise homogeneous volume conductor, and then for an anisotropic, concentric, spherical volume conductor. Methods of solving the inverse problem are reviewed, before a new technique is presented. This technique combines prior anatomical information gained from stereotaxic studies, with a probabilistic distributed-source algorithm to yield accurate, realistic inverse solutions. The solution accuracy is enhanced by using both visual evoked electric and magnetic responses simultaneously. The numerical algorithm is then modified to perform equivalent current dipole fitting and minimum norm estimation, and these three techniques are implemented on a transputer array for fast computation. Due to the linear nature of the techniques, they can be executed on up to 22 transputers with close to linear speedup. The latter part of the thesis describes the application of the inverse methods to the analysis of visual evoked electric and magnetic responses. The CIIm peak of the pattern onset evoked magnetic response is deduced to be a product of current flowing away from the surface areas 17, 18 and 19, while the pattern reversal P100m response originates in the same areas, but from oppositely directed current. Cortical retinotopy is examined using sectorial stimuli, the CI and CIm ;peaks of the pattern onset electric and magnetic responses are found to originate from areas V1 and V2 simultaneously, and they therefore do not conform to a simple cruciform model of primary visual cortex.

The chromatic visual evoked response as an indication of visual development

In an endeavour to provide further insight into the maturation of the cortical visual system in human infants, chromatic transient pattern reversal visual evoked potentials to red/green stimuli, were studied in a group of normal full term infants between the ages of 1 and 14 weeks post term in both cross sectional and longitudinal studies. In order to produce stimuli in which luminance cues had been eliminated with an aim to eliciting a chromatic response, preliminary studies of isoluminance determination in adults and infants were undertaken using behavioural and electrophysiological techniques. The results showed close similarity between the isoluminant ratio for adults and infants and all values were close to photometric isoluminance. Pattern reversal VEPs were recorded to stimuli of a range of red/green luminance ratios and an achromatic checkerboard. No transient VEP could be elicited with an isoluminant chromatic pattern reversal stimulus from any infant less than 7 weeks post term and similarly, all infants more than 7 weeks post term showed clear chromatic VEPs. The chromatic response first appeared at that age as a major positive component (P1) of long latency. This was delayed and reduced in comparison to the achromatic response. As the infant grew older, the latency of the P1 component decreased with the appearance of N1 and N by the 10th week post term. This finding was consistent throughout all infants assessed. In a behavioural study, no infant less than 7 weeks post term demonstrated clear discrimination of the chromatic stimulus, while those infants older than 7 weeks could do so. These findings are reviewed with respect to current neural models of visual development.