Tracking in a space variant active vision system

Without the ability to foveate on and maintain foveation, active vision for applications such as surveillance, object recognition and object tracking are difficult to build. Although foveation in cartesian coordinates is being actively pursued by many, multi-resolution high accuracy foveation in log polar space has not been given much attention. This paper addresses the use of foveation to track a single object as well as multiple objects for a simulated space variant active vision system. Complex logarithmic mapping is chosen firstly because it provides high resolution and wide angle viewing. Secondly, the spatially variant structure of log polar space leads to an object increasing in size as it moves towards the fovea. This is important as we know which object is closer to the fovea at any instant in time.

Resolution considerations in spatially variant sensors

Log polar transformations for space variant systems have been proposed and used in active vision research. The idea is to generate an image with a varying resolution over a wide angle field of view. The fovea is of high resolution and the periphery is of exponentially reduced resolution. The justifications for such a sensor are: (i) it provides high resolution and a wide viewing angle, (ii) feature invariance in the fovea simplifies foveation, and (iii) it allows multiresolution analysis. The receptor density of the human retina is very high, i.e. of the order of 106 receptors at the fovea. The question is, what resolution should space variant active vision systems have? Real visual sensors have been implemented but is the resolution produced high enough? This paper investigates the resolution requirements of a space variant sensor by simulation for a tracking system using raytracing

Looking left with left neglect: the role of spatial attention when active vision selects local image features for fixation

When we actively explore the visual environment, our gaze preferentially selects regions characterized by high contrast and high density of edges, suggesting that the guidance of eye movements during visual exploration is driven to a significant degree by perceptual characteristics of a scene. Converging findings suggest that the selection of the visual target for the upcoming saccade critically depends on a covert shift of spatial attention. However, it is unclear whether attention selects the location of the next fixation uniquely on the basis of global scene structure or additionally on local perceptual information. To investigate the role of spatial attention in scene processing, we examined eye fixation patterns of patients with spatial neglect during unconstrained exploration of natural images and compared these to healthy and brain-injured control participants. We computed luminance, colour, contrast, and edge information contained in image patches surrounding each fixation and evaluated whether they differed from randomly selected image patches. At the global level, neglect patients showed the characteristic ipsilesional shift of the distribution of their fixations. At the local level, patients with neglect and control participants fixated image regions in ipsilesional space that were closely similar with respect to their local feature content. In contrast, when directing their gaze to contralesional (impaired) space neglect patients fixated regions of significantly higher local luminance and lower edge content than controls. These results suggest that intact spatial attention is necessary for the active sampling of local feature content during scene perception.

A Binocular, Foveated Active Vision System

This report documents the design and implementation of a binocular, foveated active vision system as part of the Cog project at the MIT Artificial Intelligence Laboratory. The active vision system features a three degree of freedom mechanical platform that supports four color cameras, a motion control system, and a parallel network of digital signal processors for image processing. To demonstrate the capabilities of the system, we present results from four sample visual-motor tasks.

Hardware development of the Yorick series of active vision systems

This paper presents a review of the design and development of the Yorick series of active stereo camera platforms and their integration into real-time closed loop active vision systems, whose applications span surveillance, navigation of autonomously guided vehicles (AGVs), and inspection tasks for teleoperation, including immersive visual telepresence. The mechatronic approach adopted for the design of the first system, including head/eye platform, local controller, vision engine, gaze controller and system integration, proved to be very successful. The design team comprised researchers with experience in parallel computing, robot control, mechanical design and machine vision. The success of the project has generated sufficient interest to sanction a number of revisions of the original head design, including the design of a lightweight compact head for use on a robot arm, and the further development of a robot head to look specifically at increasing visual resolution for visual telepresence. The controller and vision processing engines have also been upgraded, to include the control of robot heads on mobile platforms and control of vergence through tracking of an operator's eye movement. This paper details the hardware development of the different active vision/telepresence systems.

Hypothesis verification using parametric models and active vision strategies

This paper proposes a methodology for determining the shape and ultimately the functionality of objects from intensity images; 2D analytic functions are used to track 3D features during known camera motions. Three analytic functions are proposed that describe the relationship between pairs of points that are either stationary or moving depending on whether the points are on occluding boundaries or otherwise. Many of the problems of correspondence are reduced by using foveation, known camera motion, and active vision methods. The three analytic functions are shown to enable hypothesis refinement of the functionality of a number of 3D objects without full 3D information about the shape.

Obtaining functional parametric models using active vision strategies

This paper addresses the problem of determining which 3D shape is present, and more importantly, the dimensions of the shape in a scene. This is performed in an active vision system because it reduces the complexity of the problem through the use of gaze stabilization, choice of foveation point, and selective processing by adaptively processing regions of interest. In our case, only a small number of equations and parameters are needed for each shape and these are incorporated into functional descriptions of the shapes.

Emerging hypothesis verification using function-based geometric models and active vision strategies

This paper describes an investigation into the use of parametric 2D models describing the movement of edges for the determination of possible 3D shape and hence function of an object. An assumption of this research is that the camera can foveate and track particular features. It is argued that simple 2D analytic descriptions of the movement of edges can infer 3D shape while the camera is moved. This uses an advantage of foveation i.e. the problem becomes object centred. The problem of correspondence for numerous edge points is overcome by the use of a tree based representation for the competing hypotheses. Numerous hypothesis are maintained simultaneously and it does not rely on a single kinematic model which assumes constant velocity or acceleration. The numerous advantages of this strategy are described.

Obtaining function models using active vision strategies

This paper addresses the problem of determening which 3D shape is present, and more importantly, the dimensions of the shape within a scene. This is performed in an active vision system because it reduces the complexity of the problem through the use of gaze stabilisation, choice of foveation point and selective processing by adaptively processing regions of interest. In our case only a small number of equations and parameters are needed for each shape. For example, a container has width and height. These are incorporated into functional descriptions of the shapes.