Hardware development of the Yorick series of active vision systems

This paper presents a review of the design and development of the Yorick series of active stereo camera platforms and their integration into real-time closed loop active vision systems, whose applications span surveillance, navigation of autonomously guided vehicles (AGVs), and inspection tasks for teleoperation, including immersive visual telepresence. The mechatronic approach adopted for the design of the first system, including head/eye platform, local controller, vision engine, gaze controller and system integration, proved to be very successful. The design team comprised researchers with experience in parallel computing, robot control, mechanical design and machine vision. The success of the project has generated sufficient interest to sanction a number of revisions of the original head design, including the design of a lightweight compact head for use on a robot arm, and the further development of a robot head to look specifically at increasing visual resolution for visual telepresence. The controller and vision processing engines have also been upgraded, to include the control of robot heads on mobile platforms and control of vergence through tracking of an operator's eye movement. This paper details the hardware development of the different active vision/telepresence systems.

Looking left with left neglect: the role of spatial attention when active vision selects local image features for fixation

When we actively explore the visual environment, our gaze preferentially selects regions characterized by high contrast and high density of edges, suggesting that the guidance of eye movements during visual exploration is driven to a significant degree by perceptual characteristics of a scene. Converging findings suggest that the selection of the visual target for the upcoming saccade critically depends on a covert shift of spatial attention. However, it is unclear whether attention selects the location of the next fixation uniquely on the basis of global scene structure or additionally on local perceptual information. To investigate the role of spatial attention in scene processing, we examined eye fixation patterns of patients with spatial neglect during unconstrained exploration of natural images and compared these to healthy and brain-injured control participants. We computed luminance, colour, contrast, and edge information contained in image patches surrounding each fixation and evaluated whether they differed from randomly selected image patches. At the global level, neglect patients showed the characteristic ipsilesional shift of the distribution of their fixations. At the local level, patients with neglect and control participants fixated image regions in ipsilesional space that were closely similar with respect to their local feature content. In contrast, when directing their gaze to contralesional (impaired) space neglect patients fixated regions of significantly higher local luminance and lower edge content than controls. These results suggest that intact spatial attention is necessary for the active sampling of local feature content during scene perception.

Towards Active Vision in the DUAL Cognitive Architecture

The paper describes an extension of the cognitive architecture DUAL with a model of visual attention and perception. The goal of this attempt is to account for the construction and the categorization of object and scene representations derived from visual stimuli in the TextWorld microdomain. Low-level parallel computations are combined with an active serial deployment of visual attention enabling the construction of abstract symbolic representations. A limited-capacity short-term visual store holding information across attention shifts forms the core of the model interfacing between the low-level representation of the stimulus and DUAL’s semantic memory. The model is validated by comparing the results of a simulation with real data from an eye movement experiment with human subjects.

DESEO: an active vision system for detection, tracking and recognition

[EN]In this paper, a basic conceptual architecture aimed at the design of Computer Vision System is qualitatively described. The proposed architecture addresses the design of vision systems in a modular fashion using modules with three distinct units or components: a processing network or diagnostics unit, a control unit and a communications unit. The control of the system at the modules level is designed based on a Discrete Events Model. This basic methodology has been used to design a realtime active vision system for detection, tracking and recognition of people. It is made up of three functional modules aimed at the detection, tracking, recognition of moving individuals plus a supervision module.

An active vision system integrating fast and slow processes

[EN]This paper describes an Active Vision System whose design assumes a distinction between fast or reactive and slow or background processes. Fast processes need to operate in cycles with critical timeouts that may affect system stability. While slow processes, though necessary, do not compromise system stability if its execution is delayed. Based on this simple taxonomy, a control architecture has been proposed and a prototype implemented that is able to track people in real-time with a robotic head while trying to identify the target. In this system, the tracking module is considered as the reactive part of the system while person identification is considered a background task.

A C80 DSP-based active vision system for real-time tracking

[EN]Active Vision Systems can be considered as dynamical systems which close the loop around artificial visual perception, controlling camera parameters, motion and also controlling processing to simplify, accelerate and do more robust visual perception. Research and Development in Active Vision Systems [Aloi87], [Bajc88] is a main area of interest in Computer Vision, mainly by its potential application in different scenarios where real-time performance is needed such as robot navigation, surveillance, visual inspection, among many others. Several systems have been developed during last years using robotic-heads for this purpose...

Sistema de Visão Stereo Ativo aplicado aos robôs do ISePorto

A presente dissertação endereça o desenvolvimento de um sistema de visão stereo ativo para os robôs de futebol robótico da equipa ISePorto do ISEP, de modo a que estes tirem o máximo partido das câmaras rotativas neles existentes. Este trabalho surge da necessidade de melhorar a capacidade de perceção do ambiente por parte dos robôs, principalmente da perceção da bola quando não está no plano do campo e dos robôs adversários. Esta necessidade surge devido ao aumento da dinâmica que se tem vindo a veri car ultimamente nas competições. Para tal, foram estudados algumas trabalhos relacionados no que diz respeito a sistemas de visão stereo com baselines variáveis e eixos de rotação em ambas as câmaras, bem como fundamentos de visão stereo. Foi proposta uma arquitetura para o sistema de visão ativo de modo a ser aplicado em qualquer robô da equipa MSL (Middle Size League). Para tornar possível a implementação desta arquitetura foi desenvolvido um procedimento para a calibração e determinação em tempo real dos parâmetros extrínsecos do par stereo em função da posição angular dos eixos rotativos do robô. O sistema de visão foi também dotado de capacidade de sincronismo e foram implementadas funcionalidades ao nível de software que possibilitam a deteção de objetos na imagem, a correspondência de objetos presentes nas imagens de ambas as câmaras e consequentemente a determinação das posições tridimensionais desses objetos relativamente ao robô. O sistema desenvolvido foi testado e validado em cenário MSL ao nível de perceção da bola, robôs adversários e linhas do campo. Os resultados obtidos apresentam uma melhoria signi cativa, face à implementação atual dos robôs, na perceção tridimensional da bola quando não está no plano do campo, e dos robôs adversários.

Driving saccade to pursuit using image motion

Within the context of active vision, scant attention has been paid to the execution of motion saccades—rapid re-adjustments of the direction of gaze to attend to moving objects. In this paper we first develop a methodology for, and give real-time demonstrations of, the use of motion detection and segmentation processes to initiate capture saccades towards a moving object. The saccade is driven by both position and velocity of the moving target under the assumption of constant target velocity, using prediction to overcome the delay introduced by visual processing. We next demonstrate the use of a first order approximation to the segmented motion field to compute bounds on the time-to-contact in the presence of looming motion. If the bound falls below a safe limit, a panic saccade is fired, moving the camera away from the approaching object. We then describe the use of image motion to realize smooth pursuit, tracking using velocity information alone, where the camera is moved so as to null a single constant image motion fitted within a central image region. Finally, we glue together capture saccades with smooth pursuit, thus effecting changes in both what is being attended to and how it is being attended to. To couple the different visual activities of waiting, saccading, pursuing and panicking, we use a finite state machine which provides inherent robustness outside of visual processing and provides a means of making repeated exploration. We demonstrate in repeated trials that the transition from saccadic motion to tracking is more likely to succeed using position and velocity control, than when using position alone.

Delays versus performance of visually guided systems

The paper presents an overview of dynamic systems with inherent delays in both feedforward and feedback paths and how the performance of such systems can be affected by such delays. The authors concentrate on visually guided systems, where the behaviour of the system is largely dependent on the results of the vision sensors, with particular reference to active robot heads (real-time gaze control). We show how the performance of such systems can deteriorate substantially with the presence of unknown and/or variable delays. Considered choice of system architecture, however, allows the performance of active vision systems to be optimised with respect to the delays present in the system.