Driving saccade to pursuit using image motion

Within the context of active vision, scant attention has been paid to the execution of motion saccades—rapid re-adjustments of the direction of gaze to attend to moving objects. In this paper we first develop a methodology for, and give real-time demonstrations of, the use of motion detection and segmentation processes to initiate capture saccades towards a moving object. The saccade is driven by both position and velocity of the moving target under the assumption of constant target velocity, using prediction to overcome the delay introduced by visual processing. We next demonstrate the use of a first order approximation to the segmented motion field to compute bounds on the time-to-contact in the presence of looming motion. If the bound falls below a safe limit, a panic saccade is fired, moving the camera away from the approaching object. We then describe the use of image motion to realize smooth pursuit, tracking using velocity information alone, where the camera is moved so as to null a single constant image motion fitted within a central image region. Finally, we glue together capture saccades with smooth pursuit, thus effecting changes in both what is being attended to and how it is being attended to. To couple the different visual activities of waiting, saccading, pursuing and panicking, we use a finite state machine which provides inherent robustness outside of visual processing and provides a means of making repeated exploration. We demonstrate in repeated trials that the transition from saccadic motion to tracking is more likely to succeed using position and velocity control, than when using position alone.

Gaze pursuit and arm control of adolescent males diagnosed with attention deficit hyperactivity disorder (ADHD) and normal controls: evidence of a dissociation in processing visual information of short and long duration

Three-dimensional kinematic analysis of line of gaze, arm and ball was used to describe the visual and motor behaviour of male adolescents diagnosed with attention deficit hyperactivity disorder (ADHD). The ADHD participants were tested when both on (ADHD-On) and off (ADHD-Off) their medication and compared to age-matched normal controls in a modified table tennis task that required tracking the ball and hitting to cued right and left targets. Long-duration information was provided by a pre-cue, in which the target was illuminated approximately 2 s before the serve, and short-duration information by an early-cue illuminated about 350 ms after the serve, leaving -500 ms to select the target and perform the action. The ADHD groups differed significantly from the control group in both the pre-cue and early-cue conditions in being less accurate, in having a later onset and duration of pursuit tracking, and a higher frequency of gaze on and off the ball. The use of medication significantly reduced the gaze frequency of the ADHD participants, but surprisingly this did not lead to an increase in pursuit tracking, suggesting a barrier was reached beyond which ball flight information could not be processed. The control and ADHD groups did not differ in arm movement onset, duration and velocity in the short-duration early-cue condition; in the long-duration pre-cue condition, however, the ADHD group's movement time onset and arm velocity differed significantly from controls. The results show that the ADHD groups were able to process short-duration information without experiencing adverse effects on their motor behaviour; however, long-duration information contributed to irregular movement control.

Representative learning design in dynamic interceptive actions

The overarching aim of this thesis was to investigate how processes of perception and action emerge under changing informational constraints during performance of multi-articular interceptive actions. Interceptive actions provide unique opportunities to study processes of perception and action in dynamic performance environments. The movement model used to exemplify the functionally coupled relationship between perception and action, from an ecological dynamics perspective, was cricket batting. Ecological dynamics conceptualises the human body as a complex system composed of many interacting sub-systems, and perceptual and motor system degrees of freedom, which leads to the emergence of patterns of behaviour under changing task constraints during performance. The series of studies reported in the Chapters of this doctoral thesis contributed to understanding of human behaviour by providing evidence of key properties of complex systems in human movement systems including self-organisation under constraints and meta-stability. Specifically, the studies: i) demonstrated how movement organisation (action) and visual strategies (perception) of dynamic human behaviour are constrained by changing ecological (especially informational) task constraints; (ii) provided evidence for the importance of representative design in experiments on perception and action; and iii), provided a principled theoretical framework to guide learning design in acquisition of skill in interceptive actions like cricket batting.

Degrees of freedom and motor planning in purposive movement

This paper presents empirical evidence suggesting that healthy humans can perform a two degree of freedom visuo-motor pursuit tracking task with the same response time delay as a one degree of freedom task. In contrast, the time delay of the response is influenced markedly by the nature of the motor synergy required to produce it. We suggest a conceptual account of this evidence based on adaptive model theory, which combines theories of intermittency from psychology and adaptive optimal control from engineering. The intermittent response planning stage has a fixed period. It possesses multiple optimal trajectory generators such that multiple degrees of freedom can be planned concurrently, without requiring an increase in the planning period. In tasks which require unfamiliar motor synergies, or are deemed to be incompatible, internal adaptive models representing movement dynamics are inaccurate. This means that the actual response which is produced will deviate from the one which is planned. For a given target-response discrepancy, corrective response trajectories of longer duration are planned, consistent with the principle of speed-accuracy trade-off. Compared to familiar or compatible tasks, this results in a longer response time delay and reduced accuracy. From the standpoint of the intermittency approach, the findings of this study help make possible a more integral and predictive account of purposive action. (c) 2005 Elsevier B.V. All rights reserved.

Boundary Tracking using a Hybrid Cyclic Pursuit Scheme

There has been a lot of work in the literature, related to the mapping of boundaries of regions, using multiple agents. Most of these are based on optimization techniques or rely on potential fields to drive the agents towards the boundary and then retain them there while they space out evenly along the perimeter or surface (in two-dimensional and three-dimensional cases, respectively). In this paper an algorithm to track the boundary of a region in space is provided based on the cyclic pursuit scheme. This enables the agents to constantly move along the perimeter in a cluster, thereby tracking a dynamically changing boundary. The trajectories of the agents provide a sketch of the boundary. The use of multiple agents may facilitate minimization of tracking error by providing accurate estimates of points on the boundary, besides providing redundancy. Simulation results are provided to highlight the performance of the proposed scheme.

Neural Dynamics of Saccadic and Smooth Pursuit Eye Movement Coordination during Visual Tracking of Unpredictably Moving Targets

How does the brain use eye movements to track objects that move in unpredictable directions and speeds? Saccadic eye movements rapidly foveate peripheral visual or auditory targets and smooth pursuit eye movements keep the fovea pointed toward an attended moving target. Analyses of tracking data in monkeys and humans reveal systematic deviations from predictions of the simplest model of saccade-pursuit interactions, which would use no interactions other than common target selection and recruitment of shared motoneurons. Instead, saccadic and smooth pursuit movements cooperate to cancel errors of gaze position and velocity, and thus to maximize target visibility through time. How are these two systems coordinated to promote visual localization and identification of moving targets? How are saccades calibrated to correctly foveate a target despite its continued motion during the saccade? A neural model proposes answers to such questions. The modeled interactions encompass motion processing areas MT, MST, FPA, DLPN and NRTP; saccade planning and execution areas FEF and SC; the saccadic generator in the brain stem; and the cerebellum. Simulations illustrate the model’s ability to functionally explain and quantitatively simulate anatomical, neurophysiological and behavioral data about SAC-SPEM tracking.

Target Selection by Frontal Cortex During Coordinated Saccadic and Smooth Pursuit Eye Movement

Oculomotor tracking of moving objects is an important component of visually based cognition and planning. Such tracking is achieved by a combination of saccades and smooth pursuit eye movements. In particular, the saccadic and smooth pursuit systems interact to often choose the same target, and to maximize its visibility through time. How do multiple brain regions interact, including frontal cortical areas, to decide the choice of a target among several competing moving stimuli? How is target selection information that is created by a bias (e.g., electrical stimulation) transferred from one movement system to another? These saccade-pursuit interactions are clarified by a new computational neural model, which describes interactions among motion processing areas MT, MST, FPA, DLPN; saccade specification, selection, and planning areas LIP, FEF, SNr, SC; the saccadic generator in the brain stem; and the cerebellum. Model simulations explain a broad range of neuroanatomical and neurophysiological data. These results are in contrast with the simplest parallel model with no interactions between saccades and pursuit than common-target selection and recruitment of shared motoneurons. Actual tracking episodes in primates reveal multiple systematic deviations from predictions of the simplest parallel model, which are explained by the current model.

Video target tracking by using competitive neural networks

A target tracking algorithm able to identify the position and to pursuit moving targets in video digital sequences is proposed in this paper. The proposed approach aims to track moving targets inside the vision field of a digital camera. The position and trajectory of the target are identified by using a neural network presenting competitive learning technique. The winning neuron is trained to approximate to the target and, then, pursuit it. A digital camera provides a sequence of images and the algorithm process those frames in real time tracking the moving target. The algorithm is performed both with black and white and multi-colored images to simulate real world situations. Results show the effectiveness of the proposed algorithm, since the neurons tracked the moving targets even if there is no pre-processing image analysis. Single and multiple moving targets are followed in real time.

Impairments of peripheral motion change detection during smooth pursuit eye movements

Introduction: In team sports the ability to use peripheral vision is essential to track a number of players and the ball. By using eye-tracking devices it was found that players either use fixations and saccades to process information on the pitch or use smooth pursuit eye movements (SPEM) to keep track of single objects (Schütz, Braun, & Gegenfurtner, 2011). However, it is assumed that peripheral vision can be used best when the gaze is stable while it is unknown whether motion changes can be equally well detected when SPEM are used especially because contrast sensitivity is reduced during SPEM (Schütz, Delipetkose, Braun, Kerzel, & Gegenfurtner, 2007). Therefore, peripheral motion change detection will be examined by contrasting a fixation condition with a SPEM condition. Methods: 13 participants (7 male, 6 female) were presented with a visual display consisting of 15 white and 1 red square. Participants were instructed to follow the red square with their eyes and press a button as soon as a white square begins to move. White square movements occurred either when the red square was still (fixation condition) or moving in a circular manner with 6 °/s (pursuit condition). The to-be-detected white square movements varied in eccentricity (4 °, 8 °, 16 °) and speed (1 °/s, 2 °/s, 4 °/s) while movement time of white squares was constant at 500 ms. 180 events should be detected in total. A Vicon-integrated eye-tracking system and a button press (1000 Hz) was used to control for eye-movements and measure detection rates and response times. Response times (ms) and missed detections (%) were measured as dependent variables and analysed with a 2 (manipulation) x 3 (eccentricity) x 3 (speed) ANOVA with repeated measures on all factors. Results: Significant response time effects were found for manipulation, F(1,12) = 224.31, p < .01, ηp2 = .95, eccentricity, F(2,24) = 56.43; p < .01, ηp2 = .83, and the interaction between the two factors, F(2,24) = 64.43; p < .01, ηp2 = .84. Response times increased as a function of eccentricity for SPEM only and were overall higher than in the fixation condition. Results further showed missed events effects for manipulation, F(1,12) = 37.14; p < .01, ηp2 = .76, eccentricity, F(2,24) = 44.90; p < .01, ηp2 = .79, the interaction between the two factors, F(2,24) = 39.52; p < .01, ηp2 = .77 and the three-way interaction manipulation x eccentricity x speed, F(2,24) = 3.01; p = .03, ηp2 = .20. While less than 2% of events were missed on average in the fixation condition as well as at 4° and 8° eccentricity in the SPEM condition, missed events increased for SPEM at 16 ° eccentricity with significantly more missed events in the 4 °/s speed condition (1 °/s: M = 34.69, SD = 20.52; 2 °/s: M = 33.34, SD = 19.40; 4 °/s: M = 39.67, SD = 19.40). Discussion: It could be shown that using SPEM impairs the ability to detect peripheral motion changes at the far periphery and that fixations not only help to detect these motion changes but also to respond faster. Due to high temporal constraints especially in team sports like soccer or basketball, fast reaction are necessary for successful anticipation and decision making. Thus, it is advised to anchor gaze at a specific location if peripheral changes (e.g. movements of other players) that require a motor response have to be detected. In contrast, SPEM should only be used if a single object, like the ball in cricket or baseball, is necessary for a successful motor response. References: Schütz, A. C., Braun, D. I., & Gegenfurtner, K. R. (2011). Eye movements and perception: A selective review. Journal of Vision, 11, 1-30. Schütz, A. C., Delipetkose, E., Braun, D. I., Kerzel, D., & Gegenfurtner, K. R. (2007). Temporal contrast sensitivity during smooth pursuit eye movements. Journal of Vision, 7, 1-15.

Tracking people in 3D using position, size and shape

This paper presents a prototype tracking system for tracking people in enclosed indoor environments where there is a high rate of occlusions. The system uses a stereo camera for acquisition, and is capable of disambiguating occlusions using a combination of depth map analysis, a two step ellipse fitting people detection process, the use of motion models and Kalman filters and a novel fit metric, based on computationally simple object statistics. Testing shows that our fit metric outperforms commonly used position based metrics and histogram based metrics, resulting in more accurate tracking of people.