Target Detection and Tracking With Heterogeneous Sensors

We present a multimodal detection and tracking algorithm for sensors composed of a camera mounted between two microphones. Target localization is performed on color-based change detection in the video modality and on time difference of arrival (TDOA) estimation between the two microphones in the audio modality. The TDOA is computed by multiband generalized cross correlation (GCC) analysis. The estimated directions of arrival are then postprocessed using a Riccati Kalman filter. The visual and audio estimates are finally integrated, at the likelihood level, into a particle filter (PF) that uses a zero-order motion model, and a weighted probabilistic data association (WPDA) scheme. We demonstrate that the Kalman filtering (KF) improves the accuracy of the audio source localization and that the WPDA helps to enhance the tracking performance of sensor fusion in reverberant scenarios. The combination of multiband GCC, KF, and WPDA within the particle filtering framework improves the performance of the algorithm in noisy scenarios. We also show how the proposed audiovisual tracker summarizes the observed scene by generating metadata that can be transmitted to other network nodes instead of transmitting the raw images and can be used for very low bit rate communication. Moreover, the generated metadata can also be used to detect and monitor events of interest.

Hand-related rather than goal-related source of gaze-dependent errors in memory-guided reaching

Mechanisms for visuospatial cognition are often inferred directly from errors in behavioral reports of remembered target direction. For example, gaze-centered target representations for reach were first inferred from reach overshoots of target location relative to gaze. Here, we report evidence for the hypothesis that these gaze-dependent reach errors stem predominantly from misestimates of hand rather than target position, as was assumed in all previous studies. Subjects showed typical gaze-dependent overshoots in complete darkness, but these errors were entirely suppressed by continuous visual feedback of the finger. This manipulation could not affect target representations, so the suppressed gaze-dependent errors must have come from misestimates of hand position, likely arising in a gaze-dependent transformation of hand position signals into visual coordinates. This finding has broad implications for any task involving localization of visual targets relative to unseen limbs, in both healthy individuals and patient populations, and shows that response-related transformations cannot be ignored when deducing the sources of gaze-related errors.