An active stereo vision-based learning approach for robotic tracking, fixating and grasping control

In this paper, an active stereo vision-based learning approach is proposed for a robot to track, fixate and grasp an object in unknown environments. First, the functional mapping relationships between the joint angles of the active stereo vision system and the spatial representations of the object are derived and expressed in a three-dimensional workspace frame. Next, the self-adaptive resonance theory-based neural networks and the feedforward neural networks are used to learn the mapping relationships in a self-organized way. Then, the approach is verified by simulation using the models of an active stereo vision system which is installed in the end-effector of a robot. Finally, the simulation results confirm the effectiveness of the present approach.

A multi-wavelet based technique for calculating dense 2D disparity maps from stereo

A vision based approach for calculating accurate 3D models of the objects is presented. Generally industrial visual inspection systems capable of accurate 3D depth estimation rely on extra hardware tools like laser scanners or light pattern projectors. These tools improve the accuracy of depth estimation but also make the vision system costly and cumbersome. In the proposed algorithm, depth and dimensional accuracy of the produced 3D depth model depends on the existing reference model instead of the information from extra hardware tools. The proposed algorithm is a simple and cost effective software based approach to achieve accurate 3D depth estimation with minimal hardware involvement. The matching process uses the well-known coarse to fine strategy, involving the calculation of matching points at the coarsest level with consequent refinement up to the finest level. Vector coefficients of the wavelet transform-modulus are used as matching features, where wavelet transform-modulus maxima defines the shift invariant high-level features with phase pointing to the normal of the feature surface. The technique addresses the estimation of optimal corresponding points and the corresponding 2D disparity maps leading to the creation of accurate depth perception model.

Stereo-vision-based moving object tracking via robust linear filtering

Vision-based tracking sensors typically provide nonlinear measurements
of the targets Cartesian position and velocity state components. In this paper we derive linear measurements using an analytical measurement conversion technique which can be used with two (or more) vision sensors. We derive
linear measurements in the target’s Cartesian position and velocity components and we derive a robust version of a linear Kalman filter. We show that our linear robust filter significantly outperforms the extended Kalman Filter. Moreover, we prove that the state estimation error is bounded.

Depth estimation using multi-wavelet analysis based stereo vision approach

The problem of dimensional defects in aluminum die- casting is widespread throughout the foundry industry and their detection is of paramount importance in maintaining product quality. Due to the unpredictable factory environment and metallic, with highly reflective, nature of aluminum die-castings, it is extremely hard to estimate true dimensionality of the die-casting, autonomously. In this work, we propose a novel robust 3D reconstruction algorithm capable of reconstructing dimensionally accurate 3D depth models of the aluminum die-castings. The developed system is very simple and cost effective as it consists of only a stereo cameras pair and a simple fluorescent light. The developed system is capable of estimating surface depths within the tolerance of 1.5 mm. Moreover, the system is invariant to illuminative variations and orientation of the objects in the input image space, which makes the developed system highly robust. Due to its hardware simplicity and robustness, it can be implemented in different factory environments without a significant change in the setup.

Hyperstereopsis in helmet-mounted NVDs : time to contact estimation

The side mounting of the night-vision sensors on some helmet-mounted systems creates a situation of hyperstereopsis in which the binocular cues available to the operator are exaggerated such that distances around the point of fixation are increased. For a moving surface approaching the observer, the increased apparent distance created by hyperstereopsis should result in greater apparent speed of approach towards the surface and so an operator will have the impression they have reached the surface before contact actually occurs. We simulated motion towards a surface with hyperstereopsis and compared judgements of time to contact with that under normal stereopsis as well as under binocular viewing without stereopsis. We simulated approach of a large, random-field textured and found that time to contact estimates were shorter under the hyperstereoscopic condition than those under normal stereo and no stereo, indicating that hyperstereopsis may cause observers to underestimate time to contact leading operators to undershoot the ground plane when landing.

Hyperstereopsis in helmet-mounted NVDs : slope perception

Modern helmet-mounted night vision devices, such as the Thales TopOwl helmet, project imagery from intensifiers mounted on the sides of the helmet onto the helmet faceplate. This produces a situation of hyperstereopsis in which binocular disparities are magnified. This has the potential to distort the perception of slope in depth (an important cue to landing), because the slope cue provided by binocular disparity conflicts with veridical cues to slope, such as texture gradients and motion parallax. In the experiments, eight observers viewed sparse and dense textured surfaces tilted in depth under three viewing conditions: normal stereo hyper-stereo (4 times magnification), and hypostereo (1 / 4 magnification). The surfaces were either stationary, or rotated slowly around a central vertical axis. Stimuli were projected at 6 metres to minimise conflict between accommodation and convergence, and stereo viewing was provided by a Z-screen and passive polarised glasses. Observers matched perceived visual slope using a small tilt table set by hand. We found that slope estimates were distorted by hyperstereopsis, but to a much lesser degree than predicted by disparity magnification. The distortion was almost completely eliminated when motion parallax was present.

Stereo correspondence estimation using multiwavelets scale-space representation-based multiresolution analysis

A multiresolution technique based on multiwavelets scale-space representation for stereo correspondence estimation is presented. The technique uses the well-known coarse-to-fine strategy, involving the calculation of stereo correspondences at the coarsest resolution level with consequent refinement up to the finest level. Vector coefficients of the multiwavelets transform modulus are used as corresponding features, where modulus maxima defines the shift invariant high-level features (multiscale edges) with phase pointing to the normal of the feature surface. The technique addresses the estimation of optimal corresponding points and the corresponding 2D disparity maps. Illuminative variation that can exist between the perspective views of the same scene is controlled using scale normalization at each decomposition level by dividing the details space coefficients with approximation space. The problems of ambiguity, explicitly, and occlusion, implicitly, are addressed by using a geometric topological refinement procedure. Geometric refinement is based on a symbolic tagging procedure introduced to keep only the most consistent matches in consideration. Symbolic tagging is performed based on probability of occurrence and multiple thresholds. The whole procedure is constrained by the uniqueness and continuity of the corresponding stereo features. The comparative performance of the proposed algorithm with eight famous existing algorithms, presented in the literature, is shown to validate the claims of promising performance of the proposed algorithm.

Depth estimation using multiwavelet analysis based stereo vision approach

The problem of dimensional defects in aluminum die-casting is widespread throughout the foundry industry and their detection is of paramount importance in maintaining product quality. Due to the unpredictable factory environment and metallic, with highly reflective, nature of aluminum die-castings, it is extremely hard to estimate true dimensionality of the die-casting, autonomously. In this work, we propose a novel robust 3D reconstruction algorithm capable of reconstructing dimensionally accurate 3D depth models of the aluminum die-castings. The developed system is very simple and cost effective as it consists of only a stereo camera pair and a simple fluorescent light. The developed system is capable of estimating surface depths within the tolerance of 1.5 mm. Moreover, the system is invariant to illuminative variations and orientation of the objects in the input image space, which makes the developed system highly robust. Due to its hardware simplicity and robustness, it can be implemented in different factory environments without a significant change in the setup.

Stereo image matching using wavelet scale-space representation

A multi-resolution technique for matching a stereo pair of images based on translation invariant discrete multi-wavelet transform is presented. The technique uses the well known coarse to fine strategy, involving the calculation of matching points at the coarsest level with consequent refinement up to the finest level. Vector coefficients of the wavelet transform modulus are used as matching features, where modulus maxima defines the shift invariant high-level features (multiscale edges) with phase pointing to the normal of the feature surface. The technique addresses the estimation of optimal corresponding points and the corresponding 2D disparity maps. Illuminative variation that can exist between the perspective views of the same scene is controlled using scale normalization at each decomposition level by dividing the details space coefficients with approximation space and then using normalized correlation. The problem of ambiguity, explicitly, and occlusion, implicitly, is addressed by using a geometric topological refinement procedure and symbolic tagging.

3D depth estimation using wavelet analysis based stereo vision approach

A complete and highly robust 3D reconstruction algorithm based on stereo vision is presented. The developed system is capable of reconstructing dimensionally accurate 3D models of the objects and is very simple and cost effective due to its prominent software dependency and minimal hardware involvevment unlike existing systems.