An active stereo vision-based learning approach for robotic tracking, fixating and grasping control

In this paper, an active stereo vision-based learning approach is proposed for a robot to track, fixate and grasp an object in unknown environments. First, the functional mapping relationships between the joint angles of the active stereo vision system and the spatial representations of the object are derived and expressed in a three-dimensional workspace frame. Next, the self-adaptive resonance theory-based neural networks and the feedforward neural networks are used to learn the mapping relationships in a self-organized way. Then, the approach is verified by simulation using the models of an active stereo vision system which is installed in the end-effector of a robot. Finally, the simulation results confirm the effectiveness of the present approach.

A method for stereo-vision-based tracking for robotic applications

Vision-based tracking of an object using perspective projection inherently results in non-linear measurement equations in the Cartesian coordinates. The underlying object kinematics can be modelled by a linear system. In this paper we introduce a measurement conversion technique that analytically transforms the non-linear measurement equations obtained from a stereo-vision system into a system of linear measurement equations.We then design a robust linear filter around the converted measurement system. The state estimation error of the proposed filter is bounded and we provide a rigorous theoretical analysis of this result. The performance of the robust filter developed in this paper is demonstrated via computer simulation and via practical experimentation using a robotic manipulator as a target. The proposed filter is shown to outperform the extended Kalman filter (EKF).

A method for stereo-vision based tracking for robotic applications

Vision based tracking of an object using the ideas of perspective projection inherently consists of nonlinearly modelled measurements although the underlying dynamic system that encompasses the object and the vision sensors can be linear. Based on a necessary stereo vision setting, we introduce an appropriate measurement conversion techniques which subsequently facilitate using a linear filter. Linear filter together with the aforementioned measurement conversion approach conforms a robust linear filter that is based on the set values state estimation ideas; a particularly rich area in the robust control literature. We provide a rigorously theoretical analysis to ensure bounded state estimation errors formulated in terms of an ellipsoidal set in which the actual state is guaranteed to be included to an arbitrary high probability. Using computer simulations as well as a practical implementation consisting of a robotic manipulator, we demonstrate our linear robust filter significantly outperforms the traditionally used extended Kalman filter under this stereo vision scenario. © 2008 IEEE.

Fast vision-based catheter 3D reconstruction

Continuum robots offer better maneuverability and inherent compliance and are well-suited for surgical applications as catheters, where gentle interaction with the environment is desired. However, sensing their shape and tip position is a challenge as traditional sensors can not be employed in the way they are in rigid robotic manipulators. In this paper, a high speed vision-based shape sensing algorithm for real-time 3D reconstruction of continuum robots based on the views of two arbitrary positioned cameras is presented. The algorithm is based on the closed-form analytical solution of the reconstruction of quadratic curves in 3D space from two arbitrary perspective projections. High-speed image processing algorithms are developed for the segmentation and feature extraction from the images. The proposed algorithms are experimentally validated for accuracy by measuring the tip position, length and bending and orientation angles for known circular and elliptical catheter shaped tubes. Sensitivity analysis is also carried out to evaluate the robustness of the algorithm. Experimental results demonstrate good accuracy (maximum errors of  ±0.6 mm and  ±0.5 deg), performance (200 Hz), and robustness (maximum absolute error of 1.74 mm, 3.64 deg for the added noises) of the proposed high speed algorithms.

Machine vision system for automatic inspection of surface defects in aluminum die casting

A camera based machine vision system for the automatic inspection of surface defects in aluminum die casting is presented. The system uses a hybrid image processing algorithm based on mathematic morphology to detect defects with different sizes and shapes. The defect inspection algorithm consists of two parts. One is a parameter learning algorithm, in which a genetic algorithm is used to extract optimal structuring element parameters, and segmentation and noise removal thresholds. The second part is a defect detection algorithm, in which the parameters obtained by a genetic algorithm are used for morphological operations. The machine vision system has been applied in an industrial setting to detect two types of casting defects: parts mix-up and any defects on the surface of castings. The system performs with a 99% or higher accuracy for both part mix-up and defect detection and is currently used in industry as part of normal production.

Multi-agent model for robotic assembly system

In this paper, a multi-agent based model for a robotic assembly system is presented. Firstly, an organization model is used to construct the multi-agent model. Secondly, a dynamic self-organizing method is then put forward for the multi-agent robotic system to bid and contract the operations. Thirdly, a real multi-agent robotic system is built and assembly experiments are carried out. Finally, the experimental results confirm that the present multi-agent robotic system has flexibility, adaptation and stability.

Mcommerce - a vision in time

Global mCommerce revenues are expected to grow from $400 million in 2000, to 22.2 billion by 2005 (Raczkowski, 2002) mCommerce is a key driver in developing the global information society, with applications emerging in numerous areas including banking, financial services, security services and shopping (Khalifa and Cheng, 2002). With optimistic projections regarding its growth, many researchers are actively determining the future of mCommerce. In this paper, we have tried to synthesise some predictions and evolving definitions, explored security as a critical impediment and developing solutions; investigated the mPayment scenario; and derived a futuristic research framework. Specifically, we have attempted to capture the moving mCommerce scenario in present time, with a special focus into mPayments.

Machine vision system for automatic inspection of surface defects in aluminium die casting

A machine vision system is presented for the automatic inspection of surface defects in aluminium die casting. The system uses a hybrid image processing algorithm based on mathematic morphology to detect defects with different sizes and shapes. The defect inspection algorithm consists of two parts. One is a parameter learning algorithm, in which a genetic algorithm is used to extract optimal structuring element parameters, and segmentation and noise removal thresholds. The second part is a defect detection algorithm, in which the parameters obtained by a genetic algorithm are used for morphological operations. The machine vision system has been applied in an industrial setting to detect two types of casting defects: parts mix-up and any defects on the surface of castings. The system performs with a 99% or higher accuracy for both part mix-up and defect detection and is currently used in industry as part of normal production.