A semi-local method for iterative depth-map refinement

We present an iterative hierarchical algorithm for multi-view stereo. The algorithm attempts to utilise as much contextual information as is available to compute highly accurate and robust depth maps. There are three novel aspects to the approach: 1) firstly we incrementally improve the depth fidelity as the algorithm progresses through the image pyramid; 2) secondly we show how to incorporate visual hull information (when available) to constrain depth searches; and 3) we show how to simultaneously enforce the consistency of the depth-map by continual comparison with neighbouring depth-maps. We show that this approach produces highly accurate depth-maps and, since it is essentially a local method, is both extremely fast and simple to implement.

Frame-rate stereopsis using non-parametric transforms and programmable logic

A frame-rate stereo vision system, based on non-parametric matching metrics, is described. Traditional metrics, such as normalized cross-correlation, are expensive in terms of logic. Non-parametric measures require only simple, parallelizable, functions such as comparators, counters and exclusive-or, and are thus very well suited to implementation in reprogrammable logic.

Sensing and control for a small-size helicopter

This paper details the progress to date, toward developing a small autonomous helicopter. We describe system architecture, avionics, visual state estimation, custom IMU design, aircraft modelling, as well as various linear and neuro/fuzzy control algorithms. Experimental results are presented for state estimation using fused stereo vision and IMU data, heading control, and attitude control. FAM attitude and velocity controllers have been shown to be effective in simulation.

Error modeling and calibration of exteroceptive sensors for accurate mapping applications

Reliable robotic perception and planning are critical to performing autonomous actions in uncertain, unstructured environments. In field robotic systems, automation is achieved by interpreting exteroceptive sensor information to infer something about the world. This is then mapped to provide a consistent spatial context, so that actions can be planned around the predicted future interaction of the robot and the world. The whole system is as reliable as the weakest link in this chain. In this paper, the term mapping is used broadly to describe the transformation of range-based exteroceptive sensor data (such as LIDAR or stereo vision) to a fixed navigation frame, so that it can be used to form an internal representation of the environment. The coordinate transformation from the sensor frame to the navigation frame is analyzed to produce a spatial error model that captures the dominant geometric and temporal sources of mapping error. This allows the mapping accuracy to be calculated at run time. A generic extrinsic calibration method for exteroceptive range-based sensors is then presented to determine the sensor location and orientation. This allows systematic errors in individual sensors to be minimized, and when multiple sensors are used, it minimizes the systematic contradiction between them to enable reliable multisensor data fusion. The mathematical derivations at the core of this model are not particularly novel or complicated, but the rigorous analysis and application to field robotics seems to be largely absent from the literature to date. The techniques in this paper are simple to implement, and they offer a significant improvement to the accuracy, precision, and integrity of mapped information. Consequently, they should be employed whenever maps are formed from range-based exteroceptive sensor data. © 2009 Wiley Periodicals, Inc.

Novelty-based visual obstacle detection in agriculture

This paper describes a novel obstacle detection system for autonomous robots in agricultural field environments that uses a novelty detector to inform stereo matching. Stereo vision alone erroneously detects obstacles in environments with ambiguous appearance and ground plane such as in broad-acre crop fields with harvested crop residue. The novelty detector estimates the probability density in image descriptor space and incorporates image-space positional understanding to identify potential regions for obstacle detection using dense stereo matching. The results demonstrate that the system is able to detect obstacles typical to a farm at day and night. This system was successfully used as the sole means of obstacle detection for an autonomous robot performing a long term two hour coverage task travelling 8.5 km.

Autonomous helicopter hover using an artificial neural network

Details the developments to date of an unmanned air vehicle (UAV) based on a standard size 60 model helicopter. The design goal is to have the helicopter achieve stable hover with the aid of an INS and stereo vision. The focus of the paper is on the development of an artificial neural network (ANN) that makes use of only the INS data to generate hover commands, which are used to directly manipulate the flight servos. Current results show that networks incorporating some form of recurrency (state history) offer little advantage over those without. At this stage, the ANN has partially maintained periods of hover even with misaligned sensors.

Constrained 3D sketching with haptics and active motion annotation

The aim of this work is to enable seamless transformation of product concepts to CAD models. This necessitates availability of 3D product sketches. The present work concerns intuitive generation of 3D strokes and intrinsic support for space sharing and articulation for the components of the product being sketched. Direct creation of 3D strokes in air lacks in precision, stability and control. The inadequacy of proprioceptive feedback for the task is complimented in this work with stereo vision and haptics. Three novel methods based on pencil-paper interaction analogy for haptic rendering of strokes have been investigated. The pen-tilt based rendering is simpler and found to be more effective. For the spatial conformity, two modes of constraints for the stylus movements, corresponding to the motions on a control surface and in a control volume have been studied using novel reactive and field based haptic rendering schemes. The field based haptics, which in effect creates an attractive force field near a surface, though non-realistic, provided highly effective support for the control-surface constraints. The efficacy of the reactive haptic rendering scheme for the constrained environments has been demonstrated using scribble strokes. This can enable distributed collaborative 3D concept development. The notion of motion constraints, defined through sketch strokes enables intuitive generation of articulated 3D sketches and direct exploration of motion annotations found in most product concepts. The work, thus, establishes that modeling of the constraints is a central issue in 3D sketching.

Automated sparse 3D point cloud generation of infrastructure using its distinctive visual features

The commercial far-range (>10 m) spatial data collection methods for acquiring infrastructure’s geometric data are not completely automated because of the necessary manual pre- and/or post-processing work. The required amount of human intervention and, in some cases, the high equipment costs associated with these methods impede their adoption by the majority of infrastructure mapping activities. This paper presents an automated stereo vision-based method, as an alternative and inexpensive solution, to producing a sparse Euclidean 3D point cloud of an infrastructure scene utilizing two video streams captured by a set of two calibrated cameras. In this process SURF features are automatically detected and matched between each pair of stereo video frames. 3D coordinates of the matched feature points are then calculated via triangulation. The detected SURF features in two successive video frames are automatically matched and the RANSAC algorithm is used to discard mismatches. The quaternion motion estimation method is then used along with bundle adjustment optimization to register successive point clouds. The method was tested on a database of infrastructure stereo video streams. The validity and statistical significance of the results were evaluated by comparing the spatial distance of randomly selected feature points with their corresponding tape measurements.

Comparison of Civil Infrastructure Optical-based Spatial Data Acquisition Techniques

Infrastructure spatial data, such as the orientation and the location of in place structures and these structures' boundaries and areas, play a very important role for many civil infrastructure development and rehabilitation applications, such as defect detection, site planning, on-site safety assistance and others. In order to acquire these data, a number of modern optical-based spatial data acquisition techniques can be used. These techniques are based on stereo vision, optics, time of flight, etc., and have distinct characteristics, benefits and limitations. The main purpose of this paper is to compare these infrastructure optical-based spatial data acquisition techniques based on civil infrastructure application requirements. In order to achieve this goal, the benefits and limitations of these techniques were identified. Subsequently, these techniques were compared according to applications' requirements, such as spatial accuracy, the automation of acquisition, the portability of devices and others. With the help of this comparison, unique characteristics of these techniques were identified so that practitioners will be able to select an appropriate technique for their own applications.

Comparison of Civil Infrastructure Optical-based Spatial Data Acquisition Techniques

Infrastructure spatial data, such as the orientation and the location of in place structures and these structures' boundaries and areas, play a very important role for many civil infrastructure development and rehabilitation applications, such as defect detection, site planning, on-site safety assistance and others. In order to acquire these data, a number of modern optical-based spatial data acquisition techniques can be used. These techniques are based on stereo vision, optics, time of flight, etc., and have distinct characteristics, benefits and limitations. The main purpose of this paper is to compare these infrastructure optical-based spatial data acquisition techniques based on civil infrastructure application requirements. In order to achieve this goal, the benefits and limitations of these techniques were identified. Subsequently, these techniques were compared according to applications' requirements, such as spatial accuracy, the automation of acquisition, the portability of devices and others. With the help of this comparison, unique characteristics of these techniques were identified so that practitioners will be able to select an appropriate technique for their own applications.

单相机全向立体视觉研究

折反射全向成像系统是由普通透视相机和反射镜面组成的全向成像装置，可实时获取360°无需拼接的全景图像，近年来已成为研究热点并在视频会议、三维重建和移动机器人导航等领域有着广泛的应用。 本文主要对单相机全向立体视觉系统的设计、标定、匹配以及三维重建展开研究。介绍了一种可实时获取全向三维信息的折反射全向立体视觉光学装置OSVOD（Omnidirectional Stereo Vision Optical Device），OSVOD由两个双曲面镜和一个普通透视相机组成。其中两个双曲镜面上下同轴、间隔一定距离固定在一个玻璃筒内，下镜面中间开有一孔，上镜面通过下镜面的孔在相机像平面上成像，这样空间一点经上下反射镜的反射在像平面上有两个像点，用一个相机实现了立体视觉。两镜面的共同轴和相机镜头的光轴共线，共同焦点和镜头的光心重合，该配置能保证系统满足单一视点约束SVP（Single View-Point）。本结构配置也使系统的外极线呈一系列的放射线，对应点匹配简单。此外两镜面的间隔安装也使得系统的等效基线较长，从而具有较高的精度。 本文第一部分对当前的各种全向成像方法进行了简单介绍，并对各方法的特点做了归纳。第二部分介绍折反射全向视觉的研究现状，就各种反射镜面的成像特点做了对比。 第三部分介绍OSVOD的设计方法，包括机构的设计和镜面的设计，并对设计的结果做了误差分析。 第四部分是OSVOD的标定研究。给出了一种包括OSVOD中相机和镜面位置关系在内的系统参数的标定方法。该方法利用空间坐标已知的标定点在像平面上成的像，结合系统成像模型反算出标定点的空间坐标，再利用标定点的已知空间坐标和反算出的空间坐标建立方程，运用基于Levenberg-Marquardt的反向传播算法(backpropagation)标定相机与反射镜面间的安装偏差。该标定方法可推广到所有的折反射成像系统。 第五部分是基于全向图像的匹配研究。针对系统获取的立体图像对之间成像比例存在较大的差异，首先将图像展开成柱面投影图像，然后就下镜面成像展开的柱面做Canny边缘检测，得到了图像的边缘点；就得到的边缘点在展开的两幅柱面图像上做直接相关匹配。最后将获取的匹配点做一致性校验，并对一致性校验通过的匹配点做三维计算，生产稀疏的三维图像。 最后是结论和将来的工作展望。

基于单相机的全向立体视觉关键技术研究

获取全向三维信息对移动机器人导航和行动规划具有重要意义。尽管有许多其他方法可以完成这一任务，如超声传感器和激光测距仪，但是折反射立体视觉系统在大多数情况下可以获得更高的精度和更大的视场，并且不消耗额外的能量。本文采用了一种新型的折反射全向立体视觉光学装置(OSVOD)进行立体视觉的研究，OSVOD是由两个双曲面镜和一个透视相机所组成的系统，能够由单幅图像实现立体视觉。 本文重点对单相机全向立体视觉系统的标定、单幅图像匹配、运动估计和多目匹配这三项最关键的技术进行了研究。在系统标定方面，给出了一种包括OSVOD中相机和镜面位置关系在内的系统参数的标定方法。该方法利用空间坐标已知的标定点在像平面上成的像，结合系统成像模型反算出标定点的空间坐标，再利用标定点的已知空间坐标和反算出的空间坐标建立方程，运用基于Levenberg-Marquardt的反向传播算法标定相机与反射镜面间的安装偏差。该标定方法可推广到所有的折反射成像系统。 在单幅图像匹配方面，针对系统获取的立体图像对之间成像比例存在较大的差异和畸变的问题，将图像展开成柱面投影图像和俯视投影图像，并提出了一种三步算法，首先匹配非歧义的点，从而将匹配划分为小的独立的子问题，并且每条极线只匹配到最远的特征点，从而避免在远端的不可靠匹配。在随后的动态规划算法中，设计了一个特定的能量函数，对不同的纹理强度和置信程度分别加权，得到了可靠的全向致密深度图。 在运动估计与多目匹配方面，以Harris角点做为待匹配的特征，由相关性得到初匹配结果。该结果中不可避免地存在误匹配，采用随机采样一致算法来得到运动估计。完成运动估计后，利用不同位置得到的图像进行了基于边缘检测的多目匹配，快速准确地获取了障碍物信息。

立体视觉中误匹配滤波方法的研究

从二维空间和三维空间2种角度研究误匹配滤波算法,提出在匹配前用于降低误匹配的灰度预处理算法和一种基于真实控制点的视差滤波算法。前者只针对2幅图像的重叠区域进行灰度均衡,可以减少计算量,后者在传统视差均值滤波的基础上可进一步提高误匹配的滤波效率。基于真实图像的实验结果表明,新算法可以有效滤除误匹配,提高三维重建精度,保证重建效果。

基于立体视觉的移动机器人楼梯识别

本文提出了一种结构化环境下,基于立体视觉的机器人楼梯识别算法,并将算法该应到自主移动机器人上。该算法首先利用二维图像分析的方法搜索楼梯的疑似区域;进而利用立体视觉对各个疑似区域进行精确三维重建,结合三维信息重构楼梯平面,排除虚假疑似楼梯区域;最后判定机器人和楼梯的相对位姿关系,引导机器人爬楼梯。最终我们将该算法应用到了自主移动机器人上,通过在各种光照条件下的实验,进一步验证了该算法的准确性和快速性。

基于视觉伺服的输电线巡检机器人抓线控制

巡检机器人在越障时,需要完成机器人手臂的准确抓线控制.结合输电线的几何特征和摄像机成像原理,提出了一种基于单摄像机的立体视觉方法来确定输电线的位置和姿态.基于该定位方法及视觉伺服理论,建立机械手抓线伺服控制模型.利用自行研制的巡检机器人进行了视觉伺服抓线实验;实验结果验证了该方法的有效性.