Integrated vision-based attitude estimation

For many years, computer vision has lured researchers with promises of a low-cost, passive, lightweight and information-rich sensor suitable for navigation purposes. The prime difficulty in vision-based navigation is that the navigation solution will continually drift with time unless external information is available, whether it be cues from the appearance of the scene, a map of features (whether built online or known a priori), or from an externally-referenced sensor. It is not merely position that is of interest in the navigation problem. Attitude (i.e. the angular orientation of a body with respect to a reference frame) is integral to a visionbased navigation solution and is often of interest in its own right (e.g. flight control). This thesis examines vision-based attitude estimation in an aerospace environment, and two methods are proposed for constraining drift in the attitude solution; one through a novel integration of optical flow and the detection of the sky horizon, and the other through a loosely-coupled integration of Visual Odometry and GPS position measurements. In the first method, roll angle, pitch angle and the three aircraft body rates are recovered though a novel method of tracking the horizon over time and integrating the horizonderived attitude information with optical flow. An image processing front-end is used to select several candidate lines in a image that may or may not correspond to the true horizon, and the optical flow is calculated for each candidate line. Using an Extended Kalman Filter (EKF), the previously estimated aircraft state is propagated using a motion model and a candidate horizon line is associated using a statistical test based on the optical flow measurements and location of the horizon in the image. Once associated, the selected horizon line, along with the associated optical flow, is used as a measurement to the EKF. To evaluate the accuracy of the algorithm, two flights were conducted, one using a highly dynamic Uninhabited Airborne Vehicle (UAV) in clear flight conditions and the other in a human-piloted Cessna 172 in conditions where the horizon was partially obscured by terrain, haze and smoke. The UAV flight resulted in pitch and roll error standard deviations of 0.42° and 0.71° respectively when compared with a truth attitude source. The Cessna 172 flight resulted in pitch and roll error standard deviations of 1.79° and 1.75° respectively. In the second method for estimating attitude, a novel integrated GPS/Visual Odometry (GPS/VO) navigation filter is proposed, using a structure similar to a classic looselycoupled GPS/INS error-state navigation filter. Under such an arrangement, the error dynamics of the system are derived and a Kalman Filter is developed for estimating the errors in position and attitude. Through similar analysis to the GPS/INS problem, it is shown that the proposed filter is capable of recovering the complete attitude (i.e. pitch, roll and yaw) of the platform when subjected to acceleration not parallel to velocity for both the monocular and stereo variants of the filter. Furthermore, it is shown that under general straight line motion (e.g. constant velocity), only the component of attitude in the direction of motion is unobservable. Numerical simulations are performed to demonstrate the observability properties of the GPS/VO filter in both the monocular and stereo camera configurations. Furthermore, the proposed filter is tested on imagery collected using a Cessna 172 to demonstrate the observability properties on real-world data. The proposed GPS/VO filter does not require additional restrictions or assumptions such as platform-specific dynamics, map-matching, feature-tracking, visual loop-closing, gravity vector or additional sensors such as an IMU or magnetic compass. Since no platformspecific dynamics are required, the proposed filter is not limited to the aerospace domain and has the potential to be deployed in other platforms such as ground robots or mobile phones.

Relationship among CFH and ARMS2 genotypes, macular pigment optical density, and neuroretinal function in persons without age-related macular degeneration

Purpose: To determine whether there is a difference in neuroretinal function and in macular pigment optical density between persons with high- and low-risk gene variants for age-related macular degeneration (AMD) and no ophthalmoscopic signs of AMD, and to compare the results on neuroretinal function to patients with manifest early AMD. Methods and Participants: Neuroretinal function was assessed with the multifocal electroretinogram (mfERG) for 32 participants (22 healthy persons with no AMD and 10 early AMD patients). The 22 healthy participants with no AMD had high- or low-risk genotypes for either CFH (rs380390) and/or ARMS2 (rs10490924). Trough-to-peak response densities and peak-implicit times were analyzed in 5 concentric rings. Macular pigment optical densitometry was assessed by customized heterochromatic flicker photometry. Results: Trough-to-peak response densities for concentric rings 1 to 3 were, on average, significantly greater in participants with high-risk genotypes than in participants with low-risk genotypes and in persons with early AMD after correction for age and smoking (p<0.05). The group peak- implicit times for ring 1 were, on average, delayed in the patients with early AMD compared with the participants with high- or low-risk genotypes, although these differences were not significant. There was no significant correlation between genotypes and macular pigment optical density. Conclusion: Increased neuroretinal activity in persons who carry high-risk AMD genotypes may be due to genetically determined subclinical inflammatory and/or histological changes in the retina. Neuroretinal function in healthy persons genetically susceptible to AMD may be a useful additional early biomarker (in combination with genetics) before there is clinical manifestation.

Frame-rate stereopsis using non-parametric transforms and programmable logic

A frame-rate stereo vision system, based on non-parametric matching metrics, is described. Traditional metrics, such as normalized cross-correlation, are expensive in terms of logic. Non-parametric measures require only simple, parallelizable, functions such as comparators, counters and exclusive-or, and are thus very well suited to implementation in reprogrammable logic.

Estimating the probability of an invalid match for matching using the rank transform

A fundamental problem faced by stereo matching algorithms is the matching or correspondence problem. A wide range of algorithms have been proposed for the correspondence problem. For all matching algorithms, it would be useful to be able to compute a measure of the probability of correctness, or reliability of a match. This paper focuses in particular on one class for matching algorithms, which are based on the rank transform. The interest in these algorithms for stereo matching stems from their invariance to radiometric distortion, and their amenability to fast hardware implementation. This work differs from previous work in that it derives, from first principles, an expression for the probability of a correct match. This method was based on an enumeration of all possible symbols for matching. The theoretical results for disparity error prediction, obtained using this method, were found to agree well with experimental results. However, disadvantages of the technique developed in this chapter are that it is not easily applicable to real images, and also that it is too computationally expensive for practical window sizes. Nevertheless, the exercise provides an interesting and novel analysis of match reliability.

Visual sea-floor mapping from low overlap imagery using bi-objective bundle adjustment and constrained motion

In most visual mapping applications suited to Autonomous Underwater Vehicles (AUVs), stereo visual odometry (VO) is rarely utilised as a pose estimator as imagery is typically of very low framerate due to energy conservation and data storage requirements. This adversely affects the robustness of a vision-based pose estimator and its ability to generate a smooth trajectory. This paper presents a novel VO pipeline for low-overlap imagery from an AUV that utilises constrained motion and integrates magnetometer data in a bi-objective bundle adjustment stage to achieve low-drift pose estimates over large trajectories. We analyse the performance of a standard stereo VO algorithm and compare the results to the modified vo algorithm. Results are demonstrated in a virtual environment in addition to low-overlap imagery gathered from an AUV. The modified VO algorithm shows significantly improved pose accuracy and performance over trajectories of more than 300m. In addition, dense 3D meshes generated from the visual odometry pipeline are presented as a qualitative output of the solution.

Sensing and control for a small-size helicopter

This paper details the progress to date, toward developing a small autonomous helicopter. We describe system architecture, avionics, visual state estimation, custom IMU design, aircraft modelling, as well as various linear and neuro/fuzzy control algorithms. Experimental results are presented for state estimation using fused stereo vision and IMU data, heading control, and attitude control. FAM attitude and velocity controllers have been shown to be effective in simulation.

A framework for a dense matching algorithm : Results using matching metrics and order statistic filters

An algorithm for computing dense correspondences between images of a stereo pair or image sequence is presented. The algorithm can make use of both standard matching metrics and the rank and census filters, two filters based on order statistics which have been applied to the image matching problem. Their advantages include robustness to radiometric distortion and amenability to hardware implementation. Results obtained using both real stereo pairs and a synthetic stereo pair with ground truth were compared. The rank and census filters were shown to significantly improve performance in the case of radiometric distortion. In all cases, the results obtained were comparable to, if not better than, those obtained using standard matching metrics. Furthermore, the rank and census have the additional advantage that their computational overhead is less than these metrics. For all techniques tested, the difference between the results obtained for the synthetic stereo pair, and the ground truth results was small.

Order statistic filters for image matching

The rank and census are two filters based on order statistics which have been applied to the image matching problem for stereo pairs. Advantages of these filters include their robustness to radiometric distortion and small amounts of random noise, and their amenability to hardware implementation. In this paper, a new matching algorithm is presented, which provides an overall framework for matching, and is used to compare the rank and census techniques with standard matching metrics. The algorithm was tested using both real stereo pairs and a synthetic pair with ground truth. The rank and census filters were shown to significantly improve performance in the case of radiometric distortion. In all cases, the results obtained were comparable to, if not better than, those obtained using standard matching metrics. Furthermore, the rank and census have the additional advantage that their computational overhead is less than these metrics. For all techniques tested, the difference between the results obtained for the synthetic stereo pair, and the ground truth results was small.

Nonparametric semantic segmentation for 3D street scenes

In this paper we propose a method to generate a large scale and accurate dense 3D semantic map of street scenes. A dense 3D semantic model of the environment can significantly improve a number of robotic applications such as autonomous driving, navigation or localisation. Instead of using offline trained classifiers for semantic segmentation, our approach employs a data-driven, nonparametric method to parse scenes which easily scale to a large environment and generalise to different scenes. We use stereo image pairs collected from cameras mounted on a moving car to produce dense depth maps which are combined into a global 3D reconstruction using camera poses from stereo visual odometry. Simultaneously, 2D automatic semantic segmentation using a nonparametric scene parsing method is fused into the 3D model. Furthermore, the resultant 3D semantic model is improved with the consideration of moving objects in the scene. We demonstrate our method on the publicly available KITTI dataset and evaluate the performance against manually generated ground truth.

The Norfolk Island Eye Study (NIES) : rationale, methodology and distribution of ocular biometry (Biometry of the Bounty)

Aim: To describe the recruitment, ophthalmic examination methods and distribution of ocular biometry of participants in the Norfolk Island Eye Study, who were individuals descended from the English Bounty mutineers and their Polynesian wives. Methods: All 1,275 permanent residents of Norfolk Island aged over 15 years were invited to participate, including 602 individuals involved in a 2001 cardiovascular disease study. Participants completed a detailed questionnaire and underwent a comprehensive eye assessment including stereo disc and retinal photography, ocular coherence topography and conjunctival autofluorescence assessment. Additionally, blood or saliva was taken for DNA testing. Results: 781 participants aged over 15 years were seen (54% female), comprising 61% of the permanent Island population. 343 people (43.9%) could trace their family history to the Pitcairn Islanders (Norfolk Island Pitcairn Pedigree). Mean anterior chamber depth was 3.32mm, mean axial length (AL) was 23.5mm, and mean central corneal thickness was 546 microns. There were no statistically significant differences in these characteristics between persons with and without Pitcairn Island ancestry. Mean intra-ocular pressure was lower in people with Pitcairn Island ancestry: 15.89mmHg compared to those without Pitcairn Island ancestry 16.49mmHg (P = .007). The mean keratometry value was lower in people with Pitcairn Island ancestry (43.22 vs. 43.52, P = .007). The corneas were flatter in people of Pitcairn ancestry but there was no corresponding difference in AL or refraction. Conclusion: Our study population is highly representative of the permanent population of Norfolk Island. Ocular biometry was similar to that of other white populations. Heritability estimates, linkage analysis and genome-wide studies will further elucidate the genetic determinants of chronic ocular diseases in this genetic isolate.

Error modeling and calibration of exteroceptive sensors for accurate mapping applications

Reliable robotic perception and planning are critical to performing autonomous actions in uncertain, unstructured environments. In field robotic systems, automation is achieved by interpreting exteroceptive sensor information to infer something about the world. This is then mapped to provide a consistent spatial context, so that actions can be planned around the predicted future interaction of the robot and the world. The whole system is as reliable as the weakest link in this chain. In this paper, the term mapping is used broadly to describe the transformation of range-based exteroceptive sensor data (such as LIDAR or stereo vision) to a fixed navigation frame, so that it can be used to form an internal representation of the environment. The coordinate transformation from the sensor frame to the navigation frame is analyzed to produce a spatial error model that captures the dominant geometric and temporal sources of mapping error. This allows the mapping accuracy to be calculated at run time. A generic extrinsic calibration method for exteroceptive range-based sensors is then presented to determine the sensor location and orientation. This allows systematic errors in individual sensors to be minimized, and when multiple sensors are used, it minimizes the systematic contradiction between them to enable reliable multisensor data fusion. The mathematical derivations at the core of this model are not particularly novel or complicated, but the rigorous analysis and application to field robotics seems to be largely absent from the literature to date. The techniques in this paper are simple to implement, and they offer a significant improvement to the accuracy, precision, and integrity of mapped information. Consequently, they should be employed whenever maps are formed from range-based exteroceptive sensor data. © 2009 Wiley Periodicals, Inc.

Novelty-based visual obstacle detection in agriculture

This paper describes a novel obstacle detection system for autonomous robots in agricultural field environments that uses a novelty detector to inform stereo matching. Stereo vision alone erroneously detects obstacles in environments with ambiguous appearance and ground plane such as in broad-acre crop fields with harvested crop residue. The novelty detector estimates the probability density in image descriptor space and incorporates image-space positional understanding to identify potential regions for obstacle detection using dense stereo matching. The results demonstrate that the system is able to detect obstacles typical to a farm at day and night. This system was successfully used as the sole means of obstacle detection for an autonomous robot performing a long term two hour coverage task travelling 8.5 km.

The relationship between BCOM1 gene variants and macular pigment optical density in persons with and without age-related macular degeneration

Background: Recent evidence indicates that gene variants related to carotenoid metabolism play a role in the uptake of macular pigments lutein (L) and zeaxanthine (Z). Moreover, these pigments are proposed to reduce the risk for advanced age-related macular degeneration (AMD). This study provides the initial examination of the relationship between the gene variants related to carotenoid metabolism, macular pigment optical density (MPOD) and their combined expression in healthy humans and patients with AMD. Participants and Methods: Forty-four participants were enrolled from a general population and a private practice including 20 healthy participants and 24 patients with advanced (neovascular) AMD. Participants were genotyped for the three single nucleotide polymorphisms (SNPs) upstream from BCMO1, rs11645428, rs6420424 and rs6564851 that have been shown to either up or down regulate beta-carotene conversion efficiency in the plasma. MPOD was determined by heterochromatic flicker photometry. Results: Healthy participants with the rs11645428 GG genotype, rs6420424 AA genotype and rs6564851 GG genotype all had on average significantly lower MPOD compared to those with the other genotypes (p < 0.01 for all three comparisons). When combining BCMO1 genotypes reported to have “high” (rs11645428 AA/rs6420424 GG/rs6564851 TT) and “low” (rs11645428 GG/rs6420424 AA/rs6564851 GG) beta-carotene conversion efficiency, we demonstrate clear differences in MPOD values (p<0.01). In patients with AMD there were no significant differences in MPOD for any of the three BCMO1 gene variants. Conclusion: In healthy participants MPOD levels can be related to high and low beta-carotene conversion BCMO1 genotypes. Such relationships were not found in patients with advanced neovascular AMD, indicative of additional processes influencing carotenoid uptake, possibly related to other AMD susceptibility genes. Our findings indicate that specific BCMO1 SNPs should be determined when assessing the effects of carotenoid supplementation on macular pigment and that their expression may be influenced by retinal disease.

Gene electrotransfer into murine skeletal muscle: A systematic analysis of parameters for long-term gene expression

Skeletal muscle is an attractive target tissue for delivery of therapeutic genes, since it is well vascularized, easily accessible, and has a high capacity for protein synthesis. For efficient transfection in skeletal muscle, several protocols have been described, including delivery of low voltage electric pulses and a combination of high and low voltage electric pulses. The aim of this study was to determine the influence of different parameters of electrotransfection on short-term and long-term transfection efficiency in murine skeletal muscle, and to evaluate histological changes in the treated tissue. Different parameters of electric pulses, different time lags between plasmid DNA injection and application of electric pulses, and different doses of plasmid DNA were tested for electrotransfection of tibialis cranialis muscle of C57BI/6 mice using DNA plasmid encoding green fluorescent protein (GFP). Transfection efficiency was assessed on frozen tissue sections one week after electrotransfection using a fluorescence microscope and also noninvasively, followed by an in vivo imaging system using a fluorescence stereo microscope over a period of several months. Histological changes in muscle were evaluated immediately or several months after electrotransfection by determining infiltration of inflammatory mononuclear cells and presence of necrotic muscle fibers. The most efficient electrotransfection into skeletal muscle of C57BI/6 mice in our experiments was achieved when one high voltage (HV) and four low voltage (LV) electric pulses were applied 5 seconds after the injection of 30 μg of plasmid DNA. This protocol resulted in the highest short-term as well as long-term transfection. The fluorescence intensity of the transfected area declined after 2-3 weeks, but GFP fluorescence was still detectable 18 months after electrotransfection. Extensive inflammatory mononuclear cell infiltration was observed immediately after the electrotransfection procedure using the described parameters, but no necrosis or late tissue damage was observed. This study showed that electric pulse parameters, time lag between the injection of DNA and application of electric pulses, and dose of plasmid DNA affected the duration of transgene expression in murine skeletal muscle. Therefore, transgene expression in muscle can be controlled by appropriate selection of electrotransfection protocol.

Very high resolution optical satellites for DEM generation : a review

The advent of very high resolution (VHR) optical satellites capable of producing stereo images led to a new era in extracting digital elevation model which commenced with the launch of IKONOS. The special specifications of VHR optical satellites besides, the significant economic profit stimulated other countries and companies to have their constellations such as EROS-A1 and EROS-B1 as the cooperation between Israel and ImageSat. QuickBird, WorldView-1 and WorldVew-2 were launched by DigitalGlobe. ALOS and GeoEye-1 were offered by Japan and GeoEye Respectively. In addition to aforementioned satellites, Indian and South Korea initiated their own constellation by launching CartoSat-1 and KOPOSAT-2 respectively.The availability of all so-called satellites make a huge market of stereo images for extracting of digital elevation model and other correspondent applications such as, producing orthorectifcatin images and updating maps. Therefore, there is a need for a comprehensive comparison for scientific and commercial clients to choose appropriate satellite images and methods of generating digital elevation model to obtain optimum results. This paper will thus give a review about the specifications of VHR optical satellites. Then it will discuss the automatic elaborating of digital elevation model. Finally an overview of studies and corresponding results is reported.