Airborne vision-based collision-detection system

Machine vision represents a particularly attractive solution for sensing and detecting potential collision-course targets due to the relatively low cost, size, weight, and power requirements of vision sensors (as opposed to radar and TCAS). This paper describes the development and evaluation of a real-time vision-based collision detection system suitable for fixed-wing aerial robotics. Using two fixed-wing UAVs to recreate various collision-course scenarios, we were able to capture highly realistic vision (from an onboard camera perspective) of the moments leading up to a collision. This type of image data is extremely scarce and was invaluable in evaluating the detection performance of two candidate target detection approaches. Based on the collected data, our detection approaches were able to detect targets at distances ranging from 400m to about 900m. These distances (with some assumptions about closing speeds and aircraft trajectories) translate to an advanced warning of between 8-10 seconds ahead of impact, which approaches the 12.5 second response time recommended for human pilots. We overcame the challenge of achieving real-time computational speeds by exploiting the parallel processing architectures of graphics processing units found on commercially-off-the-shelf graphics devices. Our chosen GPU device suitable for integration onto UAV platforms can be expected to handle real-time processing of 1024 by 768 pixel image frames at a rate of approximately 30Hz. Flight trials using manned Cessna aircraft where all processing is performed onboard will be conducted in the near future, followed by further experiments with fully autonomous UAV platforms.

Single camera vision-only SLAM on a suburban road network

Simultaneous Localization And Mapping (SLAM) is one of the major challenges in mobile robotics. Probabilistic techniques using high-end range finding devices are well established in the field, but recent work has investigated vision only approaches. This paper presents a method for generating approximate rotational and translation velocity information from a single vehicle-mounted consumer camera, without the computationally expensive process of tracking landmarks. The method is tested by employing it to provide the odometric and visual information for the RatSLAM system while mapping a complex suburban road network. RatSLAM generates a coherent map of the environment during an 18 km long trip through suburban traffic at speeds of up to 60 km/hr. This result demonstrates the potential of ground based vision-only SLAM using low cost sensing and computational hardware.

Cyclist visibility at night : perceptions of visibility do not necessarily match reality

Visibility limitations make cycling at night particularly dangerous. We previously reported cyclists’ perceptions of their own visibility at night and identified clothing configurations that made them feel visible. In this study we sought to determine whether these self-perceptions reflect actual visibility when wearing these clothing configurations. In a closed-road driving environment, cyclists wore black clothing, a fluorescent vest, a reflective vest, or a reflective vest plus ankle and knee reflectors. Drivers recognised more cyclists wearing the reflective vest plus reflectors (90%) than the reflective vest alone (50%), fluorescent vest (15%) or black clothing (2%). Older drivers recognised the cyclists less often than younger drivers (51% vs 27%). The findings suggest that reflective ankle and knee markings are particularly valuable at night, while fluorescent clothing is not. Cyclists wearing fluorescent clothing may be at particular risk if they incorrectly believe themselves to be conspicuous to drivers at night.

Using reflective clothing to enhance the conspicuity of bicyclists at night

Bicycling at night is more dangerous than in the daytime and poor conspicuity is likely to be a contributing factor. The use of reflective markings on a pedestrian’s major joints to facilitate the perception of biological motion has been shown to greatly enhance pedestrian conspicuity at night, but few corresponding data exist for bicyclists. Twelve younger and twelve older participants drove around a closed-road circuit at night and indicated when they first saw a bicyclist who wore black clothing either alone, or together with a reflective bicycling vest, or a vest plus ankle and knee reflectors. The bicyclist pedaled in place on a bicycle that had either a static or flashing light, or no light on the handlebars. Bicyclist clothing significantly affected conspicuity; drivers responded to bicyclists wearing the vest plus ankle and knee reflectors at significantly longer distances than when the bicyclist wore the vest alone or black clothing without a vest. Older drivers responded to bicyclists less often and at shorter distances than younger drivers. The presence of a bicycle light, whether static or flashing, did not enhance the conspicuity of the bicyclist; this may result in bicyclists who use a bicycle light being overconfident of their own conspicuity at night. The implications of our findings are that ankle and knee markings are a simple and very effective approach for enhancing bicyclist conspicuity at night.

Even moderate visual impairments degrade drivers’ ability to see pedestrians at night

Purpose: To determine the effect of moderate levels of refractive blur and simulated cataracts on nighttime pedestrian conspicuity in the presence and absence of headlamp glare. Methods: The ability to recognize pedestrians at night was measured in 28 young adults (M=27.6 years) under three visual conditions: normal vision, refractive blur and simulated cataracts; mean acuity was 20/40 or better in all conditions. Pedestrian recognition distances were recorded while participants drove an instrumented vehicle along a closed road course at night. Pedestrians wore one of three clothing conditions and oncoming headlamps were present for 16 participants and absent for 12 participants. Results: Simulated visual impairment and glare significantly reduced the frequency with which drivers recognized pedestrians and the distance at which the drivers first recognized them. Simulated cataracts were significantly more disruptive than blur even though photopic visual acuity levels were matched. With normal vision, drivers responded to pedestrians at 3.6x and 5.5x longer distances on average than for the blur or cataract conditions, respectively. Even in the presence of visual impairment and glare, pedestrians were recognized more often and at longer distances when they wore a “biological motion” reflective clothing configuration than when they wore a reflective vest or black clothing. Conclusions: Drivers’ ability to recognize pedestrians at night is degraded by common visual impairments even when the drivers’ mean visual acuity meets licensing requirements. To maximize drivers’ ability to see pedestrians, drivers should wear their optimum optical correction, and cataract surgery should be performed early enough to avoid potentially dangerous reductions in visual performance.

Towards robust night and day place recognition using visible and thermal imaging

The chief challenge facing persistent robotic navigation using vision sensors is the recognition of previously visited locations under different lighting and illumination conditions. The majority of successful approaches to outdoor robot navigation use active sensors such as LIDAR, but the associated weight and power draw of these systems makes them unsuitable for widespread deployment on mobile robots. In this paper we investigate methods to combine representations for visible and long-wave infrared (LWIR) thermal images with time information to combat the time-of-day-based limitations of each sensing modality. We calculate appearance-based match likelihoods using the state-of-the-art FAB-MAP [1] algorithm to analyse loop closure detection reliability across different times of day. We present preliminary results on a dataset of 10 successive traverses of a combined urban-parkland environment, recorded in 2-hour intervals from before dawn to after dusk. Improved location recognition throughout an entire day is demonstrated using the combined system compared with methods which use visible or thermal sensing alone.

Research initiatives to improve the visibility and hence safety of road workers at night-time

Research has demonstrated the benefits that clothing incorporating retroreflective markers can provide in significantly improving visibility and reducing accidents, especially at night. Adding biomotion markings to standard vests can enhance the night-time conspicuity of roadway workers by capitalizing on perceptual capabilities.

Handheld 3D thermography using range sensing and computer vision

This thesis developed a method for real-time and handheld 3D temperature mapping using a combination of off-the-shelf devices and efficient computer algorithms. It contributes a new sensing and data processing framework to the science of 3D thermography, unlocking its potential for application areas such as building energy auditing and industrial monitoring. New techniques for the precise calibration of multi-sensor configurations were developed, along with several algorithms that ensure both accurate and comprehensive surface temperature estimates can be made for rich 3D models as they are generated by a non-expert user.

Vision based guidance for robot navigation in agriculture

This paper describes a novel vision based texture tracking method to guide autonomous vehicles in agricultural fields where the crop rows are challenging to detect. Existing methods require sufficient visual difference between the crop and soil for segmentation, or explicit knowledge of the structure of the crop rows. This method works by extracting and tracking the direction and lateral offset of the dominant parallel texture in a simulated overhead view of the scene and hence abstracts away crop-specific details such as colour, spacing and periodicity. The results demonstrate that the method is able to track crop rows across fields with extremely varied appearance during day and night. We demonstrate this method can autonomously guide a robot along the crop rows.

Vision-based simultaneous localization and mapping in changing outdoor environments

For robots operating in outdoor environments, a number of factors, including weather, time of day, rough terrain, high speeds, and hardware limitations, make performing vision-based simultaneous localization and mapping with current techniques infeasible due to factors such as image blur and/or underexposure, especially on smaller platforms and low-cost hardware. In this paper, we present novel visual place-recognition and odometry techniques that address the challenges posed by low lighting, perceptual change, and low-cost cameras. Our primary contribution is a novel two-step algorithm that combines fast low-resolution whole image matching with a higher-resolution patch-verification step, as well as image saliency methods that simultaneously improve performance and decrease computing time. The algorithms are demonstrated using consumer cameras mounted on a small vehicle in a mixed urban and vegetated environment and a car traversing highway and suburban streets, at different times of day and night and in various weather conditions. The algorithms achieve reliable mapping over the course of a day, both when incrementally incorporating new visual scenes from different times of day into an existing map, and when using a static map comprising visual scenes captured at only one point in time. Using the two-step place-recognition process, we demonstrate for the first time single-image, error-free place recognition at recall rates above 50% across a day-night dataset without prior training or utilization of image sequences. This place-recognition performance enables topologically correct mapping across day-night cycles.

Differential effects of refractive blur on day and night-time driving performance

Purpose To investigate the effect of different levels of refractive blur on real-world driving performance measured under day and nighttime conditions. Methods Participants included 12 visually normal, young adults (mean age = 25.8 ± 5.2 years) who drove an instrumented research vehicle around a 4 km closed road circuit with three different levels of binocular spherical refractive blur (+0.50 diopter sphere [DS], +1.00 DS, +2.00 DS) compared with a baseline condition. The subjects wore optimal spherocylinder correction and the additional blur lenses were mounted in modified full-field goggles; the order of testing of the blur conditions was randomized. Driving performance was assessed in two different sessions under day and nighttime conditions and included measures of road signs recognized, hazard detection and avoidance, gap detection, lane-keeping, sign recognition distance, speed, and time to complete the course. Results Refractive blur and time of day had significant effects on driving performance (P < 0.05), where increasing blur and nighttime driving reduced performance on all driving tasks except gap judgment and lane keeping. There was also a significant interaction between blur and time of day (P < 0.05), such that the effects of blur were exacerbated under nighttime driving conditions; performance differences were evident even for +0.50 DS blur relative to baseline for some measures. Conclusions The effects of blur were greatest under nighttime conditions, even for levels of binocular refractive blur as low as +0.50 DS. These results emphasize the importance of accurate and up-to-date refractive correction of even low levels of refractive error when driving at night.

Towards Vision-Based Pose- and Condition-Invariant Place Recognition along Routes

Vision-based place recognition involves recognising familiar places despite changes in environmental conditions or camera viewpoint (pose). Existing training-free methods exhibit excellent invariance to either of these challenges, but not both simultaneously. In this paper, we present a technique for condition-invariant place recognition across large lateral platform pose variance for vehicles or robots travelling along routes. Our approach combines sideways facing cameras with a new multi-scale image comparison technique that generates synthetic views for input into the condition-invariant Sequence Matching Across Route Traversals (SMART) algorithm. We evaluate the system’s performance on multi-lane roads in two different environments across day-night cycles. In the extreme case of day-night place recognition across the entire width of a four-lane-plus-median-strip highway, we demonstrate performance of up to 44% recall at 100% precision, where current state-of-the-art fails.

Kiwi Forego Vision in the Guidance of Their Nocturnal Activities

Background In vision, there is a trade-off between sensitivity and resolution, and any eye which maximises information gain at low light levels needs to be large. This imposes exacting constraints upon vision in nocturnal flying birds. Eyes are essentially heavy, fluid-filled chambers, and in flying birds their increased size is countered by selection for both reduced body mass and the distribution of mass towards the body core. Freed from these mass constraints, it would be predicted that in flightless birds nocturnality should favour the evolution of large eyes and reliance upon visual cues for the guidance of activity. Methodology/Principal Findings We show that in Kiwi (Apterygidae), flightlessness and nocturnality have, in fact, resulted in the opposite outcome. Kiwi show minimal reliance upon vision indicated by eye structure, visual field topography, and brain structures, and increased reliance upon tactile and olfactory information. Conclusions/Significance This lack of reliance upon vision and increased reliance upon tactile and olfactory information in Kiwi is markedly similar to the situation in nocturnal mammals that exploit the forest floor. That Kiwi and mammals evolved to exploit these habitats quite independently provides evidence for convergent evolution in their sensory capacities that are tuned to a common set of perceptual challenges found in forest floor habitats at night and which cannot be met by the vertebrate visual system. We propose that the Kiwi visual system has undergone adaptive regressive evolution driven by the trade-off between the relatively low rate of gain of visual information that is possible at low light levels, and the metabolic costs of extracting that information.

Effects of different levels of refractive blur on night-time pedestrian visibility

Purpose The aim of this study was to systematically investigate the effect of different levels of refractive blur and driver age on night-time pedestrian recognition and determine whether clothing that has been shown to improve pedestrian conspicuity is robust to the effects of blur. Methods Night-time pedestrian recognition was measured for 24 visually normal participants (12 younger M=24.9±4.5 years and 12 older adults M=77.6±5.7 years) for three levels of binocular blur (+0.50 D, +1.00 D, +2.00 D) compared to baseline (optimal refractive correction). Pedestrians walked in place on a closed road circuit and wore one of three clothing conditions: i) everyday clothing, ii) a retro-reflective vest and iii) retro-reflective tape positioned on the extremities in a configuration that conveyed biological motion (known as “biomotion”); the order of conditions was randomized between participants. Pedestrian recognition distances were recorded for each blur and pedestrian clothing combination while participants drove an instrumented vehicle around a closed road course. Results The recognition distances for pedestrians were significantly reduced (p<0.05) by all levels of blur compared to baseline. Pedestrians wearing “biomotion” clothing were recognized at significantly longer distances than for the other clothing configurations in all blur conditions. However, these effects were smaller for the older adults, who had much shorter recognition distances for all conditions tested. Conclusions In summary, even small amounts of blur had a significant detrimental effect on night-time pedestrian recognition. Biomotion retro-reflective clothing was effective, even under moderately degraded visibility conditions, for both young and older drivers.