Towards Vision-Based Pose- and Condition-Invariant Place Recognition along Routes

Vision-based place recognition involves recognising familiar places despite changes in environmental conditions or camera viewpoint (pose). Existing training-free methods exhibit excellent invariance to either of these challenges, but not both simultaneously. In this paper, we present a technique for condition-invariant place recognition across large lateral platform pose variance for vehicles or robots travelling along routes. Our approach combines sideways facing cameras with a new multi-scale image comparison technique that generates synthetic views for input into the condition-invariant Sequence Matching Across Route Traversals (SMART) algorithm. We evaluate the system’s performance on multi-lane roads in two different environments across day-night cycles. In the extreme case of day-night place recognition across the entire width of a four-lane-plus-median-strip highway, we demonstrate performance of up to 44% recall at 100% precision, where current state-of-the-art fails.

Unsupervised online learning of condition-invariant images for place recognition

This paper presents an online, unsupervised training algorithm enabling vision-based place recognition across a wide range of changing environmental conditions such as those caused by weather, seasons, and day-night cycles. The technique applies principal component analysis to distinguish between aspects of a location’s appearance that are condition-dependent and those that are condition-invariant. Removing the dimensions associated with environmental conditions produces condition-invariant images that can be used by appearance-based place recognition methods. This approach has a unique benefit – it requires training images from only one type of environmental condition, unlike existing data-driven methods that require training images with labelled frame correspondences from two or more environmental conditions. The method is applied to two benchmark variable condition datasets. Performance is equivalent or superior to the current state of the art despite the lesser training requirements, and is demonstrated to generalise to previously unseen locations.

Understanding welding practices on shipyards : an ethnographic study for designing an interactive robot welder

The aim of this ethnographic study was to understand welding practices in shipyard environments with the purpose of designing an interactive welding robot that can help workers with their daily job. The robot is meant to be deployed for automatic welding on jack-up rig structures. The design of the robot turns out to be a challenging task due to several problematic working conditions on the shipyard, such as dust, irregular floor, high temperature, wind variations, elevated working platforms, narrow spaces, and circular welding paths requiring a robotic arm with more than 6 degrees of freedom. Additionally, the environment is very noisy and the workers – mostly foreigners – have a very basic level of English. These two issues need to be taken into account when designing the interactive user interface for the robot. Ideally, the communication flow between the two parties involved should be as frictionless as possible. The paper presents the results of our field observations and welders’ interviews, as well as our robot design recommendation for the next project stage.

Convolutional Neural Network-based place recognition

Recently Convolutional Neural Networks (CNNs) have been shown to achieve state-of-the-art performance on various classification tasks. In this paper, we present for the first time a place recognition technique based on CNN models, by combining the powerful features learnt by CNNs with a spatial and sequential filter. Applying the system to a 70 km benchmark place recognition dataset we achieve a 75% increase in recall at 100% precision, significantly outperforming all previous state of the art techniques. We also conduct a comprehensive performance comparison of the utility of features from all 21 layers for place recognition, both for the benchmark dataset and for a second dataset with more significant viewpoint changes.

Towards coordinated multirobot missions for lunar sample collection in an unknown environment

In this paper, the recent results of the space project IMPERA are presented. The goal of IMPERA is the development of a multirobot planning and plan execution architecture with a focus on a lunar sample collection scenario in an unknown environment. We describe the implementation and verification of different modules that are integrated into a distributed system architecture. The modules include a mission planning approach for a multirobot system and modules for task and skill execution within a lunar use-case scenario. The skills needed for the test scenario include cooperative exploration and mapping strategies for an unknown environment, the localization and classification of sample containers using a novel approach of semantic perception, and the skill of transporting sample containers to a collection point using a mobile manipulation robot. Additionally, we present our approach of a reliable communication framework that can deal with communication loss during the mission. Several modules are tested within several experiments in the domain of planning and plan execution, communication, coordinated exploration, perception, and object transportation. An overall system integration is tested on a mission scenario experiment using three robots.

Practical path planning and obstacle avoidance for autonomous mowing

There is a need for systems which can autonomously perform coverage tasks on large outdoor areas. Unfortunately, the state-of-the-art is to use GPS based localization, which is not suitable for precise operations near trees and other obstructions. In this paper we present a robotic platform for autonomous coverage tasks. The system architecture integrates laser based localization and mapping using the Atlas Framework with Rapidly-Exploring Random Trees path planning and Virtual Force Field obstacle avoidance. We demonstrate the performance of the system in simulation as well as with real world experiments.

Automated sensory data alignment for environmental and epidermal change monitoring

In this paper we present research adapting a state of the art condition-invariant robotic place recognition algorithm to the role of automated inter- and intra-image alignment of sensor observations of environmental and skin change over time. The approach involves inverting the typical criteria placed upon navigation algorithms in robotics; we exploit rather than attempt to fix the limited camera viewpoint invariance of such algorithms, showing that approximate viewpoint repetition is realistic in a wide range of environments and medical applications. We demonstrate the algorithms automatically aligning challenging visual data from a range of real-world applications: ecological monitoring of environmental change, aerial observation of natural disasters including flooding, tsunamis and bushfires and tracking wound recovery and sun damage over time and present a prototype active guidance system for enforcing viewpoint repetition. We hope to provide an interesting case study for how traditional research criteria in robotics can be inverted to provide useful outcomes in applied situations.

Development and integration of a solar powered unmanned aerial vehicle and a wireless sensor network to monitor greenhouse gases

Measuring gases for environmental monitoring is a demanding task that requires long periods of observation and large numbers of sensors. Wireless Sensor Networks (WSNs) and Unmanned Aerial Vehicles (UAVs) currently represent the best alternative to monitor large, remote, and difficult access areas, as these technologies have the possibility of carrying specialized gas sensing systems. This paper presents the development and integration of a WSN and an UAV powered by solar energy in order to enhance their functionality and broader their applications. A gas sensing system implementing nanostructured metal oxide (MOX) and non-dispersive infrared sensors was developed to measure concentrations of CH4 and CO2. Laboratory, bench and field testing results demonstrate the capability of UAV to capture, analyze and geo-locate a gas sample during flight operations. The field testing integrated ground sensor nodes and the UAV to measure CO2 concentration at ground and low aerial altitudes, simultaneously. Data collected during the mission was transmitted in real time to a central node for analysis and 3D mapping of the target gas. The results highlights the accomplishment of the first flight mission of a solar powered UAV equipped with a CO2 sensing system integrated with a WSN. The system provides an effective 3D monitoring and can be used in a wide range of environmental applications such as agriculture, bushfires, mining studies, zoology and botanical studies using a ubiquitous low cost technology.

On the use of optical flow for scene change detection and description

We propose the use of optical flow information as a method for detecting and describing changes in the environment, from the perspective of a mobile camera. We analyze the characteristics of the optical flow signal and demonstrate how robust flow vectors can be generated and used for the detection of depth discontinuities and appearance changes at key locations. To successfully achieve this task, a full discussion on camera positioning, distortion compensation, noise filtering, and parameter estimation is presented. We then extract statistical attributes from the flow signal to describe the location of the scene changes. We also employ clustering and dominant shape of vectors to increase the descriptiveness. Once a database of nodes (where a node is a detected scene change) and their corresponding flow features is created, matching can be performed whenever nodes are encountered, such that topological localization can be achieved. We retrieve the most likely node according to the Mahalanobis and Chi-square distances between the current frame and the database. The results illustrate the applicability of the technique for detecting and describing scene changes in diverse lighting conditions, considering indoor and outdoor environments and different robot platforms.

A method to quantify a descriptor's illumination variance

This paper presents a new metric, which we call the lighting variance ratio, for quantifying descriptors in terms of their variance to illumination changes. In many applications it is desirable to have descriptors that are robust to changes in illumination, especially in outdoor environments. The lighting variance ratio is useful for comparing descriptors and determining if a descriptor is lighting invariant enough for a given environment. The metric is analysed across a number of datasets, cameras and descriptors. The results show that the upright SIFT descriptor is typically the most lighting invariant descriptor.

Tracking control of a class of Hamiltonian mechanical systems with disturbances

This paper presents a trajectory-tracking control strategy for a class of mechanical systems in Hamiltonian form. The class is characterised by a simplectic interconnection arising from the use of generalised coordinates and full actuation. The tracking error dynamic is modelled as a port-Hamiltonian Systems (PHS). The control action is designed to take the error dynamics into a desired closed-loop PHS characterised by a constant mass matrix and a potential energy with a minimum at the origin. A transformation of the momentum and a feedback control is exploited to obtain a constant generalised mass matrix in closed loop. The stability of the close-loop system is shown using the close-loop Hamiltonian as a Lyapunov function. The paper also considers the addition of integral action to design a robust controller that ensures tracking in spite of disturbances. As a case study, the proposed control design methodology is applied to a fully actuated robotic manipulator.

Herds of methane chambers grazing bubbles

Water to air methane emissions from freshwater reservoirs can be dominated by sediment bubbling (ebullitive) events. Previous work to quantify methane bubbling from a number of Australian sub-tropical reservoirs has shown that this can contribute as much as 95% of total emissions. These bubbling events are controlled by a variety of different factors including water depth, surface and internal waves, wind seiching, atmospheric pressure changes and water levels changes. Key to quantifying the magnitude of this emission pathway is estimating both the bubbling rate as well as the areal extent of bubbling. Both bubbling rate and areal extent are seldom constant and require persistent monitoring over extended time periods before true estimates can be generated. In this paper we present a novel system for persistent monitoring of both bubbling rate and areal extent using multiple robotic surface chambers and adaptive sampling (grazing) algorithms to automate the quantification process. Individual chambers are self-propelled and guided and communicate between each other without the need for supervised control. They can maintain station at a sampling site for a desired incubation period and continuously monitor, record and report fluxes during the incubation. To exploit the methane sensor detection capabilities, the chamber can be automatically lowered to decrease the head-space and increase concentration. The grazing algorithms assign a hierarchical order to chambers within a preselected zone. Chambers then converge on the individual recording the highest 15 minute bubbling rate. Individuals maintain a specified distance apart from each other during each sampling period before all individuals are then required to move to different locations based on a sampling algorithm (systematic or adaptive) exploiting prior measurements. This system has been field tested on a large-scale subtropical reservoir, Little Nerang Dam, and over monthly timescales. Using this technique, localised bubbling zones on the water storage were found to produce over 50,000 mg m-2 d-1 and the areal extent ranged from 1.8 to 7% of the total reservoir area. The drivers behind these changes as well as lessons learnt from the system implementation are presented. This system exploits relatively cheap materials, sensing and computing and can be applied to a wide variety of aquatic and terrestrial systems.

Automatic camera exposure control

It is commonplace to use digital video cameras in robotic applications. These cameras have built-in exposure control but they do not have any knowledge of the environment, the lens being used, the important areas of the image and do not always produce optimal image exposure. Therefore, it is desirable and often necessary to control the exposure off the camera. In this paper we present a scheme for exposure control which enables the user application to determine the area of interest. The proposed scheme introduces an intermediate transparent layer between the camera and the user application which combines the information from these for optimal exposure production. We present results from indoor and outdoor scenarios using directional and fish-eye lenses showing the performance and advantages of this framework.

Superpixel-based appearance change prediction for long-term navigation across seasons

Changing environments pose a serious problem to current robotic systems aiming at long term operation under varying seasons or local weather conditions. This paper is built on our previous work where we propose to learn to predict the changes in an environment. Our key insight is that the occurring scene changes are in part systematic, repeatable and therefore predictable. The goal of our work is to support existing approaches to place recognition by learning how the visual appearance of an environment changes over time and by using this learned knowledge to predict its appearance under different environmental conditions. We describe the general idea of appearance change prediction (ACP) and investigate properties of our novel implementation based on vocabularies of superpixels (SP-ACP). Our previous work showed that the proposed approach significantly improves the performance of SeqSLAM and BRIEF-Gist for place recognition on a subset of the Nordland dataset under extremely different environmental conditions in summer and winter. This paper deepens the understanding of the proposed SP-ACP system and evaluates the influence of its parameters. We present the results of a large-scale experiment on the complete 10 h Nordland dataset and appearance change predictions between different combinations of seasons.

Phobos and deimos on Mars - two autonomous robots for the DLR SpaceBot Cup

In 2013, ten teams from German universities and research institutes participated in a national robot competition called SpaceBot Cup organized by the DLR Space Administration. The robots had one hour to autonomously explore and map a challenging Mars-like environment, find, transport, and manipulate two objects, and navigate back to the landing site. Localization without GPS in an unstructured environment was a major issue as was mobile manipulation and very restricted communication. This paper describes our system of two rovers operating on the ground plus a quadrotor UAV simulating an observing orbiting satellite. We relied on ROS (robot operating system) as the software infrastructure and describe the main ROS components utilized in performing the tasks. Despite (or because of) faults, communication loss and breakdowns, it was a valuable experience with many lessons learned.