Capping computation time and storage requirements for appearance-based localization with CAT-SLAM

Appearance-based localization is increasingly used for loop closure detection in metric SLAM systems. Since it relies only upon the appearance-based similarity between images from two locations, it can perform loop closure regardless of accumulated metric error. However, the computation time and memory requirements of current appearance-based methods scale linearly not only with the size of the environment but also with the operation time of the platform. These properties impose severe restrictions on longterm autonomy for mobile robots, as loop closure performance will inevitably degrade with increased operation time. We present a set of improvements to the appearance-based SLAM algorithm CAT-SLAM to constrain computation scaling and memory usage with minimal degradation in performance over time. The appearance-based comparison stage is accelerated by exploiting properties of the particle observation update, and nodes in the continuous trajectory map are removed according to minimal information loss criteria. We demonstrate constant time and space loop closure detection in a large urban environment with recall performance exceeding FAB-MAP by a factor of 3 at 100% precision, and investigate the minimum computational and memory requirements for maintaining mapping performance.

Application of an appearance-based intervention to improve sun protection outcomes of outdoor workers in Queensland, Australia

Outdoor workers are exposed to high levels of ultraviolet radiation (UVR) and may thus be at greater risk to experience UVR-related health effects such as skin cancer, sun burn, and cataracts. A number of intervention trials (n=14) have aimed to improve outdoor workers’ work-related sun protection cognitions and behaviours. Only one study however has reported the use of UV-photography as part of a multi-component intervention. This study was performed in the USA and showed long-term (12 months) improvements in work-related sun protection behaviours. Intervention effects of the other studies have varied greatly, depending on the population studied, intervention applied, and measurement of effect. Previous studies have not assessed whether: - Interventions are similarly effective for workers in stringent and less stringent policy organisations; - Policy effect is translated into workers’ leisure time protection; - Implemented interventions are effective in the long-term; - The facial UV-photograph technique is effective in Australian male outdoor workers without a large additional intervention package, and; - Such interventions will also affect workers’ leisure time sun-related cognitions and behaviours. Therefore, the present Protection of Outdoor Workers from Environmental Radiation [POWER]-study aimed to fill these gaps and had the objectives of: a) assessing outdoor workers’ sun-related cognitions and behaviours at work and during leisure time in stringent and less stringent sun protection policy environments; b) assessing the effect of an appearance-based intervention on workers’ risk perceptions, intentions and behaviours over time; c) assessing whether the intervention was equally effective within the two policy settings; and d) assessing the immediate post-intervention effect. Effectiveness was described in terms of changes in sun-related risk perceptions and intentions (as these factors were shown to be main precursors of behaviour change in many health promotion theories) and behaviour. The study purposefully selected and recruited two organisations with a large outdoor worker contingent in Queensland, Australia within a 40 kilometre radius of Brisbane. The two organisations differed in the stringency of implementation and reinforcement of their organisational sun protection policy. Data were collected from 154 male predominantly Australian born outdoor workers with an average age of 37 years and predominantly medium to fair skin (83%). Sun-related cognitions and behaviours of workers were assessed using self-report questionnaires at baseline and six to twelve months later. Variation in follow-up time was due to a time difference in the recruitment of the two organisations. Participants within each organisation were assigned to an intervention or control group. The intervention group participants received a one-off personalised Skin Cancer Risk Assessment Tool [SCRAT]-letter and a facial UV-photograph with detailed verbal information. This was followed by an immediate post-intervention questionnaire within three months of the start of the study. The control group only received the baseline and follow-up questionnaire. Data were analysed using a variety of techniques including: descriptive analyses, parametric and non-parametric tests, and generalised estimating equations. A 15% proportional difference observed was deemed of clinical significance, with the addition of reported statistical significance (p<0.05) where applicable. Objective 1: Assess and compare the current sun-related risk perceptions, intentions, behaviours, and policy awareness of outdoor workers in stringent and less stringent sun protection policy settings. Workers within the two organisations (stringent n=89 and less stringent n=65) were similar in their knowledge about skin cancer, self efficacy, attitudes, and social norms regarding sun protection at work and during leisure time. Participants were predominantly in favour of sun protection. Results highlighted that compared to workers in a less stringent policy organisation working for an organisation with stringent sun protection policies and practices resulted in more desirable sun protection intentions (less willing to tan p=0.03) ; actual behaviours at work (sufficient use of upper and lower body protection, headgear, and sunglasses (p<0.001 for all comparisons), and greater policy awareness (awareness of repercussions if Personal Protective Equipment (PPE) was not used, p<0.001)). However the effect of the work-related sun protection policy was found not to extend to leisure time sun protection. Objective 2: Compare changes in sun-related risk perceptions, intentions, and behaviours between the intervention and control group. The effect of the intervention was minimal and mainly resulted in a clinically significant reduction in work-related self-perceived risk of developing skin cancer in the intervention compared to the control group (16% and 32% for intervention and control group, respectively estimated their risk higher compared to other outdoor workers: , p=0.11). No other clinical significant effects were observed at 12 months follow-up. Objective 3: Assess whether the intervention was equally effective in the stringent sun protection policy organisation and the less stringent sun protection policy organisation. The appearance-based intervention resulted in a clinically significant improvement in the stringent policy intervention group participants’ intention to protect from the sun at work (workplace*time interaction, p=0.01). In addition to a reduction in their willingness to tan both at work (will tan at baseline: 17% and 61%, p=0.06, at follow-up: 54% and 33%, p=0.07, stringent and less stringent policy intervention group respectively. The workplace*time interaction was significant p<0.001) and during leisure time (will tan at baseline: 42% and 78%, p=0.01, at follow-up: 50% and 63%, p=0.43, stringent and less stringent policy intervention group respectively. The workplace*time interaction was significant p=0.01) over the course of the study compared to the less stringent policy intervention group. However, no changes in actual sun protection behaviours were found. Objective 4: Examine the effect of the intervention on level of alarm and concern regarding the health of the skin as well as sun protection behaviours in both organisations. The immediate post-intervention results showed that the stringent policy organisation participants indicated to be less alarmed (p=0.04) and concerned (p<0.01) about the health of their skin and less likely to show the facial UV-photograph to others (family p=0.03) compared to the less stringent policy participants. A clinically significantly larger proportion of participants from the stringent policy organisation reported they worried more about skin cancer (65%) and skin freckling (43%) compared to those in the less stringent policy organisation (46%,and 23% respectively , after seeing the UV-photograph). In summary the results of this study suggest that the having a stringent work-related sun protection policy was significantly related to for work-time sun protection practices, but did not extend to leisure time sun protection. This could reflect the insufficient level of sun protection found in the general Australian population during leisure time. Alternatively, reactance caused by being restricted in personal decisions through work-time policy could have contributed to lower leisure time sun protection. Finally, other factors could have also contributed to the less than optimal leisure time sun protection behaviours reported, such as unmeasured personal or cultural barriers. All these factors combined may have lead to reduced willingness to take proper preventive action during leisure time exposure. The intervention did not result in any measurable difference between the intervention and control groups in sun protection behaviours in this population, potentially due to the long lag time between the implementation of the intervention and assessment at 12-months follow-up. In addition, high levels of sun protection behaviours were found at baseline (ceiling effect) which left little room for improvement. Further, this study did not assess sunscreen use, which was the predominant behaviour assessed in previous effective appearance-based interventions trials. Additionally, previous trials were mainly conducted in female populations, whilst the POWER-study’s population was all male. The observed immediate post-intervention result could be due to more emphasis being placed on sun protection and risks related to sun exposure in the stringent policy organisation. Therefore participants from the stringent policy organisation could have been more aware of harmful effects of UVR and hence, by knowing that they usually protect adequately, not be as alarmed or concerned as the participants from the less stringent policy organisation. In conclusion, a facial UV-photograph and SCRAT-letter information alone may not achieve large changes in sun-related cognitions and behaviour, especially of assessed 6-12 months after the intervention was implemented and in workers who are already quite well protected. Differences found between workers in the present study appear to be more attributable to organisational policy. However, against a background of organisational policy, this intervention may be a useful addition to sun-related workplace health and safety programs. The study findings have been interpreted while respecting a number of limitations. These have included non-random allocation of participants due to pre-organised allocation of participants to study group in one organisation and difficulty in separating participants from either study group. Due to the transient nature of the outdoor worker population, only 105 of 154 workers available at baseline could be reached for follow-up. (attrition rate=32%). In addition the discrepancy in the time to follow-up assessment between the two organisations was a limitation of the current study. Given the caveats of this research, the following recommendations were made for future research: - Consensus should be reached to define "outdoor worker" in terms of time spent outside at work as well as in the way sun protection behaviours are measured and reported. - Future studies should implement and assess the value of the facial UV-photographs in a wide range of outdoor worker organisations and countries. - More timely and frequent follow-up assessments should be implemented in intervention studies to determine the intervention effect and to identify the best timing of booster sessions to optimise results. - Future research should continue to aim to target outdoor workers’ leisure time cognitions and behaviours and improve these if possible. Overall, policy appears to be an important factor in workers’ compliance with work-time use of sun protection. Given the evidence generated by this research, organisations employing outdoor workers should consider stringent implementation and reinforcement of a sun protection policy. Finally, more research is needed to improve ways to generate desirable behaviour in this population during leisure time.

Towards training-free appearance-based localization : probabilistic models for whole-image descriptors

Whole image descriptors have recently been shown to be remarkably robust to perceptual change especially compared to local features. However, whole-image-based localization systems typically rely on heuristic methods for determining appropriate matching thresholds in a particular environment. These environment-specific tuning requirements and the lack of a meaningful interpretation of these arbitrary thresholds limits the general applicability of these systems. In this paper we present a Bayesian model of probability for whole-image descriptors that can be seamlessly integrated into localization systems designed for probabilistic visual input. We demonstrate this method using CAT-Graph, an appearance-based visual localization system originally designed for a FAB-MAP-style probabilistic input. We show that using whole-image descriptors as visual input extends CAT-Graph’s functionality to environments that experience a greater amount of perceptual change. We also present a method of estimating whole-image probability models in an online manner, removing the need for a prior training phase. We show that this online, automated training method can perform comparably to pre-trained, manually tuned local descriptor methods.

An adaptive appearance-based map for long-term topological localization of mobile robots

"This work considers a mobile service robot which uses an appearance-based representation of its workplace as a map, where the current view and the map are used to estimate the current position in the environment. Due to the nature of real-world environments such as houses and offices, where the appearance keeps changing, the internal representation may become out of date after some time. To solve this problem the robot needs to be able to adapt its internal representation continually to the changes in the environment. This paper presents a method for creating an adaptive map for long-term appearance-based localization of a mobile robot using long-term and short-term memory concepts, with omni-directional vision as the external sensor."--publisher website

An anticipative reactive control strategy to deal with unforeseen obstacles during a multi-sensor-based navigation task

We address the problem of the rangefinder-based avoidance of unforeseen static obstacles during a visual navigation task. We extend previous strategies which are efficient in most cases but remain still hampered by some drawbacks (e.g., risks of collisions or of local minima in some particular cases, etc.). The key idea is to complete the control strategy by adding a controller providing the robot some anticipative skills to guarantee non collision and by defining more general transition conditions to deal with local minima. Simulation results show the proposed strategy efficiency.

Superpixel-based appearance change prediction for long-term navigation across seasons

Changing environments pose a serious problem to current robotic systems aiming at long term operation under varying seasons or local weather conditions. This paper is built on our previous work where we propose to learn to predict the changes in an environment. Our key insight is that the occurring scene changes are in part systematic, repeatable and therefore predictable. The goal of our work is to support existing approaches to place recognition by learning how the visual appearance of an environment changes over time and by using this learned knowledge to predict its appearance under different environmental conditions. We describe the general idea of appearance change prediction (ACP) and investigate properties of our novel implementation based on vocabularies of superpixels (SP-ACP). Our previous work showed that the proposed approach significantly improves the performance of SeqSLAM and BRIEF-Gist for place recognition on a subset of the Nordland dataset under extremely different environmental conditions in summer and winter. This paper deepens the understanding of the proposed SP-ACP system and evaluates the influence of its parameters. We present the results of a large-scale experiment on the complete 10 h Nordland dataset and appearance change predictions between different combinations of seasons.

Low-Cost vision-based AUV guidance system for reef navigation

Ensuring the long term viability of reef environments requires essential monitoring of many aspects of these ecosystems. However, the sheer size of these unstructured environments (for example Australia’s Great Barrier Reef pose a number of challenges for current monitoring platforms which are typically remote operated and required significant resources and infrastructure. Therefore, a primary objective of the CSIRO robotic reef monitoring project is to develop and deploy a large number of AUV teams to perform broadscale reef surveying. In order to achieve this, the platforms must be cheap, even possibly disposable. This paper presents the results of a preliminary investigation into the performance of a low-cost sensor suite and associated processing techniques for vision and inertial-based navigation within a highly unstructured reef environment.

Integrated vision-based attitude estimation

For many years, computer vision has lured researchers with promises of a low-cost, passive, lightweight and information-rich sensor suitable for navigation purposes. The prime difficulty in vision-based navigation is that the navigation solution will continually drift with time unless external information is available, whether it be cues from the appearance of the scene, a map of features (whether built online or known a priori), or from an externally-referenced sensor. It is not merely position that is of interest in the navigation problem. Attitude (i.e. the angular orientation of a body with respect to a reference frame) is integral to a visionbased navigation solution and is often of interest in its own right (e.g. flight control). This thesis examines vision-based attitude estimation in an aerospace environment, and two methods are proposed for constraining drift in the attitude solution; one through a novel integration of optical flow and the detection of the sky horizon, and the other through a loosely-coupled integration of Visual Odometry and GPS position measurements. In the first method, roll angle, pitch angle and the three aircraft body rates are recovered though a novel method of tracking the horizon over time and integrating the horizonderived attitude information with optical flow. An image processing front-end is used to select several candidate lines in a image that may or may not correspond to the true horizon, and the optical flow is calculated for each candidate line. Using an Extended Kalman Filter (EKF), the previously estimated aircraft state is propagated using a motion model and a candidate horizon line is associated using a statistical test based on the optical flow measurements and location of the horizon in the image. Once associated, the selected horizon line, along with the associated optical flow, is used as a measurement to the EKF. To evaluate the accuracy of the algorithm, two flights were conducted, one using a highly dynamic Uninhabited Airborne Vehicle (UAV) in clear flight conditions and the other in a human-piloted Cessna 172 in conditions where the horizon was partially obscured by terrain, haze and smoke. The UAV flight resulted in pitch and roll error standard deviations of 0.42° and 0.71° respectively when compared with a truth attitude source. The Cessna 172 flight resulted in pitch and roll error standard deviations of 1.79° and 1.75° respectively. In the second method for estimating attitude, a novel integrated GPS/Visual Odometry (GPS/VO) navigation filter is proposed, using a structure similar to a classic looselycoupled GPS/INS error-state navigation filter. Under such an arrangement, the error dynamics of the system are derived and a Kalman Filter is developed for estimating the errors in position and attitude. Through similar analysis to the GPS/INS problem, it is shown that the proposed filter is capable of recovering the complete attitude (i.e. pitch, roll and yaw) of the platform when subjected to acceleration not parallel to velocity for both the monocular and stereo variants of the filter. Furthermore, it is shown that under general straight line motion (e.g. constant velocity), only the component of attitude in the direction of motion is unobservable. Numerical simulations are performed to demonstrate the observability properties of the GPS/VO filter in both the monocular and stereo camera configurations. Furthermore, the proposed filter is tested on imagery collected using a Cessna 172 to demonstrate the observability properties on real-world data. The proposed GPS/VO filter does not require additional restrictions or assumptions such as platform-specific dynamics, map-matching, feature-tracking, visual loop-closing, gravity vector or additional sensors such as an IMU or magnetic compass. Since no platformspecific dynamics are required, the proposed filter is not limited to the aerospace domain and has the potential to be deployed in other platforms such as ground robots or mobile phones.

OpenRatSLAM : an open source brain-based SLAM system

RatSLAM is a navigation system based on the neural processes underlying navigation in the rodent brain, capable of operating with low resolution monocular image data. Seminal experiments using RatSLAM include mapping an entire suburb with a web camera and a long term robot delivery trial. This paper describes OpenRatSLAM, an open-source version of RatSLAM with bindings to the Robot Operating System framework to leverage advantages such as robot and sensor abstraction, networking, data playback, and visualization. OpenRatSLAM comprises connected ROS nodes to represent RatSLAM’s pose cells, experience map, and local view cells, as well as a fourth node that provides visual odometry estimates. The nodes are described with reference to the RatSLAM model and salient details of the ROS implementation such as topics, messages, parameters, class diagrams, sequence diagrams, and parameter tuning strategies. The performance of the system is demonstrated on three publicly available open-source datasets.

Vision-only autonomous navigation using topometric maps

This paper presents a mapping and navigation system for a mobile robot, which uses vision as its sole sensor modality. The system enables the robot to navigate autonomously, plan paths and avoid obstacles using a vision based topometric map of its environment. The map consists of a globally-consistent pose-graph with a local 3D point cloud attached to each of its nodes. These point clouds are used for direction independent loop closure and to dynamically generate 2D metric maps for locally optimal path planning. Using this locally semi-continuous metric space, the robot performs shortest path planning instead of following the nodes of the graph --- as is done with most other vision-only navigation approaches. The system exploits the local accuracy of visual odometry in creating local metric maps, and uses pose graph SLAM, visual appearance-based place recognition and point clouds registration to create the topometric map. The ability of the framework to sustain vision-only navigation is validated experimentally, and the system is provided as open-source software.

Visual place recognition for persistent robot navigation in changing environments

This thesis demonstrates that robots can learn about how the world changes, and can use this information to recognise where they are, even when the appearance of the environment has changed a great deal. The ability to localise in highly dynamic environments using vision only is a key tool for achieving long-term, autonomous navigation in unstructured outdoor environments. The proposed learning algorithms are designed to be unsupervised, and can be generated by the robot online in response to its observations of the world, without requiring information from a human operator or other external source.

Robot navigation in sensor space

This thesis investigates the problem of robot navigation using only landmark bearings. The proposed system allows a robot to move to a ground target location specified by the sensor values observed at this ground target posi- tion. The control actions are computed based on the difference between the current landmark bearings and the target landmark bearings. No Cartesian coordinates with respect to the ground are computed by the control system. The robot navigates using solely information from the bearing sensor space. Most existing robot navigation systems require a ground frame (2D Cartesian coordinate system) in order to navigate from a ground point A to a ground point B. The commonly used sensors such as laser range scanner, sonar, infrared, and vision do not directly provide the 2D ground coordi- nates of the robot. The existing systems use the sensor measurements to localise the robot with respect to a map, a set of 2D coordinates of the objects of interest. It is more natural to navigate between the points in the sensor space corresponding to A and B without requiring the Cartesian map and the localisation process. Research on animals has revealed how insects are able to exploit very limited computational and memory resources to successfully navigate to a desired destination without computing Cartesian positions. For example, a honeybee balances the left and right optical flows to navigate in a nar- row corridor. Unlike many other ants, Cataglyphis bicolor does not secrete pheromone trails in order to find its way home but instead uses the sun as a compass to keep track of its home direction vector. The home vector can be inaccurate, so the ant also uses landmark recognition. More precisely, it takes snapshots and compass headings of some landmarks. To return home, the ant tries to line up the landmarks exactly as they were before it started wandering. This thesis introduces a navigation method based on reflex actions in sensor space. The sensor vector is made of the bearings of some landmarks, and the reflex action is a gradient descent with respect to the distance in sensor space between the current sensor vector and the target sensor vec- tor. Our theoretical analysis shows that except for some fully characterized pathological cases, any point is reachable from any other point by reflex action in the bearing sensor space provided the environment contains three landmarks and is free of obstacles. The trajectories of a robot using reflex navigation, like other image- based visual control strategies, do not correspond necessarily to the shortest paths on the ground, because the sensor error is minimized, not the moving distance on the ground. However, we show that the use of a sequence of waypoints in sensor space can address this problem. In order to identify relevant waypoints, we train a Self Organising Map (SOM) from a set of observations uniformly distributed with respect to the ground. This SOM provides a sense of location to the robot, and allows a form of path planning in sensor space. The navigation proposed system is analysed theoretically, and evaluated both in simulation and with experiments on a real robot.

A Hybrid AUV design for shallow water reef navigation

The highly unstructured nature of coral reef environments makes them difficult for current robotic vehicles to efficiently navigate. Typical research and commercial platforms have limited autonomy within these environments and generally require tethers and significant external infrastructure. This paper outlines the development of a new robotic vehicle for underwater monitoring and surveying in highly unstructured environments and presents experimental results illustrating the vehicle’s performance. The hybrid AUV design developed by the CSIRO robotic reef monitoring team realises a compromise between endurance, manoeuvrability and functionality. The vehicle represents a new era in AUV design specifically focused at providing a truly low-cost research capability that will progress environmental monitoring through unaided navigation, cooperative robotics, sensor network distribution and data harvesting.

Geometry vs appearance for discriminating between posed and spontaneous emotions

Spontaneous facial expressions differ from posed ones in appearance, timing and accompanying head movements. Still images cannot provide timing or head movement information directly. However, indirectly the distances between key points on a face extracted from a still image using active shape models can capture some movement and pose changes. This information is superposed on information about non-rigid facial movement that is also part of the expression. Does geometric information improve the discrimination between spontaneous and posed facial expressions arising from discrete emotions? We investigate the performance of a machine vision system for discrimination between posed and spontaneous versions of six basic emotions that uses SIFT appearance based features and FAP geometric features. Experimental results on the NVIE database demonstrate that fusion of geometric information leads only to marginal improvement over appearance features. Using fusion features, surprise is the easiest emotion (83.4% accuracy) to be distinguished, while disgust is the most difficult (76.1%). Our results find different important facial regions between discriminating posed versus spontaneous version of one emotion and classifying the same emotion versus other emotions. The distribution of the selected SIFT features shows that mouth is more important for sadness, while nose is more important for surprise, however, both the nose and mouth are important for disgust, fear, and happiness. Eyebrows, eyes, nose and mouth are important for anger.

Towards robust night and day place recognition using visible and thermal imaging

The chief challenge facing persistent robotic navigation using vision sensors is the recognition of previously visited locations under different lighting and illumination conditions. The majority of successful approaches to outdoor robot navigation use active sensors such as LIDAR, but the associated weight and power draw of these systems makes them unsuitable for widespread deployment on mobile robots. In this paper we investigate methods to combine representations for visible and long-wave infrared (LWIR) thermal images with time information to combat the time-of-day-based limitations of each sensing modality. We calculate appearance-based match likelihoods using the state-of-the-art FAB-MAP [1] algorithm to analyse loop closure detection reliability across different times of day. We present preliminary results on a dataset of 10 successive traverses of a combined urban-parkland environment, recorded in 2-hour intervals from before dawn to after dusk. Improved location recognition throughout an entire day is demonstrated using the combined system compared with methods which use visible or thermal sensing alone.