OpenFABMAP : an open source toolbox for appearance-based loop closure detection

Appearance-based loop closure techniques, which leverage the high information content of visual images and can be used independently of pose, are now widely used in robotic applications. The current state-of-the-art in the field is Fast Appearance-Based Mapping (FAB-MAP) having been demonstrated in several seminal robotic mapping experiments. In this paper, we describe OpenFABMAP, a fully open source implementation of the original FAB-MAP algorithm. Beyond the benefits of full user access to the source code, OpenFABMAP provides a number of configurable options including rapid codebook training and interest point feature tuning. We demonstrate the performance of OpenFABMAP on a number of published datasets and demonstrate the advantages of quick algorithm customisation. We present results from OpenFABMAP’s application in a highly varied range of robotics research scenarios.

Application of an appearance-based intervention to improve sun protection outcomes of outdoor workers in Queensland, Australia

Outdoor workers are exposed to high levels of ultraviolet radiation (UVR) and may thus be at greater risk to experience UVR-related health effects such as skin cancer, sun burn, and cataracts. A number of intervention trials (n=14) have aimed to improve outdoor workers’ work-related sun protection cognitions and behaviours. Only one study however has reported the use of UV-photography as part of a multi-component intervention. This study was performed in the USA and showed long-term (12 months) improvements in work-related sun protection behaviours. Intervention effects of the other studies have varied greatly, depending on the population studied, intervention applied, and measurement of effect. Previous studies have not assessed whether: - Interventions are similarly effective for workers in stringent and less stringent policy organisations; - Policy effect is translated into workers’ leisure time protection; - Implemented interventions are effective in the long-term; - The facial UV-photograph technique is effective in Australian male outdoor workers without a large additional intervention package, and; - Such interventions will also affect workers’ leisure time sun-related cognitions and behaviours. Therefore, the present Protection of Outdoor Workers from Environmental Radiation [POWER]-study aimed to fill these gaps and had the objectives of: a) assessing outdoor workers’ sun-related cognitions and behaviours at work and during leisure time in stringent and less stringent sun protection policy environments; b) assessing the effect of an appearance-based intervention on workers’ risk perceptions, intentions and behaviours over time; c) assessing whether the intervention was equally effective within the two policy settings; and d) assessing the immediate post-intervention effect. Effectiveness was described in terms of changes in sun-related risk perceptions and intentions (as these factors were shown to be main precursors of behaviour change in many health promotion theories) and behaviour. The study purposefully selected and recruited two organisations with a large outdoor worker contingent in Queensland, Australia within a 40 kilometre radius of Brisbane. The two organisations differed in the stringency of implementation and reinforcement of their organisational sun protection policy. Data were collected from 154 male predominantly Australian born outdoor workers with an average age of 37 years and predominantly medium to fair skin (83%). Sun-related cognitions and behaviours of workers were assessed using self-report questionnaires at baseline and six to twelve months later. Variation in follow-up time was due to a time difference in the recruitment of the two organisations. Participants within each organisation were assigned to an intervention or control group. The intervention group participants received a one-off personalised Skin Cancer Risk Assessment Tool [SCRAT]-letter and a facial UV-photograph with detailed verbal information. This was followed by an immediate post-intervention questionnaire within three months of the start of the study. The control group only received the baseline and follow-up questionnaire. Data were analysed using a variety of techniques including: descriptive analyses, parametric and non-parametric tests, and generalised estimating equations. A 15% proportional difference observed was deemed of clinical significance, with the addition of reported statistical significance (p<0.05) where applicable. Objective 1: Assess and compare the current sun-related risk perceptions, intentions, behaviours, and policy awareness of outdoor workers in stringent and less stringent sun protection policy settings. Workers within the two organisations (stringent n=89 and less stringent n=65) were similar in their knowledge about skin cancer, self efficacy, attitudes, and social norms regarding sun protection at work and during leisure time. Participants were predominantly in favour of sun protection. Results highlighted that compared to workers in a less stringent policy organisation working for an organisation with stringent sun protection policies and practices resulted in more desirable sun protection intentions (less willing to tan p=0.03) ; actual behaviours at work (sufficient use of upper and lower body protection, headgear, and sunglasses (p<0.001 for all comparisons), and greater policy awareness (awareness of repercussions if Personal Protective Equipment (PPE) was not used, p<0.001)). However the effect of the work-related sun protection policy was found not to extend to leisure time sun protection. Objective 2: Compare changes in sun-related risk perceptions, intentions, and behaviours between the intervention and control group. The effect of the intervention was minimal and mainly resulted in a clinically significant reduction in work-related self-perceived risk of developing skin cancer in the intervention compared to the control group (16% and 32% for intervention and control group, respectively estimated their risk higher compared to other outdoor workers: , p=0.11). No other clinical significant effects were observed at 12 months follow-up. Objective 3: Assess whether the intervention was equally effective in the stringent sun protection policy organisation and the less stringent sun protection policy organisation. The appearance-based intervention resulted in a clinically significant improvement in the stringent policy intervention group participants’ intention to protect from the sun at work (workplace*time interaction, p=0.01). In addition to a reduction in their willingness to tan both at work (will tan at baseline: 17% and 61%, p=0.06, at follow-up: 54% and 33%, p=0.07, stringent and less stringent policy intervention group respectively. The workplace*time interaction was significant p<0.001) and during leisure time (will tan at baseline: 42% and 78%, p=0.01, at follow-up: 50% and 63%, p=0.43, stringent and less stringent policy intervention group respectively. The workplace*time interaction was significant p=0.01) over the course of the study compared to the less stringent policy intervention group. However, no changes in actual sun protection behaviours were found. Objective 4: Examine the effect of the intervention on level of alarm and concern regarding the health of the skin as well as sun protection behaviours in both organisations. The immediate post-intervention results showed that the stringent policy organisation participants indicated to be less alarmed (p=0.04) and concerned (p<0.01) about the health of their skin and less likely to show the facial UV-photograph to others (family p=0.03) compared to the less stringent policy participants. A clinically significantly larger proportion of participants from the stringent policy organisation reported they worried more about skin cancer (65%) and skin freckling (43%) compared to those in the less stringent policy organisation (46%,and 23% respectively , after seeing the UV-photograph). In summary the results of this study suggest that the having a stringent work-related sun protection policy was significantly related to for work-time sun protection practices, but did not extend to leisure time sun protection. This could reflect the insufficient level of sun protection found in the general Australian population during leisure time. Alternatively, reactance caused by being restricted in personal decisions through work-time policy could have contributed to lower leisure time sun protection. Finally, other factors could have also contributed to the less than optimal leisure time sun protection behaviours reported, such as unmeasured personal or cultural barriers. All these factors combined may have lead to reduced willingness to take proper preventive action during leisure time exposure. The intervention did not result in any measurable difference between the intervention and control groups in sun protection behaviours in this population, potentially due to the long lag time between the implementation of the intervention and assessment at 12-months follow-up. In addition, high levels of sun protection behaviours were found at baseline (ceiling effect) which left little room for improvement. Further, this study did not assess sunscreen use, which was the predominant behaviour assessed in previous effective appearance-based interventions trials. Additionally, previous trials were mainly conducted in female populations, whilst the POWER-study’s population was all male. The observed immediate post-intervention result could be due to more emphasis being placed on sun protection and risks related to sun exposure in the stringent policy organisation. Therefore participants from the stringent policy organisation could have been more aware of harmful effects of UVR and hence, by knowing that they usually protect adequately, not be as alarmed or concerned as the participants from the less stringent policy organisation. In conclusion, a facial UV-photograph and SCRAT-letter information alone may not achieve large changes in sun-related cognitions and behaviour, especially of assessed 6-12 months after the intervention was implemented and in workers who are already quite well protected. Differences found between workers in the present study appear to be more attributable to organisational policy. However, against a background of organisational policy, this intervention may be a useful addition to sun-related workplace health and safety programs. The study findings have been interpreted while respecting a number of limitations. These have included non-random allocation of participants due to pre-organised allocation of participants to study group in one organisation and difficulty in separating participants from either study group. Due to the transient nature of the outdoor worker population, only 105 of 154 workers available at baseline could be reached for follow-up. (attrition rate=32%). In addition the discrepancy in the time to follow-up assessment between the two organisations was a limitation of the current study. Given the caveats of this research, the following recommendations were made for future research: - Consensus should be reached to define "outdoor worker" in terms of time spent outside at work as well as in the way sun protection behaviours are measured and reported. - Future studies should implement and assess the value of the facial UV-photographs in a wide range of outdoor worker organisations and countries. - More timely and frequent follow-up assessments should be implemented in intervention studies to determine the intervention effect and to identify the best timing of booster sessions to optimise results. - Future research should continue to aim to target outdoor workers’ leisure time cognitions and behaviours and improve these if possible. Overall, policy appears to be an important factor in workers’ compliance with work-time use of sun protection. Given the evidence generated by this research, organisations employing outdoor workers should consider stringent implementation and reinforcement of a sun protection policy. Finally, more research is needed to improve ways to generate desirable behaviour in this population during leisure time.

Image Region Salience for Improving Appearance-Based Place Recognition using a Supervised Classifier System

Many state of the art vision-based Simultaneous Localisation And Mapping (SLAM) and place recognition systems compute the salience of visual features in their environment. As computing salience can be problematic in radically changing environments new low resolution feature-less systems have been introduced, such as SeqSLAM, all of which consider the whole image. In this paper, we implement a supervised classifier system (UCS) to learn the salience of image regions for place recognition by feature-less systems. SeqSLAM only slightly benefits from the results of training, on the challenging real world Eynsham dataset, as it already appears to filter less useful regions of a panoramic image. However, when recognition is limited to specific image regions performance improves by more than an order of magnitude by utilising the learnt image region saliency. We then investigate whether the region salience generated from the Eynsham dataset generalizes to another car-based dataset using a perspective camera. The results suggest the general applicability of an image region salience mask for optimizing route-based navigation applications.

An adaptive appearance-based map for long-term topological localization of mobile robots

"This work considers a mobile service robot which uses an appearance-based representation of its workplace as a map, where the current view and the map are used to estimate the current position in the environment. Due to the nature of real-world environments such as houses and offices, where the appearance keeps changing, the internal representation may become out of date after some time. To solve this problem the robot needs to be able to adapt its internal representation continually to the changes in the environment. This paper presents a method for creating an adaptive map for long-term appearance-based localization of a mobile robot using long-term and short-term memory concepts, with omni-directional vision as the external sensor."--publisher website

Comparative analysis of traffic state estimation ─ Cumulative counts-based and trajectory-based methods

The Macroscopic Fundamental Diagram (MFD) relates space-mean density and flow. Since the MFD represents the area-wide network traffic performance, studies on perimeter control strategies and network-wide traffic state estimation utilising the MFD concept have been reported. Most previous works have utilised data from fixed sensors, such as inductive loops, to estimate the MFD, which can cause biased estimation in urban networks due to queue spillovers at intersections. To overcome the limitation, recent literature reports the use of trajectory data obtained from probe vehicles. However, these studies have been conducted using simulated datasets; limited works have discussed the limitations of real datasets and their impact on the variable estimation. This study compares two methods for estimating traffic state variables of signalised arterial sections: a method based on cumulative vehicle counts (CUPRITE), and one based on vehicles’ trajectory from taxi Global Positioning System (GPS) log. The comparisons reveal some characteristics of taxi trajectory data available in Brisbane, Australia. The current trajectory data have limitations in quantity (i.e., the penetration rate), due to which the traffic state variables tend to be underestimated. Nevertheless, the trajectory-based method successfully captures the features of traffic states, which suggests that the trajectories from taxis can be a good estimator for the network-wide traffic states.

Multi-shot person re-identification via relational stein divergence

Person re-identification is particularly challenging due to significant appearance changes across separate camera views. In order to re-identify people, a representative human signature should effectively handle differences in illumination, pose and camera parameters. While general appearance-based methods are modelled in Euclidean spaces, it has been argued that some applications in image and video analysis are better modelled via non-Euclidean manifold geometry. To this end, recent approaches represent images as covariance matrices, and interpret such matrices as points on Riemannian manifolds. As direct classification on such manifolds can be difficult, in this paper we propose to represent each manifold point as a vector of similarities to class representers, via a recently introduced form of Bregman matrix divergence known as the Stein divergence. This is followed by using a discriminative mapping of similarity vectors for final classification. The use of similarity vectors is in contrast to the traditional approach of embedding manifolds into tangent spaces, which can suffer from representing the manifold structure inaccurately. Comparative evaluations on benchmark ETHZ and iLIDS datasets for the person re-identification task show that the proposed approach obtains better performance than recent techniques such as Histogram Plus Epitome, Partial Least Squares, and Symmetry-Driven Accumulation of Local Features.

Superpixel-based appearance change prediction for long-term navigation across seasons

Changing environments pose a serious problem to current robotic systems aiming at long term operation under varying seasons or local weather conditions. This paper is built on our previous work where we propose to learn to predict the changes in an environment. Our key insight is that the occurring scene changes are in part systematic, repeatable and therefore predictable. The goal of our work is to support existing approaches to place recognition by learning how the visual appearance of an environment changes over time and by using this learned knowledge to predict its appearance under different environmental conditions. We describe the general idea of appearance change prediction (ACP) and investigate properties of our novel implementation based on vocabularies of superpixels (SP-ACP). Our previous work showed that the proposed approach significantly improves the performance of SeqSLAM and BRIEF-Gist for place recognition on a subset of the Nordland dataset under extremely different environmental conditions in summer and winter. This paper deepens the understanding of the proposed SP-ACP system and evaluates the influence of its parameters. We present the results of a large-scale experiment on the complete 10 h Nordland dataset and appearance change predictions between different combinations of seasons.

Improved correlation-based modal strain energy method for global damage detection of truss bridge structures

The previous investigations have shown that the modal strain energy correlation method, MSEC, could successfully identify the damage of truss bridge structures. However, it has to incorporate the sensitivity matrix to estimate damage and is not reliable in certain damage detection cases. This paper presents an improved MSEC method where the prediction of modal strain energy change vector is differently obtained by running the eigensolutions on-line in optimisation iterations. The particular trail damage treatment group maximising the fitness function close to unity is identified as the detected damage location. This improvement is then compared with the original MSEC method along with other typical correlation-based methods on the finite element model of a simple truss bridge. The contributions to damage detection accuracy of each considered mode is also weighed and discussed. The iterative searching process is operated by using genetic algorithm. The results demonstrate that the improved MSEC method suffices the demand in detecting the damage of truss bridge structures, even when noised measurement is considered.

Probability based data analysis for road asset management

Road agencies require comprehensive, relevan and quality data describing their road assets to support their investment decisions. An investment decision support system for raod maintenance and rehabilitation mainly comprise three important supporting elements namely: road asset data, decision support tools and criteria for decision-making. Probability-based methods have played a crucial role in helping decision makers understand the relationship among road related data, asset performance and uncertainties in estimating budgets/costs for road management investment. This paper presents applications of the probability-bsed method for road asset management.

Geometry vs appearance for discriminating between posed and spontaneous emotions

Spontaneous facial expressions differ from posed ones in appearance, timing and accompanying head movements. Still images cannot provide timing or head movement information directly. However, indirectly the distances between key points on a face extracted from a still image using active shape models can capture some movement and pose changes. This information is superposed on information about non-rigid facial movement that is also part of the expression. Does geometric information improve the discrimination between spontaneous and posed facial expressions arising from discrete emotions? We investigate the performance of a machine vision system for discrimination between posed and spontaneous versions of six basic emotions that uses SIFT appearance based features and FAP geometric features. Experimental results on the NVIE database demonstrate that fusion of geometric information leads only to marginal improvement over appearance features. Using fusion features, surprise is the easiest emotion (83.4% accuracy) to be distinguished, while disgust is the most difficult (76.1%). Our results find different important facial regions between discriminating posed versus spontaneous version of one emotion and classifying the same emotion versus other emotions. The distribution of the selected SIFT features shows that mouth is more important for sadness, while nose is more important for surprise, however, both the nose and mouth are important for disgust, fear, and happiness. Eyebrows, eyes, nose and mouth are important for anger.

In the pursuit of effective affective computing : the relationship between features and registration

For facial expression recognition systems to be applicable in the real world, they need to be able to detect and track a previously unseen person's face and its facial movements accurately in realistic environments. A highly plausible solution involves performing a "dense" form of alignment, where 60-70 fiducial facial points are tracked with high accuracy. The problem is that, in practice, this type of dense alignment had so far been impossible to achieve in a generic sense, mainly due to poor reliability and robustness. Instead, many expression detection methods have opted for a "coarse" form of face alignment, followed by an application of a biologically inspired appearance descriptor such as the histogram of oriented gradients or Gabor magnitudes. Encouragingly, recent advances to a number of dense alignment algorithms have demonstrated both high reliability and accuracy for unseen subjects [e.g., constrained local models (CLMs)]. This begs the question: Aside from countering against illumination variation, what do these appearance descriptors do that standard pixel representations do not? In this paper, we show that, when close to perfect alignment is obtained, there is no real benefit in employing these different appearance-based representations (under consistent illumination conditions). In fact, when misalignment does occur, we show that these appearance descriptors do work well by encoding robustness to alignment error. For this work, we compared two popular methods for dense alignment-subject-dependent active appearance models versus subject-independent CLMs-on the task of action-unit detection. These comparisons were conducted through a battery of experiments across various publicly available data sets (i.e., CK+, Pain, M3, and GEMEP-FERA). We also report our performance in the recent 2011 Facial Expression Recognition and Analysis Challenge for the subject-independent task.

Towards robust night and day place recognition using visible and thermal imaging

The chief challenge facing persistent robotic navigation using vision sensors is the recognition of previously visited locations under different lighting and illumination conditions. The majority of successful approaches to outdoor robot navigation use active sensors such as LIDAR, but the associated weight and power draw of these systems makes them unsuitable for widespread deployment on mobile robots. In this paper we investigate methods to combine representations for visible and long-wave infrared (LWIR) thermal images with time information to combat the time-of-day-based limitations of each sensing modality. We calculate appearance-based match likelihoods using the state-of-the-art FAB-MAP [1] algorithm to analyse loop closure detection reliability across different times of day. We present preliminary results on a dataset of 10 successive traverses of a combined urban-parkland environment, recorded in 2-hour intervals from before dawn to after dusk. Improved location recognition throughout an entire day is demonstrated using the combined system compared with methods which use visible or thermal sensing alone.

Correlation-based damage detection for complicated truss bridges using multi-layer genetic algorithms

The study presents a multi-layer genetic algorithm (GA) approach using correlation-based methods to facilitate damage determination for through-truss bridge structures. To begin, the structure’s damage-suspicious elements are divided into several groups. In the first GA layer, the damage is initially optimised for all groups using correlation objective function. In the second layer, the groups are combined to larger groups and the optimisation starts over at the normalised point of the first layer result. Then the identification process repeats until reaching the final layer where one group includes all structural elements and only minor optimisations are required to fine tune the final result. Several damage scenarios on a complicated through-truss bridge example are nominated to address the proposed approach’s effectiveness. Structural modal strain energy has been employed as the variable vector in the correlation function for damage determination. Simulations and comparison with the traditional single-layer optimisation shows that the proposed approach is efficient and feasible for complicated truss bridge structures when the measurement noise is taken into account.

Visualisation of complex flows using texture-based techniques

Complex flow datasets are often difficult to represent in detail using traditional vector visualisation techniques such as arrow plots and streamlines. This is particularly true when the flow regime changes in time. Texture-based techniques, which are based on the advection of dense textures, are novel techniques for visualising such flows (i.e., complex dynamics and time-dependent). In this paper, we review two popular texture-based techniques and their application to flow datasets sourced from real research projects. The texture-based techniques investigated were Line Integral Convolution (LIC), and Image-Based Flow Visualisation (IBFV). We evaluated these techniques and in this paper report on their visualisation effectiveness (when compared with traditional techniques), their ease of implementation, and their computational overhead.