Image Region Salience for Improving Appearance-Based Place Recognition using a Supervised Classifier System

Many state of the art vision-based Simultaneous Localisation And Mapping (SLAM) and place recognition systems compute the salience of visual features in their environment. As computing salience can be problematic in radically changing environments new low resolution feature-less systems have been introduced, such as SeqSLAM, all of which consider the whole image. In this paper, we implement a supervised classifier system (UCS) to learn the salience of image regions for place recognition by feature-less systems. SeqSLAM only slightly benefits from the results of training, on the challenging real world Eynsham dataset, as it already appears to filter less useful regions of a panoramic image. However, when recognition is limited to specific image regions performance improves by more than an order of magnitude by utilising the learnt image region saliency. We then investigate whether the region salience generated from the Eynsham dataset generalizes to another car-based dataset using a perspective camera. The results suggest the general applicability of an image region salience mask for optimizing route-based navigation applications.

Airborne collision scenario flight tests : impact of angle measurement errors on reactive vision-based avoidance control

The future emergence of many types of airborne vehicles and unpiloted aircraft in the national airspace means collision avoidance is of primary concern in an uncooperative airspace environment. The ability to replicate a pilot’s see and avoid capability using cameras coupled with vision based avoidance control is an important part of an overall collision avoidance strategy. But unfortunately without range collision avoidance has no direct way to guarantee a level of safety. Collision scenario flight tests with two aircraft and a monocular camera threat detection and tracking system were used to study the accuracy of image-derived angle measurements. The effect of image-derived angle errors on reactive vision-based avoidance performance was then studied by simulation. The results show that whilst large angle measurement errors can significantly affect minimum ranging characteristics across a variety of initial conditions and closing speeds, the minimum range is always bounded and a collision never occurs.

Disaster strikes, then what? Using evaluation in narrative driven (oral history & digital storytelling) community-based projects

In 2011 Queensland suffered both floods and cyclones, leaving residents without homes and their communities in ruins (2011). This paper presents how researchers from QUT, who are also members of the Oral History Association of Australia (OHAA) Queensland’s chapter, are using oral history, photographs, videography and digital storytelling to help heal and empower rural communities around the state and how evaluation has become a key element of our research. QUT researchers ran storytelling workshops in the capital city of Brisbane i early 2011, after the city suffered sever flooding. Cyclone Yasi then struck the town of Cardwell (in February 2011) destroying their historical museum and recording equipment. We delivered an 'emergency workshop', offering participants hands on use of the equipment, ethical and interviewing theory, so that the community could start to build a new collection. We included oral history workshops as well as sessions on how best to use a video camera, digital camera and creative writing sessions, so the community would also know how to make 'products' or exhibition pieces out of the interviews they were recording. We returned six months later to conduct follow-up workshops and the material produced by and with the community had been amazing. More funding has now been secured to replicate audio/visual/writing workshops in other remote rural Queensland communities including Townsville, Mackay and Cunnamulla and Toowoomba in 2012, highlighting the need for a multi media approach, to leverage the most out of OH interviews as a mechanism to restore and promote community resilience and pride.

Groundwater visualisation system (GVS) : a software framework for integrated display and interrogation of conceptual hydrogeological models, data and time-series animation

Management of groundwater systems requires realistic conceptual hydrogeological models as a framework for numerical simulation modelling, but also for system understanding and communicating this to stakeholders and the broader community. To help overcome these challenges we developed GVS (Groundwater Visualisation System), a stand-alone desktop software package that uses interactive 3D visualisation and animation techniques. The goal was a user-friendly groundwater management tool that could support a range of existing real-world and pre-processed data, both surface and subsurface, including geology and various types of temporal hydrological information. GVS allows these data to be integrated into a single conceptual hydrogeological model. In addition, 3D geological models produced externally using other software packages, can readily be imported into GVS models, as can outputs of simulations (e.g. piezometric surfaces) produced by software such as MODFLOW or FEFLOW. Boreholes can be integrated, showing any down-hole data and properties, including screen information, intersected geology, water level data and water chemistry. Animation is used to display spatial and temporal changes, with time-series data such as rainfall, standing water levels and electrical conductivity, displaying dynamic processes. Time and space variations can be presented using a range of contouring and colour mapping techniques, in addition to interactive plots of time-series parameters. Other types of data, for example, demographics and cultural information, can also be readily incorporated. The GVS software can execute on a standard Windows or Linux-based PC with a minimum of 2 GB RAM, and the model output is easy and inexpensive to distribute, by download or via USB/DVD/CD. Example models are described here for three groundwater systems in Queensland, northeastern Australia: two unconfined alluvial groundwater systems with intensive irrigation, the Lockyer Valley and the upper Condamine Valley, and the Surat Basin, a large sedimentary basin of confined artesian aquifers. This latter example required more detail in the hydrostratigraphy, correlation of formations with drillholes and visualisation of simulation piezometric surfaces. Both alluvial system GVS models were developed during drought conditions to support government strategies to implement groundwater management. The Surat Basin model was industry sponsored research, for coal seam gas groundwater management and community information and consultation. The “virtual” groundwater systems in these 3D GVS models can be interactively interrogated by standard functions, plus production of 2D cross-sections, data selection from the 3D scene, rear end database and plot displays. A unique feature is that GVS allows investigation of time-series data across different display modes, both 2D and 3D. GVS has been used successfully as a tool to enhance community/stakeholder understanding and knowledge of groundwater systems and is of value for training and educational purposes. Projects completed confirm that GVS provides a powerful support to management and decision making, and as a tool for interpretation of groundwater system hydrological processes. A highly effective visualisation output is the production of short videos (e.g. 2–5 min) based on sequences of camera ‘fly-throughs’ and screen images. Further work involves developing support for multi-screen displays and touch-screen technologies, distributed rendering, gestural interaction systems. To highlight the visualisation and animation capability of the GVS software, links to related multimedia hosted online sites are included in the references.

Supermassive

“Supermassive” is a synchronised four-channel video installation with sound. Each video channel shows a different camera view of an animated three-dimensional scene, which visually references galactic or astral imagery. This scene is comprised of forty-four separate clusters of slowly orbiting white text. Each cluster refers to a different topic that has been sourced online. The topics are diverse with recurring subjects relating to spirituality, science, popular culture, food and experiences of contemporary urban life. The slow movements of the text and camera views are reinforced through a rhythmic, contemplative soundtrack. As an immersive installation, “Supermassive” operates somewhere between a meditational mind map and a representation of a contemporary data stream. “Supermassive” contributes to studies in the field of contemporary art. It is particularly concerned with the ways that graphic representations of language can operate in the exploration of contemporary lived experiences, whether actual or virtual. Artists such as Ed Ruscha and Christopher Wool have long explored the emotive and psychological potentials of graphic text. Other artists such as Doug Aitken and Pipilotti Rist have engaged with the physical and spatial potentials of audio-visual installations to create emotive and symbolic experiences for their audiences. Using a practice-led research methodology, “Supermassive” extends these creative inquiries. By creating a reflective atmosphere in which divergent textual subjects are pictured together, the work explores not only how we navigate information, but also how such navigations inform understandings of our physical and psychological realities. “Supermassive” has been exhibited internationally at LA Louver Gallery, Venice, California in 2013 and nationally with GBK as part of Art Month Sydney, also in 2013. It has been critically reviewed in The Los Angeles Times.

Testing the accuracy of luminance maps acquired by smart phone cameras

This paper looks at the accuracy of using the built-in camera of smart phones and free software as an economical way to quantify and analyse light exposure by producing luminance maps from High Dynamic Range (HDR) images. HDR images were captured with an Apple iPhone 4S to capture a wide variation of luminance within an indoor and outdoor scene. The HDR images were then processed using Photosphere software (Ward, 2010.) to produce luminance maps, where individual pixel values were compared with calibrated luminance meter readings. This comparison has shown an average luminance error of ~8% between the HDR image pixel values and luminance meter readings, when the range of luminances in the image is limited to approximately 1,500cd/m2.

Analysis of regionally enhanced GPS orbit and clock solutions and contribution to improvement of real-time precise point positioning

This work experimentally examines the performance benefits of a regional CORS network to the GPS orbit and clock solutions for supporting real-time Precise Point Positioning (PPP). The regionally enhanced GPS precise orbit solutions are derived from a global evenly distributed CORS network added with a densely distributed network in Australia and New Zealand. A series of computational schemes for different network configurations are adopted in the GAMIT-GLOBK and PANDA data processing. The precise GPS orbit results show that the regionally enhanced solutions achieve the overall orbit improvements with respect to the solutions derived from the global network only. Additionally, the orbital differences over GPS satellite arcs that are visible by any of the five Australia-wide CORS stations show a higher percentage of overall improvements compared to the satellite arcs that are not visible from these stations. The regional GPS clock and Uncalibrated Phase Delay (UPD) products are derived using the PANDA real time processing module from Australian CORS networks of 35 and 79 stations respectively. Analysis of PANDA kinematic PPP and kinematic PPP-AR solutions show certain overall improvements in the positioning performance from a denser network configuration after solution convergence. However, the clock and UPD enhancement on kinematic PPP solutions is marginal. It is suggested that other factors, such as effects of ionosphere, incorrectly fixed ambiguities, may be the more dominating, deserving further research attentions.

Featureless visual processing for SLAM in changing outdoor environments

Vision-based SLAM is mostly a solved problem providing clear, sharp images can be obtained. However, in outdoor environments a number of factors such as rough terrain, high speeds and hardware limitations can result in these conditions not being met. High speed transit on rough terrain can lead to image blur and under/over exposure, problems that cannot easily be dealt with using low cost hardware. Furthermore, recently there has been a growth in interest in lifelong autonomy for robots, which brings with it the challenge in outdoor environments of dealing with a moving sun and lack of constant artificial lighting. In this paper, we present a lightweight approach to visual localization and visual odometry that addresses the challenges posed by perceptual change and low cost cameras. The approach combines low resolution imagery with the SLAM algorithm, RatSLAM. We test the system using a cheap consumer camera mounted on a small vehicle in a mixed urban and vegetated environment, at times ranging from dawn to dusk and in conditions ranging from sunny weather to rain. We first show that the system is able to provide reliable mapping and recall over the course of the day and incrementally incorporate new visual scenes from different times into an existing map. We then restrict the system to only learning visual scenes at one time of day, and show that the system is still able to localize and map at other times of day. The results demonstrate the viability of the approach in situations where image quality is poor and environmental or hardware factors preclude the use of visual features.

Cross-calbration of push-broom 2D LIDARs and cameras in natural scenes

This paper addresses the problem of automatically estimating the relative pose between a push-broom LIDAR and a camera without the need for artificial calibration targets or other human intervention. Further we do not require the sensors to have an overlapping field of view, it is enough that they observe the same scene but at different times from a moving platform. Matching between sensor modalities is achieved without feature extraction. We present results from field trials which suggest that this new approach achieves an extrinsic calibration accuracy of millimeters in translation and deci-degrees in rotation.

The relationship between hip abductor muscle strength and magnitude of pelvic drop following a 3 week strengthening protocol in non-specific low back pain patients.

Purpose: To examine the relationship between hip abductor muscle (HABD) strength and the magnitude of pelvic drop (MPD) for patients with non-specific low back pain (NSLBP) and controls (CON) prior to and following a 3-week HABD strengthening protocol. At baseline, we hypothesized that NSLBP patients would exhibit reduced HABD strength and greater MPD compared to CON. Following the protocol, we hypothesized that strength would increase and MPD would decrease. Relevance: The Trendelenburg test (TT) is a common clinical test used to examine the ability of the HABD to maintain horizontal pelvic position during single limb stance. However, no study has specifically tested this theory. Moreover, no study has investigated the relationship between HABD strength and pelvic motion during walking or tested whether increased HABD strength would reduce the MPD. Methods: Quasi-experimental with 3-week exercise intervention. Fifteen NSLBP patients (32.5yrs,range 21-51yrs; VAS baseline: 5.3cm) and 10 CON (29.5yrs,range 22-47yrs) were recruited. Isometric HABD strength was measured using a force dynamometer and the average of three maximal voluntary contractions were normalized to body mass (N/kg). Two-dimensional MPD (degrees) was measured using a 60 Hz camera and was derived from two retroreflective-markers placed on the posterior superior iliac spines. MPD was measured while performing the static TT and while walking and averaged over 10 consecutive footfalls. NSLBP patients completed a 3-week HABD strengthening protocol consisting of 2 open-kinetic-chain exercises then all measures were repeated. Non-parametric analysis was used for group comparisons and correlation analysis. Results: At baseline, the NSLBP patients demonstrated 31% reduced HABD strength (mean=6.6 N/kg) compared to CON (mean=9.5 N/kg: p=0.03) and no significant differences in maximal pelvic frontal plane excursion while walking (NSLBP:mean=8.1°, CON:mean=7.1°: p=0.72). No significant correlations were measured between left HABD strength and right MPD (r=-0.37, p=0.11), or between right HABD strength and left MPD (r=-0.04, p=0.84) while performing the static TT. Following the 3-week strengthening protocol, NSLBP patients demonstrated a 12% improvement in strength (Post:mean=7.4 N/kg: p=0.02), a reduction in pain (VAS followup: 2.8cm), but no significant decreases in MPD while walking (p=0.92). Conclusions: NSLBP patients demonstrated reduced HABD strength at baseline and were able to increase strength and reduce pain in a 3-week period. However, despite increases in HABD strength, the NSLBP group exhibited similar MPD motion during the static TT and while walking compared to baseline and controls. Implications: The results suggest that the HABD alone may not be primarily responsible for controlling a horizontal pelvic position during static and dynamic conditions. Increasing the strength of the hip abductors resulted in a reduction of pain in NSLBP patients providing evidence for further research to identify specific musculature responsible for controlling pelvic motion.

Long exposure localization in darkness using consumer cameras

In this paper we demonstrate passive vision-based localization in environments more than two orders of magnitude darker than the current benchmark using a 100 webcam and a 500 camera. Our approach uses the camera’s maximum exposure duration and sensor gain to achieve appropriately exposed images even in unlit night-time environments, albeit with extreme levels of motion blur. Using the SeqSLAM algorithm, we first evaluate the effect of variable motion blur caused by simulated exposures of 132 ms to 10000 ms duration on localization performance. We then use actual long exposure camera datasets to demonstrate day-night localization in two different environments. Finally we perform a statistical analysis that compares the baseline performance of matching unprocessed greyscale images to using patch normalization and local neighbourhood normalization – the two key SeqSLAM components. Our results and analysis show for the first time why the SeqSLAM algorithm is effective, and demonstrate the potential for cheap camera-based localization systems that function across extreme perceptual change.

Image-based visual navigation for mobile robots

We introduce a new image-based visual navigation algorithm that allows the Cartesian velocity of a robot to be defined with respect to a set of visually observed features corresponding to previously unseen and unmapped world points. The technique is well suited to mobile robot tasks such as moving along a road or flying over the ground. We describe the algorithm in general form and present detailed simulation results for an aerial robot scenario using a spherical camera and a wide angle perspective camera, and present experimental results for a mobile ground robot.

Online calibration of stereo rigs for long-term autonomy

Stereo-based visual odometry algorithms are heavily dependent on an accurate calibration of the rigidly fixed stereo pair. Even small shifts in the rigid transform between the cameras can impact on feature matching and 3D scene triangulation, adversely affecting pose estimates and applications dependent on long-term autonomy. In many field-based scenarios where vibration, knocks and pressure change affect a robotic vehicle, maintaining an accurate stereo calibration cannot be guaranteed over long periods. This paper presents a novel method of recalibrating overlapping stereo camera rigs from online visual data while simultaneously providing an up-to-date and up-to-scale pose estimate. The proposed technique implements a novel form of partitioned bundle adjustment that explicitly includes the homogeneous transform between a stereo camera pair to generate an optimal calibration. Pose estimates are computed in parallel to the calibration, providing online recalibration which seamlessly integrates into a stereo visual odometry framework. We present results demonstrating accurate performance of the algorithm on both simulated scenarios and real data gathered from a wide-baseline stereo pair on a ground vehicle traversing urban roads.

The use of electronic portal imaging to verify patient position during intensity-modulated radiotherapy delivered by the dynamic MLC technique

Purpose: The precise shape of the three-dimensional dose distributions created by intensity-modulated radiotherapy means that the verification of patient position and setup is crucial to the outcome of the treatment. In this paper, we investigate and compare the use of two different image calibration procedures that allow extraction of patient anatomy from measured electronic portal images of intensity-modulated treatment beams. Methods and Materials: Electronic portal images of the intensity-modulated treatment beam delivered using the dynamic multileaf collimator technique were acquired. The images were formed by measuring a series of frames or segments throughout the delivery of the beams. The frames were then summed to produce an integrated portal image of the delivered beam. Two different methods for calibrating the integrated image were investigated with the aim of removing the intensity modulations of the beam. The first involved a simple point-by-point division of the integrated image by a single calibration image of the intensity-modulated beam delivered to a homogeneous polymethyl methacrylate (PMMA) phantom. The second calibration method is known as the quadratic calibration method and required a series of calibration images of the intensity-modulated beam delivered to different thicknesses of homogeneous PMMA blocks. Measurements were made using two different detector systems: a Varian amorphous silicon flat-panel imager and a Theraview camera-based system. The methods were tested first using a contrast phantom before images were acquired of intensity-modulated radiotherapy treatment delivered to the prostate and pelvic nodes of cancer patients at the Royal Marsden Hospital. Results: The results indicate that the calibration methods can be used to remove the intensity modulations of the beam, making it possible to see the outlines of bony anatomy that could be used for patient position verification. This was shown for both posterior and lateral delivered fields. Conclusions: Very little difference between the two calibration methods was observed, so the simpler division method, requiring only the single extra calibration measurement and much simpler computation, was the favored method. This new method could provide a complementary tool to existing position verification methods, and it has the advantage that it is completely passive, requiring no further dose to the patient and using only the treatment fields.

Investigation of megavoltage beams and detectors for electronic portal imaging

Accuracy of dose delivery in external beam radiotherapy is usually verified with electronic portal imaging (EPI) in which the treatment beam is used to check the positioning of the patient. However the resulting megavoltage x-ray images suffer from poor quality. The image quality can be improved by developing a special operating mode in the linear accelerator. The existing treatment beam is modified such that it produces enough low-energy photons for imaging. In this work the problem of optimizing the beam/detector combination to achieve optimal electronic portal image quality is addressed. The linac used for this study was modified to produce two experimental photon beams. These beams, named Al6 and Al10, were non-flat and were produced by 4MeV electrons hitting aluminum targets, 6 and 10mm thick respectively. The images produced by a conventional EPI system (6MV treatment beam and camera-based EPID with a Cu plate & Gd2O2S screen ) were compared with the images produced by the experimental beams and various screens with the same camera). The contrast of 0.8cm bone equivalent material in 5 cm water increased from 1.5% for the conventional system to 11% for the combination of Al6 beam with a 200mg/cm2 Gd2O2S screen. The signal-to-noise ratio calculated for 1cGy flood field images increased by about a factor of two for the same EPI systems. The spatial resolution of the two imaging systems was comparable. This work demonstrates that significant improvements in portal image contrast can be obtained by simultaneous optimization of the linac spectrum and EPI detector.