Vision only pose estimation and scene reconstruction on airborne platforms

We aim to demonstrate unaided visual 3D pose estimation and map reconstruction using both monocular and stereo vision techniques. To date, our work has focused on collecting data from Unmanned Aerial Vehicles, which generates a number of significant issues specific to the application. Such issues include scene reconstruction degeneracy from planar data, poor structure initialisation for monocular schemes and difficult 3D reconstruction due to high feature covariance. Most modern Visual Odometry (VO) and related SLAM systems make use of a number of sensors to inform pose and map generation, including laser range-finders, radar, inertial units and vision [1]. By fusing sensor inputs, the advantages and deficiencies of each sensor type can be handled in an efficient manner. However, many of these sensors are costly and each adds to the complexity of such robotic systems. With continual advances in the abilities, small size, passivity and low cost of visual sensors along with the dense, information rich data that they provide our research focuses on the use of unaided vision to generate pose estimates and maps from robotic platforms. We propose that highly accurate (�5cm) dense 3D reconstructions of large scale environments can be obtained in addition to the localisation of the platform described in other work [2]. Using images taken from cameras, our algorithm simultaneously generates an initial visual odometry estimate and scene reconstruction from visible features, then passes this estimate to a bundle-adjustment routine to optimise the solution. From this optimised scene structure and the original images, we aim to create a detailed, textured reconstruction of the scene. By applying such techniques to a unique airborne scenario, we hope to expose new robotic applications of SLAM techniques. The ability to obtain highly accurate 3D measurements of an environment at a low cost is critical in a number of agricultural and urban monitoring situations. We focus on cameras as such sensors are small, cheap and light-weight and can therefore be deployed in smaller aerial vehicles. This, coupled with the ability of small aerial vehicles to fly near to the ground in a controlled fashion, will assist in increasing the effective resolution of the reconstructed maps.

Inferior visual field reductions are associated with poorer functional status among older adults with glaucoma

Purpose: To examine the relationship between visual impairment and functional status in a community-dwelling sample of older adults with glaucoma. Methods: This study included 74 community-dwelling older adults with open-angle glaucoma (aged 74 ± 6 years). Assessment of central vision included high-contrast visual acuity and Pelli-Robson contrast sensitivity. Binocular integrated visual fields were derived from merged monocular Humphrey Field Analyser visual field plots. Functional status outcome measures included physical performance tests (6-min walk test, timed up and go test and lower limb strength), a physical activity questionnaire (Physical Activity Scale for the Elderly) and an overall functional status score. Correlation and linear regression analyses, adjusting for age and gender, examined the association between visual impairment and functional status outcomes. Results: Greater levels of visual impairment were significantly associated with lower levels of functional status among community-dwelling older adults with glaucoma, independent of age and gender. Specifically, lower levels of visual function were associated with slower timed up and go performance, weaker lower limb strength, lower self-reported physical activity, and lower overall functional status scores. Of the components of vision examined, the inferior visual field and contrast factors were the strongest predictors of these functional outcomes, whereas the superior visual field factor was not related to functional status. Conclusions: Greater visual impairment, particularly in the inferior visual field and loss of contrast sensitivity, was associated with poorer functional status among older adults with glaucoma. The findings of this study highlight the potential links between visual impairment and the onset of functional decline. Interventions which promote physical activity among older adults with glaucoma may assist in preventing functional decline, frailty and falls, and improve overall health and well-being.

Image-based safety assessment : automated spatial safety risk identification of earthmoving and surface mining activities

This paper presents an automated image‐based safety assessment method for earthmoving and surface mining activities. The literature review revealed the possible causes of accidents on earthmoving operations, investigated the spatial risk factors of these types of accident, and identified spatial data needs for automated safety assessment based on current safety regulations. Image‐based data collection devices and algorithms for safety assessment were then evaluated. Analysis methods and rules for monitoring safety violations were also discussed. The experimental results showed that the safety assessment method collected spatial data using stereo vision cameras, applied object identification and tracking algorithms, and finally utilized identified and tracked object information for safety decision making.

Risk of falls, injurious falls, and other injuries resulting from visual impairment among older adults with age-related macular degeneration

Purpose: Age-related macular degeneration (AMD) is the leading cause of irreversible visual impairment among older adults. This study explored the relationship between AMD, falls risk and other injuries and identified visual risk factors for these adverse events. Methods: Participants included 76 community-dwelling individuals with a range of severity of AMD (mean age, 77.0±6.9 years). Baseline assessment included binocular visual acuity, contrast sensitivity and merged visual fields. Participants completed monthly falls and injury diaries for one year following the baseline assessment. Results: Overall, 74% of participants reported having either a fall, injurious fall or other injury. Fifty-four percent of participants reported a fall and 30% reported more than one fall; of the 102 falls reported, 63% resulted in an injury. Most occurred outdoors (52%), between late morning and late afternoon (61%) and when navigating on level ground (62%). The most common non-fall injuries were lacerations (36%) and collisions with an object (35%). Reduced contrast sensitivity and visual acuity were associated with increased fall rate, after controlling for age, gender, cognitive function, cataract severity and self-reported physical function. Reduced contrast sensitivity was the only significant predictor of falls and other injuries. Conclusion: Among older adults with AMD, increased visual impairment was significantly associated with an increased incidence of falls and other injuries. Reduced contrast sensitivity was significantly associated with increased rates of falls, injurious falls and injuries, while reduced visual acuity was only associated with increased falls risk. These findings have important implications for the assessment of visually impaired older adults.

Navigating, recognizing and describing urban spaces with vision and lasers

In this paper we describe a body of work aimed at extending the reach of mobile navigation and mapping. We describe how running topological and metric mapping and pose estimation processes concurrently, using vision and laser ranging, has produced a full six-degree-of-freedom outdoor navigation system. It is capable of producing intricate three-dimensional maps over many kilometers and in real time. We consider issues concerning the intrinsic quality of the built maps and describe our progress towards adding semantic labels to maps via scene de-construction and labeling. We show how our choices of representation, inference methods and use of both topological and metric techniques naturally allow us to fuse maps built from multiple sessions with no need for manual frame alignment or data association.

Gait energy volumes and frontal gait recognition using depth images

Gait energy images (GEIs) and its variants form the basis of many recent appearance-based gait recognition systems. The GEI combines good recognition performance with a simple implementation, though it suffers problems inherent to appearance-based approaches, such as being highly view dependent. In this paper, we extend the concept of the GEI to 3D, to create what we call the gait energy volume, or GEV. A basic GEV implementation is tested on the CMU MoBo database, showing improvements over both the GEI baseline and a fused multi-view GEI approach. We also demonstrate the efficacy of this approach on partial volume reconstructions created from frontal depth images, which can be more practically acquired, for example, in biometric portals implemented with stereo cameras, or other depth acquisition systems. Experiments on frontal depth images are evaluated on an in-house developed database captured using the Microsoft Kinect, and demonstrate the validity of the proposed approach.

Inferior field loss increases rate of falls in older adults with glaucoma

PURPOSE: To examine the visual predictors of falls and injurious falls among older adults with glaucoma. METHODS: Prospective falls data were collected for 71 community-dwelling adults with primary open-angle glaucoma, mean age 73.9 ± 5.7 years, for one year using monthly falls diaries. Baseline assessment of central visual function included high-contrast visual acuity and Pelli-Robson contrast sensitivity. Binocular integrated visual fields were derived from monocular Humphrey Field Analyser plots. Rate ratios (RR) for falls and injurious falls with 95% confidence intervals (CIs) were based on negative binomial regression models. RESULTS: During the one year follow-up, 31 (44%) participants experienced at least one fall and 22 (31%) experienced falls that resulted in an injury. Greater visual impairment was associated with increased falls rate, independent of age and gender. In a multivariate model, more extensive field loss in the inferior region was associated with higher rate of falls (RR 1.57, 95%CI 1.06, 2.32) and falls with injury (RR 1.80, 95%CI 1.12, 2.98), adjusted for all other vision measures and potential confounding factors. Visual acuity, contrast sensitivity, and superior field loss were not associated with the rate of falls; topical beta-blocker use was also not associated with increased falls risk. CONCLUSIONS: Falls are common among older adults with glaucoma and occur more frequently in those with greater visual impairment, particularly in the inferior field region. This finding highlights the importance of the inferior visual field region in falls risk and assists in identifying older adults with glaucoma at risk of future falls, for whom potential interventions should be targeted. KEY WORDS: glaucoma, visual field, visual impairment, falls, injury

Driving speed is altered by monocular Neutral Density filters : the Enright phenomenon

Introduction: An observer, looking sideways from a moving vehicle, while wearing a neutral density filter over one eye, can have a distorted perception of speed, known as the Enright phenomenon. The purpose of this study was to determine how the Enright phenomenon influences driving behaviour. Methods: A geometric model of the Enright phenomenon was developed. Ten young, visually normal, participants (mean age = 25.4 years) were tested on a straight section of a closed driving circuit and instructed to look out of the right side of the vehicle and drive at either 40 Km/h or 60 Km/h under the following binocular viewing conditions: with a 0.9 ND filter over the left eye (leading eye); 0.9 ND filter over the right eye (trailing eye); 0.9 ND filters over both eyes, and with no filters over either eye. The order of filter conditions was randomised and the speed driven recorded for each condition. Results: Speed judgments did not differ significantly between the two baseline conditions (no filters and both eyes filtered) for either speed tested. For the baseline conditions, when subjects were asked to drive at 60 Km/h they matched this speed well (61 ± 10.2 Km/h) but drove significantly faster than requested (51.6 ± 9.4 Km/h) when asked to drive at 40 Km/h. Subjects significantly exceeded baseline speeds by 8.7± 5.0 Km/h, when the trailing eye was filtered and travelled slower than baseline speeds by 3.7± 4.6 Km/h when the leading eye was filtered. Conclusions: This is the first quantitative study demonstrating how the Enright effect can influence perceptions of driving speed, and demonstrates that monocular filtering of an eye can significantly impact driving speeds, albeit to a lesser extent than predicted by geometric models of the phenomenon.

Towards automatic object segmentation with sequential multiple views

Object segmentation is one of the fundamental steps for a number of robotic applications such as manipulation, object detection, and obstacle avoidance. This paper proposes a visual method for incorporating colour and depth information from sequential multiview stereo images to segment objects of interest from complex and cluttered environments. Rather than segmenting objects using information from a single frame in the sequence, we incorporate information from neighbouring views to increase the reliability of the information and improve the overall segmentation result. Specifically, dense depth information of a scene is computed using multiple view stereo. Depths from neighbouring views are reprojected into the reference frame to be segmented compensating for imperfect depth computations for individual frames. The multiple depth layers are then combined with color information from the reference frame to create a Markov random field to model the segmentation problem. Finally, graphcut optimisation is employed to infer pixels belonging to the object to be segmented. The segmentation accuracy is evaluated over images from an outdoor video sequence demonstrating the viability for automatic object segmentation for mobile robots using monocular cameras as a primary sensor.

Feature-based visual odometry and featureless place recognition for SLAM in 2.5D environments

In this paper we present a novel algorithm for localization during navigation that performs matching over local image sequences. Instead of calculating the single location most likely to correspond to a current visual scene, the approach finds candidate matching locations within every section (subroute) of all learned routes. Through this approach, we reduce the demands upon the image processing front-end, requiring it to only be able to correctly pick the best matching image from within a short local image sequence, rather than globally. We applied this algorithm to a challenging downhill mountain biking visual dataset where there was significant perceptual or environment change between repeated traverses of the environment, and compared performance to applying the feature-based algorithm FAB-MAP. The results demonstrate the potential for localization using visual sequences, even when there are no visual features that can be reliably detected.

A semi-local method for iterative depth-map refinement

We present an iterative hierarchical algorithm for multi-view stereo. The algorithm attempts to utilise as much contextual information as is available to compute highly accurate and robust depth maps. There are three novel aspects to the approach: 1) firstly we incrementally improve the depth fidelity as the algorithm progresses through the image pyramid; 2) secondly we show how to incorporate visual hull information (when available) to constrain depth searches; and 3) we show how to simultaneously enforce the consistency of the depth-map by continual comparison with neighbouring depth-maps. We show that this approach produces highly accurate depth-maps and, since it is essentially a local method, is both extremely fast and simple to implement.

Statistical eye model for normal eyes

Purpose. To create a binocular statistical eye model based on previously measured ocular biometric data. Methods. Thirty-nine parameters were determined for a group of 127 healthy subjects (37 male, 90 female; 96.8% Caucasian) with an average age of 39.9 ± 12.2 years and spherical equivalent refraction of −0.98 ± 1.77 D. These parameters described the biometry of both eyes and the subjects' age. Missing parameters were complemented by data from a previously published study. After confirmation of the Gaussian shape of their distributions, these parameters were used to calculate their mean and covariance matrices. These matrices were then used to calculate a multivariate Gaussian distribution. From this, an amount of random biometric data could be generated, which were then randomly selected to create a realistic population of random eyes. Results. All parameters had Gaussian distributions, with the exception of the parameters that describe total refraction (i.e., three parameters per eye). After these non-Gaussian parameters were omitted from the model, the generated data were found to be statistically indistinguishable from the original data for the remaining 33 parameters (TOST [two one-sided t tests]; P < 0.01). Parameters derived from the generated data were also significantly indistinguishable from those calculated with the original data (P > 0.05). The only exception to this was the lens refractive index, for which the generated data had a significantly larger SD. Conclusions. A statistical eye model can describe the biometric variations found in a population and is a useful addition to the classic eye models.

The way

In this wall-mounted sculpture, a car stereo is mounted into a photographic image of a redwood forest. It plays a sparse and evocative guitar soundtrack. The supporting cabinet is finished with timber veneer to resemble a retro home stereo or piece of designer furniture. This work examines how we construct, represent and deploy notions of nature in our contemporary lives. It mixes the languages of furniture design, landscape photography and sculpture. Drawing on Zygmunt Bauman’s theoretical work on “liquid modernity”, this work questions how and where we find space for contemplation and reflection in a contemporary context increasingly defined by temporary social bonds and consumer choices.

Integrated vision-based attitude estimation

For many years, computer vision has lured researchers with promises of a low-cost, passive, lightweight and information-rich sensor suitable for navigation purposes. The prime difficulty in vision-based navigation is that the navigation solution will continually drift with time unless external information is available, whether it be cues from the appearance of the scene, a map of features (whether built online or known a priori), or from an externally-referenced sensor. It is not merely position that is of interest in the navigation problem. Attitude (i.e. the angular orientation of a body with respect to a reference frame) is integral to a visionbased navigation solution and is often of interest in its own right (e.g. flight control). This thesis examines vision-based attitude estimation in an aerospace environment, and two methods are proposed for constraining drift in the attitude solution; one through a novel integration of optical flow and the detection of the sky horizon, and the other through a loosely-coupled integration of Visual Odometry and GPS position measurements. In the first method, roll angle, pitch angle and the three aircraft body rates are recovered though a novel method of tracking the horizon over time and integrating the horizonderived attitude information with optical flow. An image processing front-end is used to select several candidate lines in a image that may or may not correspond to the true horizon, and the optical flow is calculated for each candidate line. Using an Extended Kalman Filter (EKF), the previously estimated aircraft state is propagated using a motion model and a candidate horizon line is associated using a statistical test based on the optical flow measurements and location of the horizon in the image. Once associated, the selected horizon line, along with the associated optical flow, is used as a measurement to the EKF. To evaluate the accuracy of the algorithm, two flights were conducted, one using a highly dynamic Uninhabited Airborne Vehicle (UAV) in clear flight conditions and the other in a human-piloted Cessna 172 in conditions where the horizon was partially obscured by terrain, haze and smoke. The UAV flight resulted in pitch and roll error standard deviations of 0.42° and 0.71° respectively when compared with a truth attitude source. The Cessna 172 flight resulted in pitch and roll error standard deviations of 1.79° and 1.75° respectively. In the second method for estimating attitude, a novel integrated GPS/Visual Odometry (GPS/VO) navigation filter is proposed, using a structure similar to a classic looselycoupled GPS/INS error-state navigation filter. Under such an arrangement, the error dynamics of the system are derived and a Kalman Filter is developed for estimating the errors in position and attitude. Through similar analysis to the GPS/INS problem, it is shown that the proposed filter is capable of recovering the complete attitude (i.e. pitch, roll and yaw) of the platform when subjected to acceleration not parallel to velocity for both the monocular and stereo variants of the filter. Furthermore, it is shown that under general straight line motion (e.g. constant velocity), only the component of attitude in the direction of motion is unobservable. Numerical simulations are performed to demonstrate the observability properties of the GPS/VO filter in both the monocular and stereo camera configurations. Furthermore, the proposed filter is tested on imagery collected using a Cessna 172 to demonstrate the observability properties on real-world data. The proposed GPS/VO filter does not require additional restrictions or assumptions such as platform-specific dynamics, map-matching, feature-tracking, visual loop-closing, gravity vector or additional sensors such as an IMU or magnetic compass. Since no platformspecific dynamics are required, the proposed filter is not limited to the aerospace domain and has the potential to be deployed in other platforms such as ground robots or mobile phones.

GABAergic modification of myopic and hyperopic ocular tissue effects on scleral fibroblast growth

Purpose: Myopia is a common eye disorder affecting up to 90% of children in South East Asia and 30% of the population worldwide. Myopia of high severity is a leading cause of blindness around the world (4th to 5th most common). Changes and remodelling of the sclera i.e. increase cellular proliferation & increase protein synthesis within scleral cells (↑ scleral DNA) and thinning and lose of extracellular matrix of sclera (↓ scleral GAG synthesis) have been linked to myopic eye growth in animal models. Signals acting on the sclera are thought to originate in the retina, and are modulated by the retinal pigment epithelium (RPE) with limited evidence suggesting that the RPE can modify scleral cell growth in culture. However, the mechanism of retinal signal transmission and the role of posterior eye cup tissue, including the RPE, in mediating changes in scleral fibroblast growth during myopia development are unclear. Retinal transmitter systems are critically involved in pathways regulating eye growth, which ultimately lead to alterations in the sclera if eye size is to change. A dopaminergic agonist and muscarinic antagonists decrease the proliferation of scleral chondrocytes when co-cultured with chick’s retinal pigment epithelium (RPE). GABA receptors have recently been localised to chick sclera. We therefore hypothesised that posterior eye cup tissue from myopic eyes would stimulate and from hyperopic eyes would inhibit growth of scleral fibroblasts in vitro and that GABAergic agents could directly interact with scleral cells or indirectly modify the effects of myopic and hyperopic posterior eye cup tissue on scleral fibroblast growth. Method: Fibroblastic cells obtained from 8-day-old chick sclera were used to establish cell banks. Two major experiments were performed. Experiment 1: To determine if posterior eye cup tissues from myopic eye stimulates and hyperopic eye inhibits scleral cell proliferation, when co-cultured with scleral cells in vitro. This study comprised two linked experiments, i) monocular visual treatments of FDM (form-deprivation myopia), LIM (lens-induced myopia) and LIH (lens-induced hyperopia) with assessment of the effect of full punch eye cup tissue on DNA and GAG synthesis by cultured chick scleral fibroblasts, and ii) binocular visual treatments comprising LIM and LIH with assessment of the effect of individual layers of eye cup tissues (neural retina, RPE and choroid) on cultured chick scleral fibroblasts. Visual treatment was applied for 3 days. Experiment 2: To determine the direct interaction of GABA agents on scleral cell growth and to establish whether GABA agents modify the stimulatory/inhibitory effect of myopic and hyperopic posterior eye cup tissues on cultured scleral cell growth in vitro. Two linked experiments were performed. i) GABA agonists (muscimol and baclofen) and GABA antagonists (bicuculine (-), CGP46381 and TPMPA) were added to scleral cell culture medium to determine their direct effect on scleral cells. ii) GABAergic agents (agonists and antagonists) were administered to scleral fibroblasts co-cultured with posterior eye cup tissue (retina, RPE, retina/RPE, RPE/choroid). Ocular tissues were obtained from chick eyes wearing +15D (LIH) or -15D lenses (LIM) for 3 days. In both experiments, tissues were added to hanging cell culture insert (pore size 1.0ìm) placed over each well of 24 well plates while scleral cells were cultured in DMEM/F12, Glutamax (Gibco) plus 10% FBS and penicillin/streptomycin (50U/ml)) and fungizone (1.25ug/ml) (Gibco), at seeding density of 30,000 cells/well at the bottom of the well and allowed to grow for 3 days. Scleral cells proliferation rate throughout the study was evaluated by determining GAG and DNA content of scleral cells using Dimethylmethylene blue (DMMB) dye and Quant-iTTm Pico Green® dsDNA reagent respectively. Results and analysis: Based on DNA and GAG content, there was no significant difference in tissue effect of LIM and LIH eyes on scleral fibroblast growth (DNA: 8.4 ± 1.1μg versus 9.3 ± 2.3 μg, p=0.23; GAG: 10.13 ± 1.4 μg versus 12.67 ± 1.2 μg, F2,23=6.16, p=0.0005) when tissues were obtained from monocularly treated chick eyes (FDM or +15D lens or -15D lens over right eyes with left eyes untreated) and co-cultured as full punch. When chick eyes were treated binocularly with -15D lens (LIM) right eye and +15D lens (LIH) left eyes and tissue layers were separated, the retina from LIM eyes did not stimulate scleral cell proliferation compared to LIH eyes (DNA: 27.2 ± 6.7 μg versus 23.2 ± 1.5 μg, p=0.23; GAG: 28.1 ±3.7 μg versus 28.7 ± 4.2 μg, p=0.21). Similarly, the LIH and LIM choroid did not produce a differential effect based on DNA (LIM 46.9 ± 6.4 μg versus LIH 53.5 ± 4.7 μg, p=0.18), however the choroid from LIH eyes induced higher scleral GAG content than from LIM eyes (32.5 ± 6.7 μg versus 18.9 ± 1.2 μg, p=0.023). In contrast, the RPE from LIM eyes caused a significant increase in fibroblast proliferation whereas the RPE from LIH eyes was relatively inhibitory (72.4 ± 6.3 μg versus 27.9 ± 2.3 μg, F1, 6=69.99, p=0.0005). GAG data were opposite to DNA data e.g. the RPE from LIH eyes increased (33.7 ± 7.9 μg) while the RPE from LIM eyes decreased (28.2 ± 3.0 μg) scleral cell growth (F1, 6=13.99, p=0.010). Based on DNA content, GABA agents had a small direct effect on scleral cell growth; GABA agonists increased (21.4 ± 1.0% and 18.3 ± 1.0% with muscimol and baclofen, p=0.0021), whereas GABA antagonists decreased fibroblast proliferation (-23.7 ± 0.9% with bicuculine & CGP46381 and -28.1 ± 0.5% with TPMPA, p=0.0004). GABA agents also modified the effect of LIM and LIH tissues (p=0.0005).The increase in proliferation rate of scleral fibroblasts co-cultured with tissues (RPE, retina, RPE/retina and RPE/choroid) from LIM treated eyes was enhanced by GABA agonists (muscimol: 27.4 ± 1.2%, 35.8 ± 1.6%, 8.4 ± 0.3% and 11.9 ± 0.6%; baclofen: 27.0 ± 1.0%, 15.8 ± 1.5%, 16.8 ± 1.2% and 15.4 ± 0.4%, p=0.014) whereas GABA antagonists further reduced scleral fibroblasts growth (bicuculine: -52.5 ± 2.5%, -36.9 ± 1.4%, -37.5 ± 0.6% and -53.7 ± 0.9%; TPMPA: 57.3 ± 1.3%, -15.7 ± 1.2%, -33.5 ± 0.4% and -45.9 ± 1.5%; CGP46381: -51.9 ± 1.6%, -28.5 ± 1.5%, -25.4 ± 2.0% and -45.5 ± 1.9% respectively, p=0.0034). GAG data were opposite to DNA data throughout the experiment e.g. GABA agonists further inhibited while antagonists relatively enhanced scleral fibroblasts growth for both LIM and LIH tissue co-culture. The effect of GABA agents was relatively lower (p=0.0004) for tissue from LIH versus LIM eyes but was in a similar direction. There was a significant drug effect on all four tissue types e.g. RPE, retina, RPE/retina and RPE/choroid for both LIM and LIH tissue co-culture (F20,92=3.928, p=0.0005). However, the effect of GABA agents was greatest in co-culture with RPE tissue (F18,36=4.865, p=0.0005). Summary and Conclusion: 1) Retinal defocus signals are transferred to RPE and choroid which then exert their modifying effect on scleral GAG and DNA synthesis either through growth stimulating factors or directly interacting with scleral cells in process of scleral remodeling during LIM and LIH visual conditions. 2) GABAergic agents affect the proliferation of scleral fibroblasts both directly and when co-cultured with ocular tissues in vitro.