Automatic object segmentation of unstructured scenes using colour and depth maps

This study presents a segmentation pipeline that fuses colour and depth information to automatically separate objects of interest in video sequences captured from a quadcopter. Many approaches assume that cameras are static with known position, a condition which cannot be preserved in most outdoor robotic applications. In this study, the authors compute depth information and camera positions from a monocular video sequence using structure from motion and use this information as an additional cue to colour for accurate segmentation. The authors model the problem similarly to standard segmentation routines as a Markov random field and perform the segmentation using graph cuts optimisation. Manual intervention is minimised and is only required to determine pixel seeds in the first frame which are then automatically reprojected into the remaining frames of the sequence. The authors also describe an automated method to adjust the relative weights for colour and depth according to their discriminative properties in each frame. Experimental results are presented for two video sequences captured using a quadcopter. The quality of the segmentation is compared to a ground truth and other state-of-the-art methods with consistently accurate results.

Nonparametric semantic segmentation for 3D street scenes

In this paper we propose a method to generate a large scale and accurate dense 3D semantic map of street scenes. A dense 3D semantic model of the environment can significantly improve a number of robotic applications such as autonomous driving, navigation or localisation. Instead of using offline trained classifiers for semantic segmentation, our approach employs a data-driven, nonparametric method to parse scenes which easily scale to a large environment and generalise to different scenes. We use stereo image pairs collected from cameras mounted on a moving car to produce dense depth maps which are combined into a global 3D reconstruction using camera poses from stereo visual odometry. Simultaneously, 2D automatic semantic segmentation using a nonparametric scene parsing method is fused into the 3D model. Furthermore, the resultant 3D semantic model is improved with the consideration of moving objects in the scene. We demonstrate our method on the publicly available KITTI dataset and evaluate the performance against manually generated ground truth.

Aircraft collision avoidance using spherical visual predictive control and single point features

This paper presents practical vision-based collision avoidance for objects approximating a single point feature. Using a spherical camera model, a visual predictive control scheme guides the aircraft around the object along a conical spiral trajectory. Visibility, state and control constraints are considered explicitly in the controller design by combining image and vehicle dynamics in the process model, and solving the nonlinear optimization problem over the resulting state space. Importantly, range is not required. Instead, the principles of conical spiral motion are used to design an objective function that simultaneously guides the aircraft along the avoidance trajectory, whilst providing an indication of the appropriate point to stop the spiral behaviour. Our approach is aimed at providing a potential solution to the See and Avoid problem for unmanned aircraft and is demonstrated through a series.

Robust scale initialization for long-range stereo visual odometry

Achieving a robust, accurately scaled pose estimate in long-range stereo presents significant challenges. For large scene depths, triangulation from a single stereo pair is inadequate and noisy. Additionally, vibration and flexible rigs in airborne applications mean accurate calibrations are often compromised. This paper presents a technique for accurately initializing a long-range stereo VO algorithm at large scene depth, with accurate scale, without explicitly computing structure from rigidly fixed camera pairs. By performing a monocular pose estimate over a window of frames from a single camera, followed by adding the secondary camera frames in a modified bundle adjustment, an accurate, metrically scaled pose estimate can be found. To achieve this the scale of the stereo pair is included in the optimization as an additional parameter. Results are presented both on simulated and field gathered data from a fixed-wing UAV flying at significant altitude, where the epipolar geometry is inaccurate due to structural deformation and triangulation from a single pair is insufficient. Comparisons are made with more conventional VO techniques where the scale is not explicitly optimized, and demonstrated over repeated trials to indicate robustness.

Visual guidance for fixed-wing unmanned aerial vehicles using feature tracking : application to power line inspection

This thesis presents novel vision based control solutions that enable fixed-wing Unmanned Aerial Vehicles to perform tasks of inspection over infrastructure including power lines, pipe lines and roads. This is achieved through the development of techniques that combine visual servoing with alternate manoeuvres that assist the UAV in both following and observing the feature from a downward facing camera. Control designs are developed through techniques of Image Based Visual Servoing to utilise sideslip through Skid-to-Turn and Forward-Slip manoeuvres. This allows the UAV to simultaneously track and collect data over the length of infrastructure, including straight segments and the transition where these meet.

A combined method for mitigating sparsity problem in tag recommendation

Tag recommendation is a specific recommendation task for recommending metadata (tag) for a web resource (item) during user annotation process. In this context, sparsity problem refers to situation where tags need to be produced for items with few annotations or for user who tags few items. Most of the state of the art approaches in tag recommendation are rarely evaluated or perform poorly under this situation. This paper presents a combined method for mitigating sparsity problem in tag recommendation by mainly expanding and ranking candidate tags based on similar items’ tags and existing tag ontology. We evaluated the approach on two public social bookmarking datasets. The experiment results show better accuracy for recommendation in sparsity situation over several state of the art methods.

Effect of temperature distribution on predicting quality of microwave dehydrated food

During food drying, many other changes occur simultaneously, resulting in an improved overall quality. Among the quality attributes, the structure and its corresponding color influence directly or indirectly other properties of food. In addition, these quality attributes are affected by process conditions, material components and the raw structure of the foodstuff. In this work, the temperature distribution within food materials during microwave drying has been taken into consideration to observe its role in color modification. In order to determine the temperature distribution of microwave-dried food (apple), a thermal imaging camera has been used. The image acquired from the digital camera has been analysed using image J software in order to get the color change of fresh and dried apple. The results show that temperature distribution plays an important role in determining the quality of the food. The thermal imaging camera was deployed to observe the temperature distribution within food materials during drying. It is clearly observed from the higher value of (ERGB =102) and the uneven color change that uneven temperature distribution can influence customer perceptions of the quality of dried food. Simulation of a mathematical model of temperature distribution during microwave drying can make it possible to predict the colour and texture of the microwaved food.

Person re-identification using group information

After first observing a person, the task of person re-identification involves recognising an individual at different locations across a network of cameras at a later time. Traditionally, this task has been performed by first extracting appearance features of an individual and then matching these features to the previous observation. However, identifying an individual based solely on appearance can be ambiguous, particularly when people wear similar clothing (i.e. people dressed in uniforms in sporting and school settings). This task is made more difficult when the resolution of the input image is small as is typically the case in multi-camera networks. To circumvent these issues, we need to use other contextual cues. In this paper, we use "group" information as our contextual feature to aid in the re-identification of a person, which is heavily motivated by the fact that people generally move together as a collective group. To encode group context, we learn a linear mapping function to assign each person to a "role" or position within the group structure. We then combine the appearance and group context cues using a weighted summation. We demonstrate how this improves performance of person re-identification in a sports environment over appearance based-features.

Distributed low-power image processing in wireless sensor networks for intelligent video surveillance applications

Distributed Wireless Smart Camera (DWSC) network is a special type of Wireless Sensor Network (WSN) that processes captured images in a distributed manner. While image processing on DWSCs sees a great potential for growth, with its applications possessing a vast practical application domain such as security surveillance and health care, it suffers from tremendous constraints. In addition to the limitations of conventional WSNs, image processing on DWSCs requires more computational power, bandwidth and energy that presents significant challenges for large scale deployments. This dissertation has developed a number of algorithms that are highly scalable, portable, energy efficient and performance efficient, with considerations of practical constraints imposed by the hardware and the nature of WSN. More specifically, these algorithms tackle the problems of multi-object tracking and localisation in distributed wireless smart camera net- works and optimal camera configuration determination. Addressing the first problem of multi-object tracking and localisation requires solving a large array of sub-problems. The sub-problems that are discussed in this dissertation are calibration of internal parameters, multi-camera calibration for localisation and object handover for tracking. These topics have been covered extensively in computer vision literatures, however new algorithms must be invented to accommodate the various constraints introduced and required by the DWSC platform. A technique has been developed for the automatic calibration of low-cost cameras which are assumed to be restricted in their freedom of movement to either pan or tilt movements. Camera internal parameters, including focal length, principal point, lens distortion parameter and the angle and axis of rotation, can be recovered from a minimum set of two images of the camera, provided that the axis of rotation between the two images goes through the camera's optical centre and is parallel to either the vertical (panning) or horizontal (tilting) axis of the image. For object localisation, a novel approach has been developed for the calibration of a network of non-overlapping DWSCs in terms of their ground plane homographies, which can then be used for localising objects. In the proposed approach, a robot travels through the camera network while updating its position in a global coordinate frame, which it broadcasts to the cameras. The cameras use this, along with the image plane location of the robot, to compute a mapping from their image planes to the global coordinate frame. This is combined with an occupancy map generated by the robot during the mapping process to localised objects moving within the network. In addition, to deal with the problem of object handover between DWSCs of non-overlapping fields of view, a highly-scalable, distributed protocol has been designed. Cameras that follow the proposed protocol transmit object descriptions to a selected set of neighbours that are determined using a predictive forwarding strategy. The received descriptions are then matched at the subsequent camera on the object's path using a probability maximisation process with locally generated descriptions. The second problem of camera placement emerges naturally when these pervasive devices are put into real use. The locations, orientations, lens types etc. of the cameras must be chosen in a way that the utility of the network is maximised (e.g. maximum coverage) while user requirements are met. To deal with this, a statistical formulation of the problem of determining optimal camera configurations has been introduced and a Trans-Dimensional Simulated Annealing (TDSA) algorithm has been proposed to effectively solve the problem.

Optical and biomechanical factors, associated with near work and myopia

Near work may play an important role in the development of myopia in the younger population. The prevalence of myopia has also been found to be higher in occupations that involve substantial near work tasks, for example in microscopists and textile workers. When nearwork is performed, it typically involves accommodation, convergence and downward gaze. A number of previous studies have examined the effects of accommodation and convergence on changes in the optics and biometrics of the eye in primary gaze. However, little is known about the influence of accommodation on the eye in downward gaze. This thesis is primarily concerned with investigating the changes in the eye during near work in downward gaze under natural viewing conditions. To measure wavefront aberrations in downward gaze under natural viewing conditions, we modified a commercial Shack-Hartmann wavefront sensor by adding a relay lens system to allow on-axis ocular aberration measurements in primary gaze and downward gaze, with binocular fixation. Measurements with the modified wavefront sensor in primary and downward gaze were validated against a conventional aberrometer using both a model eye and in 9 human subjects. We then conducted an experiment to investigate changes in ocular aberrations associated with accommodation in downward gaze over 10 mins in groups of both myopes (n = 14) and emmetropes (n =12) using the modified Shack-Hartmann wavefront sensor. During the distance accommodation task, small but significant changes in refractive power (myopic shift) and higher order aberrations were observed in downward gaze compared to primary gaze. Accommodation caused greater changes in higher order aberrations (in particular coma and spherical aberration) in downward gaze than primary gaze, and there was evidence that the changes in certain aberrations with accommodation over time were different in downward gaze compared to primary gaze. There were no obvious systematic differences in higher order aberrations between refractive error groups during accommodation or downward gaze for fixed pupils. However, myopes exhibited a significantly greater change in higher order aberrations (in particular spherical aberration) than emmetropes for natural pupils after 10 mins of a near task (5 D accommodation) in downward gaze. These findings indicated that ocular aberrations change from primary to downward gaze, particularly with accommodation. To understand the mechanism underlying these changes in greater detail, we then extended this work to examine the characteristics of the corneal optics, internal optics, anterior biometrics and axial length of the eye during a near task, in downward gaze, over 10 mins. Twenty young adult subjects (10 emmetropes and 10 myopes) participated in this study. To measure corneal topography and ocular biometrics in downward gaze, a rotating Scheimpflug camera and an optical biometer were inclined on a custom built, height and tilt adjustable table. We found that both corneal optics and internal optics change with downward gaze, resulting in a myopic shift (~0.10 D) in the spherical power of the eye. The changes in corneal optics appear to be due to eyelid pressure on the anterior surface of the cornea, whereas the changes in the internal optics (an increase in axial length and a decrease in anterior chamber depth) may be associated with movement of the crystalline lens, under the action of gravity, and the influence of altered biomechanical forces from the extraocular muscles on the globe with downward gaze. Changes in axial length with accommodation were significantly greater in downward gaze than primary gaze (p < 0.05), indicating an increased effect of the mechanical forces from the ciliary muscle and extraocular muscles. A subsequent study was conducted to investigate the changes in anterior biometrics, axial length and choroidal thickness in nine cardinal gaze directions under the actions of the extraocular muscles. Ocular biometry measurements were obtained from 30 young adults (10 emmetropes, 10 low myopes and 10 moderate myopes) through a rotating prism with 15° deviation, along the foveal axis, using a non-contact optical biometer in each of nine different cardinal directions of gaze, over 5 mins. There was a significant influence of gaze angle and time on axial length (both p < 0.001), with the greatest axial elongation (+18 ± 8 μm) occurring with infero-nasal gaze (p < 0.001) and a slight decrease in axial length in superior gaze (−12 ± 17 μm) compared with primary gaze (p < 0.001). There was a significant correlation between refractive error (spherical equivalent refraction) and the mean change in axial length in the infero-nasal gaze direction (Pearson's R2 = 0.71, p < 0.001). To further investigate the relative effect of gravity and extraocular muscle force on the axial length, we measured axial length in 15° and 25° downward gaze with the biometer inclined on a tilting table that allowed gaze shifts to occur with either full head turn but no eye turn (reflects the effect of gravity), or full eye turn with no head turn (reflects the effect of extraocular muscle forces). We observed a significant axial elongation in 15° and 25° downward gaze in the full eye turn condition. However, axial length did not change significantly in downward gaze over 5 mins (p > 0.05) in the full head turn condition. The elongation of the axial length in downward gaze appears to be due to the influence of the extraocular muscles, since the effect was not present when head turn was used instead of eye turn. The findings of these experiments collectively show the dynamic characteristics of the optics and biometrics of the eye in downward gaze during a near task, over time. These were small but significant differences between myopic and emmetropic eyes in both the optical and biomechanical changes associated with shifts of gaze direction. These differences between myopes and emmetropes could arise as a consequence of excessive eye growth associated with myopia. However the potentially additive effects of repeated or long lasting near work activities employing infero-nasal gaze could also act to promote elongation of the eye due to optical and/or biomechanical stimuli.

Robust vision-based underwater homing using self-similar landmarks

Next-generation autonomous underwater vehicles (AUVs) will be required to robustly identify underwater targets for tasks such as inspection, localization, and docking. Given their often unstructured operating environments, vision offers enormous potential in underwater navigation over more traditional methods; however, reliable target segmentation often plagues these systems. This paper addresses robust vision-based target recognition by presenting a novel scale and rotationally invariant target design and recognition routine based on self-similar landmarks that enables robust target pose estimation with respect to a single camera. These algorithms are applied to an AUV with controllers developed for vision-based docking with the target. Experimental results show that the system performs exceptionally on limited processing power and demonstrates how the combined vision and controller system enables robust target identification and docking in a variety of operating conditions.

Visualising experimental flow fields through a stormwater gross pollutant trap

Abstract An experimental dataset representing a typical flow field in a stormwater gross pollutant trap (GPT) was visualised. A technique was developed to apply the image-based flow visualisation (IBFV) algorithm to the raw dataset. Particle image velocimetry (PIV) software was previously used to capture the flow field data by tracking neutrally buoyant particles with a high speed camera. The dataset consisted of scattered 2D point velocity vectors and the IBFV visualisation facilitates flow feature characterisation within the GPT. The flow features played a pivotal role in understanding stormwater pollutant capture and retention behaviour within the GPT. It was found that the IBFV animations revealed otherwise unnoticed flow features and experimental artefacts. For example, a circular tracer marker in the IBFV program visually highlighted streamlines to investigate the possible flow paths of pollutants entering the GPT. The investigated flow paths were compared with the behaviour of pollutants monitored during experiments.

Automated aerial inspection guidance with improved turn planning

Aerial inspection of pipelines, powerlines, and other large linear infrastructure networks has emerged in a number of civilian remote sensing applications. Challenges relate to automating inspection flight for under-actuated aircraft with LiDAR/camera sensor constraints whilst subjected to wind disturbances. This paper presents new improved turn planning strategies with guidance suitable for automation of linear infrastructure inspection able to reduce inspection flight distance by including wind information. Simulation and experimental flight tests confirmed the flight distance saving, and the proposed guidance strategies exhibited good tracking performance in a range of wind conditions.

Timing and distribution of protein ingestion during prolonged recovery from resistance exercise alters myofibrillar protein synthesis

Quantity and timing of protein ingestion are major factors regulating myofibrillar protein synthesis (MPS). However, the effect of specific ingestion patterns on MPS throughout a 12 h period is unknown. We determined how different distributions of protein feeding during 12 h recovery after resistance exercise affects anabolic responses in skeletal muscle. Twenty-four healthy trained males were assigned to three groups (n = 8/group) and undertook a bout of resistance exercise followed by ingestion of 80 g of whey protein throughout 12 h recovery in one of the following protocols: 8 × 10 g every 1.5 h (PULSE); 4 × 20 g every 3 h (intermediate: INT); or 2 × 40 g every 6 h (BOLUS). Muscle biopsies were obtained at rest and after 1, 4, 6, 7 and 12 h post exercise. Resting and post-exercise MPS (l-[ring-(13)C6] phenylalanine), and muscle mRNA abundance and cell signalling were assessed. All ingestion protocols increased MPS above rest throughout 1-12 h recovery (88-148%, P < 0.02), but INT elicited greater MPS than PULSE and BOLUS (31-48%, P < 0.02). In general signalling showed a BOLUS>INT>PULSE hierarchy in magnitude of phosphorylation. MuRF-1 and SLC38A2 mRNA were differentially expressed with BOLUS. In conclusion, 20 g of whey protein consumed every 3 h was superior to either PULSE or BOLUS feeding patterns for stimulating MPS throughout the day. This study provides novel information on the effect of modulating the distribution of protein intake on anabolic responses in skeletal muscle and has the potential to maximize outcomes of resistance training for attaining peak muscle mass.

Low muscle glycogen concentration does not suppress the anabolic response to resistance exercise

We determined the effect of muscle glycogen concentration and postexercise nutrition on anabolic signaling and rates of myofibrillar protein synthesis after resistance exercise (REX). Sixteen young, healthy men matched for age, body mass, peak oxygen uptake (VO2peak) and strength (one repetition maximum; 1RM) were randomly assigned to either a nutrient or placebo group. After 48 h diet and exercise control, subjects undertook a glycogen-depletion protocol consisting of one-leg cycling to fatigue (LOW), whereas the other leg rested (NORM). The next morning following an overnight fast, a primed, constant infusion of L-[ring-13C6] phenylalanine was commenced and subjects completed 8 sets of 5 unilateral leg press repetitions at 80% 1RM. Immediately after REX and 2 h later, subjects consumed a 500 ml bolus of a protein/CHO (20 g whey + 40 g maltodextrin) or placebo beverage. Muscle biopsies from the vastus lateralis of both legs were taken at rest and 1 and 4 h after REX. Muscle glycogen concentration was higher in the NORM than LOW at all time points in both nutrient and placebo groups (P < 0.05). Postexercise Akt-p70S6K-rpS6 phosphorylation increased in both groups with no differences between legs (P < 0.05). mTORSer2448 phosphorylation in placebo increased 1 h after exercise in NORM (P < 0.05), whereas mTOR increased ?4-fold in LOW (P < 0.01) and ?11 fold in NORM with nutrient (P < 0.01; different between legs P < 0.05). Post-exercise rates of MPS were not different between NORM and LOW in nutrient (0.070 ± 0.022 vs. 0.068 ± 0.018 %/h) or placebo (0.045 ± 0.021 vs. 0.049 ± 0.017 %/h). We conclude that commencing high-intensity REX with low muscle glycogen availability does not compromise the anabolic signal and subsequent rates of MPS, at least during the early (4 h) postexercise recovery period.