Towards automated and in-situ, near-real time 3-D reconstruction of coral reef environments

Coral reefs are biologically complex ecosystems that support a wide variety of marine organisms. These are fragile communities under enormous threat from natural and human-based influences. Properly assessing and measuring the growth and health of reefs is essential to understanding impacts of ocean acidification, coastal urbanisation and global warming. In this paper, we present an innovative 3-D reconstruction technique based on visual imagery as a non-intrusive, repeatable, in situ method for estimating physical parameters, such as surface area and volume for efficient assessment of long-term variability. The reconstruction algorithms are presented, and benchmarked using an existing data set. We validate the technique underwater, utilising a commercial-off-the-shelf camera and a piece of staghorn coral, Acropora cervicornis. The resulting reconstruction is compared with a laser scan of the coral piece for assessment and validation. The comparison shows that 77% of the pixels in the reconstruction are within 0.3 mm of the ground truth laser scan. Reconstruction results from an unknown video camera are also presented as a segue to future applications of this research.

Fourier active appearance models

Gaining invariance to camera and illumination variations has been a well investigated topic in Active Appearance Model (AAM) fitting literature. The major problem lies in the inability of the appearance parameters of the AAM to generalize to unseen conditions. An attractive approach for gaining invariance is to fit an AAM to a multiple filter response (e.g. Gabor) representation of the input image. Naively applying this concept with a traditional AAM is computationally prohibitive, especially as the number of filter responses increase. In this paper, we present a computationally efficient AAM fitting algorithm based on the Lucas-Kanade (LK) algorithm posed in the Fourier domain that affords invariance to both expression and illumination. We refer to this as a Fourier AAM (FAAM), and show that this method gives substantial improvement in person specific AAM fitting performance over traditional AAM fitting methods.

Optimal control of distributed generators and capacitors by hybrid DPSO

In this paper, a comprehensive planning methodology is proposed that can minimize the line loss, maximize the reliability and improve the voltage profile in a distribution network. The injected active and reactive power of Distributed Generators (DG) and the installed capacitor sizes at different buses and for different load levels are optimally controlled. The tap setting of HV/MV transformer along with the line and transformer upgrading is also included in the objective function. A hybrid optimization method, called Hybrid Discrete Particle Swarm Optimization (HDPSO), is introduced to solve this nonlinear and discrete optimization problem. The proposed HDPSO approach is a developed version of DPSO in which the diversity of the optimizing variables is increased using the genetic algorithm operators to avoid trapping in local minima. The objective function is composed of the investment cost of DGs, capacitors, distribution lines and HV/MV transformer, the line loss, and the reliability. All of these elements are converted into genuine dollars. Given this, a single-objective optimization method is sufficient. The bus voltage and the line current as constraints are satisfied during the optimization procedure. The IEEE 18-bus test system is modified and employed to evaluate the proposed algorithm. The results illustrate the unavoidable need for optimal control on the DG active and reactive power and capacitors in distribution networks.

Radiometric temperature analysis of the Hayabusa spacecraft re-entry

Hayabusa, an unmanned Japanese spacecraft, was launched to study and collect samples from the surface of the asteroid 25143 Itokawa. In June 2010, the Hayabusa spacecraft completed it’s seven year voyage. The spacecraft and the sample return capsule (SRC) re-entered the Earth’s atmosphere over the central Australian desert at speeds on the order of 12 km/s. This provided a rare opportunity to experimentally investigate the radiative heat transfer from the shock-compressed gases in front of the sample return capsule at true-flight conditions. This paper reports on the results of observations from a tracking camera situated on the ground about 100 km from where the capsule experienced peak heating during re-entry.

Wide baseline correspondence extraction beyond local features

Robust, affine covariant, feature extractors provide a means to extract correspondences between images captured by widely separated cameras. Advances in wide baseline correspondence extraction require looking beyond the robust feature extraction and matching approach. This study examines new techniques of extracting correspondences that take advantage of information contained in affine feature matches. Methods of improving the accuracy of a set of putative matches, eliminating incorrect matches and extracting large numbers of additional correspondences are explored. It is assumed that knowledge of the camera geometry is not available and not immediately recoverable. The new techniques are evaluated by means of an epipolar geometry estimation task. It is shown that these methods enable the computation of camera geometry in many cases where existing feature extractors cannot produce sufficient numbers of accurate correspondences.

Robotics, Vision and Control : Fundamental Algorithms in MATLAB

The practice of robotics and computer vision each involve the application of computational algorithms to data. The research community has developed a very large body of algorithms but for a newcomer to the field this can be quite daunting. For more than 10 years the author has maintained two open-source MATLAB® Toolboxes, one for robotics and one for vision. They provide implementations of many important algorithms and allow users to work with real problems, not just trivial examples. This new book makes the fundamental algorithms of robotics, vision and control accessible to all. It weaves together theory, algorithms and examples in a narrative that covers robotics and computer vision separately and together. Using the latest versions of the Toolboxes the author shows how complex problems can be decomposed and solved using just a few simple lines of code. The topics covered are guided by real problems observed by the author over many years as a practitioner of both robotics and computer vision. It is written in a light but informative style, it is easy to read and absorb, and includes over 1000 MATLAB® and Simulink® examples and figures. The book is a real walk through the fundamentals of mobile robots, navigation, localization, arm-robot kinematics, dynamics and joint level control, then camera models, image processing, feature extraction and multi-view geometry, and finally bringing it all together with an extensive discussion of visual servo systems.

Asphalt concrete surfaces macrotexture determination from still images

Road surface macrotexture is identified as one of the factors contributing to the surface's skid resistance. Existing methods of quantifying the surface macrotexture, such as the sand patch test and the laser profilometer test, are either expensive or intrusive, requiring traffic control. High-resolution cameras have made it possible to acquire good quality images from roads for the automated analysis of texture depth. In this paper, a granulometric method based on image processing is proposed to estimate road surface texture coarseness distribution from their edge profiles. More than 1300 images were acquired from two different sites, extending to a total of 2.96 km. The images were acquired using camera orientations of 60 and 90 degrees. The road surface is modeled as a texture of particles, and the size distribution of these particles is obtained from chord lengths across edge boundaries. The mean size from each distribution is compared with the sensor measured texture depth obtained using a laser profilometer. By tuning the edge detector parameters, a coefficient of determination of up to R2 = 0.94 between the proposed method and the laser profilometer method was obtained. The high correlation is also confirmed by robust calibration parameters that enable the method to be used for unseen data after the method has been calibrated over road surface data with similar surface characteristics and under similar imaging conditions.

Particulate texture image analysis with applications

Texture analysis and textural cues have been applied for image classification, segmentation and pattern recognition. Dominant texture descriptors include directionality, coarseness, line-likeness etc. In this dissertation a class of textures known as particulate textures are defined, which are predominantly coarse or blob-like. The set of features that characterise particulate textures are different from those that characterise classical textures. These features are micro-texture, macro-texture, size, shape and compaction. Classical texture analysis techniques do not adequately capture particulate texture features. This gap is identified and new methods for analysing particulate textures are proposed. The levels of complexity in particulate textures are also presented ranging from the simplest images where blob-like particles are easily isolated from their back- ground to the more complex images where the particles and the background are not easily separable or the particles are occluded. Simple particulate images can be analysed for particle shapes and sizes. Complex particulate texture images, on the other hand, often permit only the estimation of particle dimensions. Real life applications of particulate textures are reviewed, including applications to sedimentology, granulometry and road surface texture analysis. A new framework for computation of particulate shape is proposed. A granulometric approach for particle size estimation based on edge detection is developed which can be adapted to the gray level of the images by varying its parameters. This study binds visual texture analysis and road surface macrotexture in a theoretical framework, thus making it possible to apply monocular imaging techniques to road surface texture analysis. Results from the application of the developed algorithm to road surface macro-texture, are compared with results based on Fourier spectra, the auto- correlation function and wavelet decomposition, indicating the superior performance of the proposed technique. The influence of image acquisition conditions such as illumination and camera angle on the results was systematically analysed. Experimental data was collected from over 5km of road in Brisbane and the estimated coarseness along the road was compared with laser profilometer measurements. Coefficient of determination R2 exceeding 0.9 was obtained when correlating the proposed imaging technique with the state of the art Sensor Measured Texture Depth (SMTD) obtained using laser profilometers.

Scene invariant crowd counting

This paper describes a scene invariant crowd counting algorithm that uses local features to monitor crowd size. Unlike previous algorithms that require each camera to be trained separately, the proposed method uses camera calibration to scale between viewpoints, allowing a system to be trained and tested on different scenes. A pre-trained system could therefore be used as a turn-key solution for crowd counting across a wide range of environments. The use of local features allows the proposed algorithm to calculate local occupancy statistics, and Gaussian process regression is used to scale to conditions which are unseen in the training data, also providing confidence intervals for the crowd size estimate. A new crowd counting database is introduced to the computer vision community to enable a wider evaluation over multiple scenes, and the proposed algorithm is tested on seven datasets to demonstrate scene invariance and high accuracy. To the authors' knowledge this is the first system of its kind due to its ability to scale between different scenes and viewpoints.

The effect of ocular surface conditions on blink rate and completeness

A healthy human would be expected to show periodic blinks, making a brief closure of the eyelids. Most blinks are spontaneous, occurring regularly with no external stimulus. However a reflex blink can occur in response to external stimuli such as a bright light, a sudden loud noise, or an object approaching toward the eyes. A voluntary or forced blink is another type of blink in which the person deliberately closes the eyes and the lower eyelid raises to meet the upper eyelid. A complete blink, in which the upper eyelid touches the lower eyelid, contributes to the health of ocular surface by providing a fresh layer of tears as well as maintaining optical integrity by providing a smooth tear film over the cornea. The rate of blinking and its completeness vary depending on the task undertaken during blink assessment, the direction of gaze, the emotional state of the subjects and the method under which the blink was measured. It is also well known that wearing contact lenses (both rigid and soft lenses) can induce significant changes in blink rate and completeness. It is been established that efficient blinking plays an important role in ocular surface health during contact lens wear and for improving contact lens performance and comfort. Inefficient blinking during contact lens wear may be related to a low blink rate or incomplete blinking and can often be a reason for dry eye symptoms or ocular surface staining. It has previously been shown that upward gaze can affect blink rate, causing it to become faster. In the first experiment, it was decided to expand on previous studies in this area by examining the effect of various gaze directions (i.e. upward gaze, primary gaze, downward gaze and lateral gaze) as well as head angle (recumbent position) on normal subjects’ blink rate and completeness through the use of filming with a high-speed camera. The results of this experiment showed that as the open palpebral aperture (and exposed ocular surface area) increased from downward gaze to upward gaze, the number of blinks significantly increased (p<0.04). Also, the size of closed palpebral aperture significantly increased from downward gaze to upward gaze (p<0.005). A weak positive correlation (R² = 0.18) between the blink rate and ocular surface area was found in this study. Also, it was found that the subjects showed 81% complete blinks, 19% incomplete blinks and 2% of twitch blinks in primary gaze, consistent with previous studies. The difference in the percentage of incomplete blinks between upward gaze and downward gaze was significant (p<0.004), showing more incomplete blinks in upward gaze. The findings of this experiment suggest that while blink rate becomes slower in downward gaze, the completeness of blinking is typically better, thereby potentially reducing the risk of tear instability. On the other hand, in upward gaze while the completeness of blinking becomes worse, this is potentially offset by increased blink frequency. In addition, blink rate and completeness were not affected by lateral gaze or head angle, possibly because these conditions have similar size of the open palpebral aperture compared with primary gaze. In the second experiment, an investigation into the changes in blink rate and completeness was carried out in primary gaze and downward gaze with soft and rigid contact lenses in unadapted wearers. Not surprisingly, rigid lens wear caused a significant increase in the blink rate in both primary (p<0.001) and downward gaze (p<0.02). After fitting rigid contact lenses, the closed palpebral aperture (blink completeness) did not show any changes but the open palpebral aperture showed a significant narrowing (p<0.04). This might occur from the subjects’ attempt to avoid interaction between the upper eyelid and the edge of the lens to minimize discomfort. After applying topical anaesthetic eye drops in the eye fitted with rigid lenses, the increased blink rate dropped to values similar to that before lens insertion and the open palpebral aperture returned to baseline values, suggesting that corneal and/or lid margin sensitivity was mediating the increased blink rate and narrowed palpebral aperture. We also investigated the changes in the blink rate and completeness with soft contact lenses including a soft sphere, double slab-off toric design and periballast toric design. Soft contact lenses did not cause any significant changes in the blink rate, closed palpebral aperture, open palpebral aperture and the percentage of incomplete blinks in either primary gaze or downward gaze. After applying anaesthetic eye drops, the blink rate reduced in both primary gaze and downward gaze, however this difference was not statistically significant. The size of the closed palpebral aperture and open palpebral aperture did not show any significant changes after applying anaesthetic eye drops. However it should be noted that the effects of rigid and soft contact lenses that we observed in these studies were only the immediate reaction to contact lenses and in the longer term, it is likely that these responses will vary as the eye adapts to the presence of the lenses.

Fixing the city one photo at a time : mobile logging of maintenance requests

We have designed a mobile application that takes advantage of the built-in features of smart phones such as camera and GPS that allow users to take geo-tagged photos while on the move. Urban residents can take pictures of broken street furniture and public property requiring repair, attach a brief description, and submit the information as a maintenance request to the local government organisation of their city. This paper discusses the design approach that led to the application, highlights a built-in mechanism to elicit user feedback, and evaluates the progress to date with user feedback and log statistics. It concludes with an outlook highlighting user requested features and our own design aspirations for moving from a reporting tool to a civic engagement tool.

Visual voice activity detection using frontal versus profile views

Visual activity detection of lip movements can be used to overcome the poor performance of voice activity detection based solely in the audio domain, particularly in noisy acoustic conditions. However, most of the research conducted in visual voice activity detection (VVAD) has neglected addressing variabilities in the visual domain such as viewpoint variation. In this paper we investigate the effectiveness of the visual information from the speaker’s frontal and profile views (i.e left and right side views) for the task of VVAD. As far as we are aware, our work constitutes the first real attempt to study this problem. We describe our visual front end approach and the Gaussian mixture model (GMM) based VVAD framework, and report the experimental results using the freely available CUAVE database. The experimental results show that VVAD is indeed possible from profile views and we give a quantitative comparison of VVAD based on frontal and profile views The results presented are useful in the development of multi-modal Human Machine Interaction (HMI) using a single camera, where the speaker’s face may not always be frontal.

Thinking feeling

In “Thinking Feeling” a camera zooms in and around an animated constellation of words. There are ten words, each repeated one hundred times. The individual words independently pulse and orbit an invisible nucleus. The slow movements of the words and camera are reinforced by an airy, synthesised soundtrack. Over time, various phrasal combinations form and dissolve on screen. A bit like forcing oneself to sleep, “Thinking Feeling” picks at that fine line between controlling and letting go of thoughts. It creates small mantric loops that slip in and out of focus, playing with the liminal zones between the conscious and unconscious, between language and sensation, between gripping and releasing, and between calm and irritation.

Error analysis and attitude observability of a monocular GPS/visual odometry integrated navigation filter

In this paper, we present a method for the recovery of position and absolute attitude (including pitch, roll and yaw) using a novel fusion of monocular Visual Odometry and GPS measurements in a similar manner to a classic loosely-coupled GPS/INS error state navigation filter. The proposed filter does not require additional restrictions or assumptions such as platform-specific dynamics, map-matching, feature-tracking, visual loop-closing, gravity vector or additional sensors such as an IMU or magnetic compass. An observability analysis of the proposed filter is performed, showing that the scale factor, position and attitude errors are fully observable under acceleration that is non-parallel to velocity vector in the navigation frame. The observability properties of the proposed filter are demonstrated using numerical simulations. We conclude the article with an implementation of the proposed filter using real flight data collected from a Cessna 172 equipped with a downwards-looking camera and GPS, showing the feasibility of the algorithm in real-world conditions.

Aerial object following using visual fuzzy servoing

This article presents a visual servoing system to follow a 3D moving object by a Micro Unmanned Aerial Vehicle (MUAV). The presented control strategy is based only on the visual information given by an adaptive tracking method based on the colour information. A visual fuzzy system has been developed for servoing the camera situated on a rotary wing MAUV, that also considers its own dynamics. This system is focused on continuously following of an aerial moving target object, maintaining it with a fixed safe distance and centred on the image plane. The algorithm is validated on real flights on outdoors scenarios, showing the robustness of the proposed systems against winds perturbations, illumination and weather changes among others. The obtained results indicate that the proposed algorithms is suitable for complex controls task, such object following and pursuit, flying in formation, as well as their use for indoor navigation