Towards pose-robust face recognition on video

This thesis investigates face recognition in video under the presence of large pose variations. It proposes a solution that performs simultaneous detection of facial landmarks and head poses across large pose variations, employs discriminative modelling of feature distributions of faces with varying poses, and applies fusion of multiple classifiers to pose-mismatch recognition. Experiments on several benchmark datasets have demonstrated that improved performance is achieved using the proposed solution.

An efficient and robust system for multi-person event detection in real world indoor surveillance scenes

Due to the popularity of security cameras in public places, it is of interest to design an intelligent system that can efficiently detect events automatically. This paper proposes a novel algorithm for multi-person event detection. To ensure greater than real-time performance, features are extracted directly from compressed MPEG video. A novel histogram-based feature descriptor that captures the angles between extracted particle trajectories is proposed, which allows us to capture motion patterns of multi-person events in the video. To alleviate the need for fine-grained annotation, we propose the use of Labelled Latent Dirichlet Allocation, a “weakly supervised” method that allows the use of coarse temporal annotations which are much simpler to obtain. This novel system is able to run at approximately ten times real-time, while preserving state-of-theart detection performance for multi-person events on a 100-hour real-world surveillance dataset (TRECVid SED).

Advances in gyro-stabilisation of vessel roll motion

The use of gyro-dynamic forces to counteract the wave-induced roll motion of marine vessels in a seaway was proposed over 100 years ago. These early systems showed a remarkable performance, reporting roll reductions of up to 95% in some sailing conditions. Despite this success, further developments were not pursued since the systems were unable to provide acceptable performance over an extended envelope of sailing and environmental conditions, and the invention of fin roll stabilisers provided a satisfactory alternative. This has been attributed to simplistic controls, heavy drive systems, and large structural mass required to withstand the loads given the low strength materials available at the time. Today, advances in material strength, bearings, motor technology and mechanical design methods, together with powerful signal processing algorithms, has resulted in a revitalized interest in gyro-stabilisers for ships. Advanced control systems have enabled optimisation of restoring torques across a range of wave environments and sailing conditions through adaptive control system design. All of these improvements have resulted in increased spinning speed, lower mass, and dramatically increased stabilising performance. This brief paper provides an overview of recent developments in the design and control of gyro-stabilisers of ship roll motion. In particular, the novel Halcyon Gyro-Stabilisers are introduced, and their performance is illustrated based on a simulation case study for a naval patrol vessel. Given the growing national and global interest in small combatants and patrol vessels, modem gyro-stabilisers may offer a significant technological contribution to the age old problem of comfort and mission operability on small ships, especially at loiter speeds.

An MRF based abnormal event detection approach using motion and appearance features

Abnormal event detection has attracted a lot of attention in the computer vision research community during recent years due to the increased focus on automated surveillance systems to improve security in public places. Due to the scarcity of training data and the definition of an abnormality being dependent on context, abnormal event detection is generally formulated as a data-driven approach where activities are modeled in an unsupervised fashion during the training phase. In this work, we use a Gaussian mixture model (GMM) to cluster the activities during the training phase, and propose a Gaussian mixture model based Markov random field (GMM-MRF) to estimate the likelihood scores of new videos in the testing phase. Further-more, we propose two new features: optical acceleration, and the histogram of optical flow gradients; to detect the presence of any abnormal objects and speed violations in the scene. We show that our proposed method outperforms other state of the art abnormal event detection algorithms on publicly available UCSD dataset.

Text recognition approaches for indoor robotics : a comparison

This paper evaluates the performance of different text recognition techniques for a mobile robot in an indoor (university campus) environment. We compared four different methods: our own approach using existing text detection methods (Minimally Stable Extremal Regions detector and Stroke Width Transform) combined with a convolutional neural network, two modes of the open source program Tesseract, and the experimental mobile app Google Goggles. The results show that a convolutional neural network combined with the Stroke Width Transform gives the best performance in correctly matched text on images with single characters whereas Google Goggles gives the best performance on images with multiple words. The dataset used for this work is released as well.

Machine learning for activity recognition : hip versus wrist data

Problem addressed Wrist-worn accelerometers are associated with greater compliance. However, validated algorithms for predicting activity type from wrist-worn accelerometer data are lacking. This study compared the activity recognition rates of an activity classifier trained on acceleration signal collected on the wrist and hip. Methodology 52 children and adolescents (mean age 13.7 +/- 3.1 year) completed 12 activity trials that were categorized into 7 activity classes: lying down, sitting, standing, walking, running, basketball, and dancing. During each trial, participants wore an ActiGraph GT3X+ tri-axial accelerometer on the right hip and the non-dominant wrist. Features were extracted from 10-s windows and inputted into a regularized logistic regression model using R (Glmnet + L1). Results Classification accuracy for the hip and wrist was 91.0% +/- 3.1% and 88.4% +/- 3.0%, respectively. The hip model exhibited excellent classification accuracy for sitting (91.3%), standing (95.8%), walking (95.8%), and running (96.8%); acceptable classification accuracy for lying down (88.3%) and basketball (81.9%); and modest accuracy for dance (64.1%). The wrist model exhibited excellent classification accuracy for sitting (93.0%), standing (91.7%), and walking (95.8%); acceptable classification accuracy for basketball (86.0%); and modest accuracy for running (78.8%), lying down (74.6%) and dance (69.4%). Potential Impact Both the hip and wrist algorithms achieved acceptable classification accuracy, allowing researchers to use either placement for activity recognition.

Visual predictive control of spiral motion

This paper deals with constrained image-based visual servoing of circular and conical spiral motion about an unknown object approximating a single image point feature. Effective visual control of such trajectories has many applications for small unmanned aerial vehicles, including surveillance and inspection, forced landing (homing), and collision avoidance. A spherical camera model is used to derive a novel visual-predictive controller (VPC) using stability-based design methods for general nonlinear model-predictive control. In particular, a quasi-infinite horizon visual-predictive control scheme is derived. A terminal region, which is used as a constraint in the controller structure, can be used to guide appropriate reference image features for spiral tracking with respect to nominal stability and feasibility. Robustness properties are also discussed with respect to parameter uncertainty and additive noise. A comparison with competing visual-predictive control schemes is made, and some experimental results using a small quad rotor platform are given.

Seeing pedestrians at night : effect of driver age and visual abilities

Purpose To quantify the effects of driver age on night-time pedestrian conspicuity, and to determine whether individual differences in visual performance can predict drivers' ability to recognise pedestrians at night. Methods Participants were 32 visually normal drivers (20 younger: M = 24.4 years ± 6.4 years; 12 older: M = 72.0 years ± 5.0 years). Visual performance was measured in a laboratory-based testing session including visual acuity, contrast sensitivity, motion sensitivity and the useful field of view. Night-time pedestrian recognition distances were recorded while participants drove an instrumented vehicle along a closed road course at night; to increase the workload of drivers, auditory and visual distracter tasks were presented for some of the laps. Pedestrians walked in place, sideways to the oncoming vehicles, and wore either a standard high visibility reflective vest or reflective tape positioned on the movable joints (biological motion). Results Driver age and pedestrian clothing significantly (p < 0.05) affected the distance at which the drivers first responded to the pedestrians. Older drivers recognised pedestrians at approximately half the distance of the younger drivers and pedestrians were recognised more often and at longer distances when they wore a biological motion reflective clothing configuration than when they wore a reflective vest. Motion sensitivity was an independent predictor of pedestrian recognition distance, even when controlling for driver age. Conclusions The night-time pedestrian recognition capacity of older drivers was significantly worse than that of younger drivers. The distance at which drivers first recognised pedestrians at night was best predicted by a test of motion sensitivity.

Adapting “8-bit” motion style to 3D computer animation for Wreck-it Ralph

Tangled (2011) demonstrated that Walt Disney Animation has successfully extended the traditional Disney animation aesthetic to the 3D medium. The very next film produced by the studio however, Wreck-it Ralph (2012), required the animators (trained in the traditional Disney style) to develop a limited style of animation inspired by the 8-bit motion of 1980s video games. This paper examines the 8-bit style motion in Wreck-it Ralph to understand if and how the principles of animation were adapted for the film.

Spatially varying ground motion effects on seismic response of adjacent structures considering soil-structure interaction

Spatial variation of seismic ground motions is caused by incoherence effect, wave passage, and local site conditions. This study focuses on the effects of spatial variation of earthquake ground motion on the responses of adjacent reinforced concrete (RC) frame structures. The adjacent buildings are modeled considering soil-structure interaction (SSI) so that the buildings can be interacted with each other under uniform and non-uniform ground motions. Three different site classes are used to model the soil layers of SSI system. Based on fast Fourier transformation (FFT), spatially correlated non-uniform ground motions are generated compatible with known power spectrum density function (PSDF) at different locations. Numerical analyses are carried out to investigate the displacement responses and the absolute maximum base shear forces of adjacent structures subjected to spatially varying ground motions. The results are presented in terms of related parameters affecting the structural response using three different types of soil site classes. The responses of adjacent structures have changed remarkably due to spatial variation of ground motions. The effect can be significant on rock site rather than clay site.

The pattern of coupling dynamics between postural motion, isotonic hand movements and physiological tremor

This study was designed to examine differences in the coupling dynamics between upper limb motion, physiological tremor and whole body postural sway in young healthy adults. Acceleration of the hand and fingers, forearm EMG activity and postural sway data were recorded. Estimation of the degree of bilateral and limb motion-postural sway coupling was determined by cross correlation, coherence and Cross-ApEn analyses. The results of the analysis revealed that, under postural tremor conditions, there was no significant coupling between limbs, muscles or sway across all metrics of coupling. In contrast, performing a rapid alternating flexion/extension movement about the wrist joint (with one or both limbs) resulted in stronger coupling between limb motion and postural sway. These results support the view that, for physiological tremor responses, the control of postural sway is maintained independent to tremor in the upper limb. However, increasing the level of movement about a distal segment of one arm (or both) leads to increased coupling throughout the body. The basis for this increased coupling would appear to be related to the enhanced neural drive to task-specific muscles within the upper limb.

Steady state visually evoked potential correlates of static and dynamic emotional face processing

While the neural regions associated with facial identity recognition are considered to be well defined, the neural correlates of non-moving and moving images of facial emotion processing are less clear. This study examined the brain electrical activity changes in 26 participants (14 males M = 21.64, SD = 3.99; 12 females M = 24.42, SD = 4.36), during a passive face viewing task, a scrambled face task and separate emotion and gender face discrimination tasks. The steady state visual evoked potential (SSVEP) was recorded from 64-electrode sites. Consistent with previous research, face related activity was evidenced at scalp regions over the parieto-temporal region approximately 170 ms after stimulus presentation. Results also identified different SSVEP spatio-temporal changes associated with the processing of static and dynamic facial emotions with respect to gender, with static stimuli predominately associated with an increase in inhibitory processing within the frontal region. Dynamic facial emotions were associated with changes in SSVEP response within the temporal region, which are proposed to index inhibitory processing. It is suggested that static images represent non-canonical stimuli which are processed via different mechanisms to their more ecologically valid dynamic counterparts.

Towards Vision-Based Pose- and Condition-Invariant Place Recognition along Routes

Vision-based place recognition involves recognising familiar places despite changes in environmental conditions or camera viewpoint (pose). Existing training-free methods exhibit excellent invariance to either of these challenges, but not both simultaneously. In this paper, we present a technique for condition-invariant place recognition across large lateral platform pose variance for vehicles or robots travelling along routes. Our approach combines sideways facing cameras with a new multi-scale image comparison technique that generates synthetic views for input into the condition-invariant Sequence Matching Across Route Traversals (SMART) algorithm. We evaluate the system’s performance on multi-lane roads in two different environments across day-night cycles. In the extreme case of day-night place recognition across the entire width of a four-lane-plus-median-strip highway, we demonstrate performance of up to 44% recall at 100% precision, where current state-of-the-art fails.

Visual place recognition for persistent robot navigation in changing environments

This thesis demonstrates that robots can learn about how the world changes, and can use this information to recognise where they are, even when the appearance of the environment has changed a great deal. The ability to localise in highly dynamic environments using vision only is a key tool for achieving long-term, autonomous navigation in unstructured outdoor environments. The proposed learning algorithms are designed to be unsupervised, and can be generated by the robot online in response to its observations of the world, without requiring information from a human operator or other external source.

Unsupervised online learning of condition-invariant images for place recognition

This paper presents an online, unsupervised training algorithm enabling vision-based place recognition across a wide range of changing environmental conditions such as those caused by weather, seasons, and day-night cycles. The technique applies principal component analysis to distinguish between aspects of a location’s appearance that are condition-dependent and those that are condition-invariant. Removing the dimensions associated with environmental conditions produces condition-invariant images that can be used by appearance-based place recognition methods. This approach has a unique benefit – it requires training images from only one type of environmental condition, unlike existing data-driven methods that require training images with labelled frame correspondences from two or more environmental conditions. The method is applied to two benchmark variable condition datasets. Performance is equivalent or superior to the current state of the art despite the lesser training requirements, and is demonstrated to generalise to previously unseen locations.