Towards vision-based deep reinforcement learning for robotic motion control

This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images.

Low-frequency electrical stimulation combined with a cooling vest improves recovery of elite kayakers following a simulated 1000-m race in a hot environment

This study compared the effects of a low-frequency electrical stimulation (LFES; Veinoplus® Sport, Ad Rem Technology, Paris, France), a low-frequency electrical stimulation combined with a cooling vest (LFESCR) and an active recovery combined with a cooling vest (ACTCR) as recovery strategies on performance (racing time and pacing strategies), physiologic and perceptual responses between two sprint kayak simulated races, in a hot environment (∼32 wet-bulb-globe temperature). Eight elite male kayakers performed two successive 1000-m kayak time trials (TT1 and TT2), separated by a short-term recovery period, including a 30-min of the respective recovery intervention protocol, in a randomized crossover design. Racing time, power output, and stroke rate were recorded for each time trial. Blood lactate concentration, pH, core, skin and body temperatures were measured before and after both TT1 and TT2 and at mid- and post-recovery intervention. Perceptual ratings of thermal sensation were also collected. LFESCR was associated with a very likely effect in performance restoration compared with ACTCR (99/0/1%) and LFES conditions (98/0/2%). LFESCR induced a significant decrease in body temperature and thermal sensation at post-recovery intervention, which is not observed in ACTCR condition. In conclusion, the combination of LFES and wearing a cooling vest (LFESCR) improves performance restoration between two 1000-m kayak time trials achieved by elite athletes, in the heat.

Nonparametric traversability estimation in partially occluded and deformable terrain

Terrain traversability estimation is a fundamental requirement to ensure the safety of autonomous planetary rovers and their ability to conduct long-term missions. This paper addresses two fundamental challenges for terrain traversability estimation techniques. First, representations of terrain data, which are typically built by the rover’s onboard exteroceptive sensors, are often incomplete due to occlusions and sensor limitations. Second, during terrain traversal, the rover-terrain interaction can cause terrain deformation, which may significantly alter the difficulty of traversal. We propose a novel approach built on Gaussian process (GP) regression to learn, and consequently to predict, the rover’s attitude and chassis configuration on unstructured terrain using terrain geometry information only. First, given incomplete terrain data, we make an initial prediction under the assumption that the terrain is rigid, using a learnt kernel function. Then, we refine this initial estimate to account for the effects of potential terrain deformation, using a near-to-far learning approach based on multitask GP regression. We present an extensive experimental validation of the proposed approach on terrain that is mostly rocky and whose geometry changes as a result of loads from rover traversals. This demonstrates the ability of the proposed approach to accurately predict the rover’s attitude and configuration in partially occluded and deformable terrain.

Analysing the Impact of Learning Inputs - Application to Terrain Traversability Estimation

Data-driven approaches such as Gaussian Process (GP) regression have been used extensively in recent robotics literature to achieve estimation by learning from experience. To ensure satisfactory performance, in most cases, multiple learning inputs are required. Intuitively, adding new inputs can often contribute to better estimation accuracy, however, it may come at the cost of a new sensor, larger training dataset and/or more complex learning, some- times for limited benefits. Therefore, it is crucial to have a systematic procedure to determine the actual impact each input has on the estimation performance. To address this issue, in this paper we propose to analyse the impact of each input on the estimate using a variance-based sensitivity analysis method. We propose an approach built on Analysis of Variance (ANOVA) decomposition, which can characterise how the prediction changes as one or more of the input changes, and also quantify the prediction uncertainty as attributed from each of the inputs in the framework of dependent inputs. We apply the proposed approach to a terrain-traversability estimation method we proposed in prior work, which is based on multi-task GP regression, and we validate this implementation experimentally using a rover on a Mars-analogue terrain.

Initialisation flaws in the A5-GMR-1 satphone encryption algorithm

A5-GMR-1 is a synchronous stream cipher used to provide confidentiality for communications between satellite phones and satellites. The keystream generator may be considered as a finite state machine, with an internal state of 81 bits. The design is based on four linear feedback shift registers, three of which are irregularly clocked. The keystream generator takes a 64-bit secret key and 19-bit frame number as inputs, and produces an output keystream of length between $2^8$ and $2^{10}$ bits. Analysis of the initialisation process for the keystream generator reveals serious flaws which significantly reduce the number of distinct keystreams that the generator can produce. Multiple (key, frame number) pairs produce the same keystream, and the relationship between the various pairs is easy to determine. Additionally, many of the keystream sequences produced are phase shifted versions of each other, for very small phase shifts. These features increase the effectiveness of generic time-memory tradeoff attacks on the cipher, making such attacks feasible.

A closed-form optimal tuning of mass dampers for one degree-of-freedom systems under rotating unbalance forcing

This paper is focused on the study of a vibrating system forced by a rotating unbalance and coupled to a tuned mass damper (TMD). The analysis of the dynamic response of the entire system is used to define the parameters of such device in order to achieve optimal damping properties. The inertial forcing due to the rotating unbalance depends quadratically on the forcing frequency and it leads to optimal tuning parameters that differ from classical values obtained for pure harmonic forcing. Analytical results demonstrate that frequency and damping ratios, as a function of the mass parameter, should be higher than classical optimal parameters. The analytical study is carried out for the undamped primary system, and numerically investigated for the damped primary system. We show that, for practical applications, proper TMD tuning allows to achieve a reduction in the steady-state response of about 20% with respect to the response achieved with a classically tuned damper. Copyright © 2015 by ASME.

Efficient probabilistic contingency analysis through a stability measure considering wind perturbation

The integration of stochastic wind power has accentuated a challenge for power system stability assessment. Since the power system is a time-variant system under wind generation fluctuations, pure time-domain simulations are difficult to provide real-time stability assessment. As a result, the worst-case scenario is simulated to give a very conservative assessment of system transient stability. In this study, a probabilistic contingency analysis through a stability measure method is proposed to provide a less conservative contingency analysis which covers 5-min wind fluctuations and a successive fault. This probabilistic approach would estimate the transfer limit of a critical line for a given fault with stochastic wind generation and active control devices in a multi-machine system. This approach achieves a lower computation cost and improved accuracy using a new stability measure and polynomial interpolation, and is feasible for online contingency analysis.

R1STM: One-class support tensor machine with randomised kernel

Identifying unusual or anomalous patterns in an underlying dataset is an important but challenging task in many applications. The focus of the unsupervised anomaly detection literature has mostly been on vectorised data. However, many applications are more naturally described using higher-order tensor representations. Approaches that vectorise tensorial data can destroy the structural information encoded in the high-dimensional space, and lead to the problem of the curse of dimensionality. In this paper we present the first unsupervised tensorial anomaly detection method, along with a randomised version of our method. Our anomaly detection method, the One-class Support Tensor Machine (1STM), is a generalisation of conventional one-class Support Vector Machines to higher-order spaces. 1STM preserves the multiway structure of tensor data, while achieving significant improvement in accuracy and efficiency over conventional vectorised methods. We then leverage the theory of nonlinear random projections to propose the Randomised 1STM (R1STM). Our empirical analysis on several real and synthetic datasets shows that our R1STM algorithm delivers comparable or better accuracy to a state-of-the-art deep learning method and traditional kernelised approaches for anomaly detection, while being approximately 100 times faster in training and testing.

Aircraft trajectory clustering techniques using circular statistics

This paper presents a statistical aircraft trajectory clustering approach aimed at discriminating between typical manned and expected unmanned traffic patterns. First, a resampled version of each trajectory is modelled using a mixture of Von Mises distributions (circular statistics). Second, the remodelled trajectories are globally aligned using tools from bioinformatics. Third, the alignment scores are used to cluster the trajectories using an iterative k-medoids approach and an appropriate distance function. The approach is then evaluated using synthetically generated unmanned aircraft flights combined with real air traffic position reports taken over a sector of Northern Queensland, Australia. Results suggest that the technique is useful in distinguishing between expected unmanned and manned aircraft traffic behaviour, as well as identifying some common conventional air traffic patterns.

Cost-efficient virtual machine management in data centers

Virtual Machine (VM) management is an obvious need in today's data centers for various management activities and is accomplished in two phases— finding an optimal VM placement plan and implementing that placement through live VM migrations. These phases result in two research problems— VM placement problem (VMPP) and VM migration scheduling problem (VMMSP). This research proposes and develops several evolutionary algorithms and heuristic algorithms to address the VMPP and VMMSP. Experimental results show the effectiveness and scalability of the proposed algorithms. Finally, a VM management framework has been proposed and developed to automate the VM management activity in cost-efficient way.

Evaluation of the AlgerBrush II rotating burr as a tool for inducing ocular surface failure in the New Zealand White rabbit

The New Zealand White rabbit has been widely used as a model of limbal stem cell deficiency (LSCD). Current techniques for experimental induction of LSCD utilize caustic chemicals, or organic solvents applied in conjunction with a surgical limbectomy. While generally successful in depleting epithelial progenitors, the depth and severity of injury is difficult to control using chemical-based methods. Moreover, the anterior chamber can be easily perforated while surgically excising the corneal limbus. In the interest of creating a safer and more defined LSCD model, we have therefore evaluated a mechanical debridement technique based upon use of the AlgerBrush II rotating burr. An initial comparison of debridement techniques was conducted in situ using 24 eyes in freshly acquired New Zealand White rabbit cadavers. Techniques for comparison (4 eyes each) included: (1) non-wounded control, (2) surgical limbectomy followed by treatment with 100% (v/v) n-heptanol to remove the corneal epithelium (1-2 minutes), (3) treatment of both limbus and cornea with n-heptanol alone, (4) treatment of both limbus and cornea with 20% (v/v) ethanol (2-3 minutes), (5) a 2.5-mm rounded burr applied to both the limbus and cornea, and (6) a 1-mm pointed burr applied to the limbus, followed by the 2.5-mm rounded burr applied to the cornea. All corneas were excised and processed for histology immediately following debridement. A panel of four assessors subsequently scored the degree of epithelial debridement within the cornea and limbus using masked slides. The 2.5-mm burr most consistently removed the corneal and limbal epithelia. Islands of limbal epithelial cells were occasionally retained following surgical limbectomy/heptanol treatment, or use of the 1-mm burr. Limbal epithelial cells were consistently retained following treatment with either ethanol or n-heptanol alone, with ethanol being the least effective treatment overall. The 2.5-mm burr method was subsequently evaluated in the right eye of 3 live rabbits by weekly clinical assessments (photography and slit lamp examination) for up to 5 weeks, followed by histological analyses (hematoxylin & eosin stain, periodic acid-Schiff stain and immunohistochemistry for keratin 3 and 13). All 3 eyes that had been completely debrided using the 2.5-mm burr displayed symptoms of ocular surface failure as defined by retention of a prominent epithelial defect (~40% of corneal surface at 5 weeks), corneal neovascularization (2 to 3 quadrants), reduced corneal transparency and conjunctivalization of the corneal surface (demonstrated by the presence of goblet cells and/or staining for keratin 13). In conclusion, our findings indicate that the AlgerBrush II rotating burr is an effective method for the establishment of ocular surface failure in New Zealand White rabbits. In particular, we recommend use of the 2.5-mm rotating burr for improved efficiency of epithelial debridement and safety compared to surgical limbectomy.

Developing and testing readability measurements for second language learners

This research constructed a readability measurement for French speakers who view English as a second language. It identified the true cognates, which are the similar words from these two languages, as an indicator of the difficulty of an English text for French people. A multilingual lexical resource is used to detect true cognates in text, and Statistical Language Modelling to predict the predict the readability level. The proposed enhanced statistical language model is making a step in the right direction by improving the accuracy of readability predictions for French speakers by up to 10% compared to state of the art approaches. The outcome of this study could accelerate the learning process for French speakers who are studying English. More importantly, this study also benefits the readability estimation research community, presenting an approach and evaluation at sentence level as well as innovating with the use of cognates as a new text feature.

Place categorization and semantic mapping on a mobile robot

In this paper we focus on the challenging problem of place categorization and semantic mapping on a robot with-out environment-specific training. Motivated by their ongoing success in various visual recognition tasks, we build our system upon a state-of-the-art convolutional network. We overcome its closed-set limitations by complementing the network with a series of one-vs-all classifiers that can learn to recognize new semantic classes online. Prior domain knowledge is incorporated by embedding the classification system into a Bayesian filter framework that also ensures temporal coherence. We evaluate the classification accuracy of the system on a robot that maps a variety of places on our campus in real-time. We show how semantic information can boost robotic object detection performance and how the semantic map can be used to modulate the robot’s behaviour during navigation tasks. The system is made available to the community as a ROS module.

Mixed pixel analysis and assessment for flood mapping

The most difficult operation in flood inundation mapping using optical flood images is to map the ‘wet’ areas where trees and houses are partly covered by water. This can be referred to as a typical problem of the presence of mixed pixels in the images. A number of automatic information extracting image classification algorithms have been developed over the years for flood mapping using optical remote sensing images, with most labelling a pixel as a particular class. However, they often fail to generate reliable flood inundation mapping because of the presence of mixed pixels in the images. To solve this problem, spectral unmixing methods have been developed. In this thesis, methods for selecting endmembers and the method to model the primary classes for unmixing, the two most important issues in spectral unmixing, are investigated. We conduct comparative studies of three typical spectral unmixing algorithms, Partial Constrained Linear Spectral unmixing, Multiple Endmember Selection Mixture Analysis and spectral unmixing using the Extended Support Vector Machine method. They are analysed and assessed by error analysis in flood mapping using MODIS, Landsat and World View-2 images. The Conventional Root Mean Square Error Assessment is applied to obtain errors for estimated fractions of each primary class. Moreover, a newly developed Fuzzy Error Matrix is used to obtain a clear picture of error distributions at the pixel level. This thesis shows that the Extended Support Vector Machine method is able to provide a more reliable estimation of fractional abundances and allows the use of a complete set of training samples to model a defined pure class. Furthermore, it can be applied to analysis of both pure and mixed pixels to provide integrated hard-soft classification results. Our research also identifies and explores a serious drawback in relation to endmember selections in current spectral unmixing methods which apply fixed sets of endmember classes or pure classes for mixture analysis of every pixel in an entire image. However, as it is not accurate to assume that every pixel in an image must contain all endmember classes, these methods usually cause an over-estimation of the fractional abundances in a particular pixel. In this thesis, a subset of adaptive endmembers in every pixel is derived using the proposed methods to form an endmember index matrix. The experimental results show that using the pixel-dependent endmembers in unmixing significantly improves performance.