Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Towards vision-based deep reinforcement learning for robotic motion control

This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images.

Stationary processes for object detection and non-rigid structure-from-motion

Stationary processes are random variables whose value is a signal and whose distribution is invariant to translation in the domain of the signal. They are intimately connected to convolution, and therefore to the Fourier transform, since the covariance matrix of a stationary process is a Toeplitz matrix, and Toeplitz matrices are the expression of convolution as a linear operator. This thesis utilises this connection in the study of i) efficient training algorithms for object detection and ii) trajectory-based non-rigid structure-from-motion.

Kinematic analysis and design of articulated manipulators with joint motion constraints

For an articulated manipulator with joint rotation constraints, we show that the maximum workspace is not necessarily obtained for equal link lengths but is also determined by the range and mean positions of the joint motions. We present expressions for sectional area, workspace volume, overlap volume and work area in terms of link ratios, mean positions and ranges of joint motion. We present a numerical procedure to obtain the maximum rectangular area that can be embedded in the workspace of an articulated manipulator with joint motion constraints. We demonstrate the use of analytical expressions and the numerical plots in the kinematic design of an articulated manipulator with joint rotation constraints.

Factors influencing stick-slip motion: effect of hardness, crystal structure and surface texture

In the present investigation, efforts were made to study the different frictional responses of materials with varying crystal structure and hardness during sliding against a relatively harder material of different surface textures and roughness. In the experiments, pins were made of pure metals and alloys with significantly different hardness values. Pure metals were selected based on different class of crystal structures, such as face centered cubic (FCC), body centered cubic (BCC), body centered tetragonal (BCT) and hexagonal close packed (HCP) structures. The surface textures with varying roughness were generated on the counterpart plate which was made of H-11 die steel. The experiments were conducted under dry and lubricated conditions using an inclined pin-on-plate sliding tester for various normal loads at ambient environment. In the experiments, it was found that the coefficient of friction is controlled by the surface texture of the harder mating surfaces. Further, two kinds of frictional response, namely steady-state and stick-slip, were observed during sliding. More specifically, stead-state frictional response was observed for the FCC metals, alloys and materials with higher hardness. Stick-slip frictional response was observed for the metals which have limited number of slip systems such as BCT and HCP. In addition, the stick-slip frictional response was dependent on the normal load, lubrication, hardness and surface texture of the counterpart material. However, for a given kind of surface texture, the roughness of the surface affects neither the average coefficient of friction nor the amplitude of stick-slip oscillation significantly.

Divide and Conquer: Efficient Large-Scale Structure from Motion Using Graph Partitioning

Despite significant advances in recent years, structure-from-motion (SfM) pipelines suffer from two important drawbacks. Apart from requiring significant computational power to solve the large-scale computations involved, such pipelines sometimes fail to correctly reconstruct when the accumulated error in incremental reconstruction is large or when the number of 3D to 2D correspondences are insufficient. In this paper we present a novel approach to mitigate the above-mentioned drawbacks. Using an image match graph based on matching features we partition the image data set into smaller sets or components which are reconstructed independently. Following such reconstructions we utilise the available epipolar relationships that connect images across components to correctly align the individual reconstructions in a global frame of reference. This results in both a significant speed up of at least one order of magnitude and also mitigates the problems of reconstruction failures with a marginal loss in accuracy. The effectiveness of our approach is demonstrated on some large-scale real world data sets.

The motion and the core structure of a geostrophic vortex - one-time scale inner solution and the motion of the vortex

By means of the matched asymptotic expansion method with one-time scale analysis we have shown that the inviscid geostrophic vortex solution represents our leading solution away from the vortex. Near the vortex there is a viscous core structure, with the length scale O(a). In the core the viscous stresses (or turbulent stresses) are important, the variations of the velocity and the equivalent height are finite and dependent of time. It also has been shown that the leading inner solutions of the core structure are the same for two different time scales of S/(ghoo)1/2 and S/a (ghoo)1/2. Within the accuracy of O(a) the velocity of a geostrophic vortex center is equal to the velocity of the local background flow, where the vortex is located, in the absence of the vortex. Some numerical examples demonstrate the contributions of these results.