Towards vision-based deep reinforcement learning for robotic motion control

This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images.

A Distributed Robotic Vision Service

Robotic vision is limited by line of sight and onboard camera capabilities. Robots can acquire video or images from remote cameras, but processing additional data has a computational burden. This paper applies the Distributed Robotic Vision Service, DRVS, to robot path planning using data outside line-of-sight of the robot. DRVS implements a distributed visual object detection service to distributes the computation to remote camera nodes with processing capabilities. Robots request task-specific object detection from DRVS by specifying a geographic region of interest and object type. The remote camera nodes perform the visual processing and send the high-level object information to the robot. Additionally, DRVS relieves robots of sensor discovery by dynamically distributing object detection requests to remote camera nodes. Tested over two different indoor path planning tasks DRVS showed dramatic reduction in mobile robot compute load and wireless network utilization.

Osteoporosis medication dispensing for older Australian women from 2002 to 2010: Influences of publications, guidelines, marketing activities and policy

Purpose Developments in anti-osteoporosis medications (AOMs) have led to changes in guidelines and policy, which, along with media and marketing strategies, have had an impact upon the prescribing of AOM. The aim was to examine patterns of AOM dispensing in older women (aged 76–81 years at baseline) from 2002 to 2010. Methods Administrative claims data were used to describe AOM dispensing in 4649 participants (born in 1921–1926 and still alive in 2011) in the Australian Longitudinal Study on Women's Health. The patterns were interpreted in the context of changes in guidelines, indications for subsidy, publications (scholarly and general media), and marketing activities. Results Total use of AOM increased from 134 DDD/1000/day in 2002 to 216 DDD/1000/day in 2007 but then decreased to 184 DDD/1000/day in 2010. Alendronate was the most commonly dispensed AOM but decreased from 2007, while use of risedronate (2002 onward), strontium ranelate (2007 onward) and zoledronic acid (2008 onward) increased. Etidronate and hormone replacement therapy (HRT) prescriptions gradually decreased over time. The decline in alendronate dispensing coincided with increases of other bisphosphonates and publicity about potential adverse effects of bisphosphonates, despite relaxing indications for bone density testing and subsidy for AOM. Conclusions Overall dispense of AOM from 2002 reached a peak in 2007 and thereafter declined despite increases in therapeutic options and improved subsidised access. The recent decline in overall AOM dispensing seems to be explained largely by negative publicity rather than specific changes in guidelines and policy.

Autonomous greenhouse gas sampling using multiple robotic boats

Accurately quantifying total greenhouse gas emissions (e.g. methane) from natural systems such as lakes, reservoirs and wetlands requires the spatial-temporal measurement of both diffusive and ebullitive (bubbling) emissions. Traditional, manual, measurement techniques provide only limited localised assessment of methane flux, often introducing significant errors when extrapolated to the whole-of-system. In this paper, we directly address these current sampling limitations and present a novel multiple robotic boat system configured to measure the spatiotemporal release of methane to atmosphere across inland waterways. The system, consisting of multiple networked Autonomous Surface Vehicles (ASVs) and capable of persistent operation, enables scientists to remotely evaluate the performance of sampling and modelling algorithms for real-world process quantification over extended periods of time. This paper provides an overview of the multi-robot sampling system including the vehicle and gas sampling unit design. Experimental results are shown demonstrating the system’s ability to autonomously navigate and implement an exploratory sampling algorithm to measure methane emissions on two inland reservoirs.

A Low Cost Controller Board for Teaching Robotics

This paper presents the Smarty Board; a new micro-controller board designed specifically for the robotics teaching needs of Australian schools. The primary motivation for this work was the lack of commercially available and cheap controller boards that would have all their components including interfaces on a single board. Having a single board simplifies the construction of programmable robots that can be used as platforms for teaching and learning robotics. Reducing the cost of the board as much as possible was one of the main design objectives. The target user groups for this device are the secondary and tertiary students, and hobbyists. Previous studies have shown that equipment cost is one of the major obstacles for teaching robotics in Australia. The new controller board was demonstrated at high-school seminars. In these demonstrations the new controller board was used for controlling two robots that we built. These robots are available as kits. Given the strong demand from high-school teachers, new kits will be developed for the next robotic Olympiad to be held in Australia in 2006.

Bearing-only SLAM : a vision-based navigation system for autonomous robots

To navigate successfully in a previously unexplored environment, a mobile robot must be able to estimate the spatial relationships of the objects of interest accurately. A Simultaneous Localization and Mapping (SLAM) sys- tem employs its sensors to build incrementally a map of its surroundings and to localize itself in the map simultaneously. The aim of this research project is to develop a SLAM system suitable for self propelled household lawnmowers. The proposed bearing-only SLAM system requires only an omnidirec- tional camera and some inexpensive landmarks. The main advantage of an omnidirectional camera is the panoramic view of all the landmarks in the scene. Placing landmarks in a lawn field to define the working domain is much easier and more flexible than installing the perimeter wire required by existing autonomous lawnmowers. The common approach of existing bearing-only SLAM methods relies on a motion model for predicting the robot’s pose and a sensor model for updating the pose. In the motion model, the error on the estimates of object positions is cumulated due mainly to the wheel slippage. Quantifying accu- rately the uncertainty of object positions is a fundamental requirement. In bearing-only SLAM, the Probability Density Function (PDF) of landmark position should be uniform along the observed bearing. Existing methods that approximate the PDF with a Gaussian estimation do not satisfy this uniformity requirement. This thesis introduces both geometric and proba- bilistic methods to address the above problems. The main novel contribu- tions of this thesis are: 1. A bearing-only SLAM method not requiring odometry. The proposed method relies solely on the sensor model (landmark bearings only) without relying on the motion model (odometry). The uncertainty of the estimated landmark positions depends on the vision error only, instead of the combination of both odometry and vision errors. 2. The transformation of the spatial uncertainty of objects. This thesis introduces a novel method for translating the spatial un- certainty of objects estimated from a moving frame attached to the robot into the global frame attached to the static landmarks in the environment. 3. The characterization of an improved PDF for representing landmark position in bearing-only SLAM. The proposed PDF is expressed in polar coordinates, and the marginal probability on range is constrained to be uniform. Compared to the PDF estimated from a mixture of Gaussians, the PDF developed here has far fewer parameters and can be easily adopted in a probabilistic framework, such as a particle filtering system. The main advantages of our proposed bearing-only SLAM system are its lower production cost and flexibility of use. The proposed system can be adopted in other domestic robots as well, such as vacuum cleaners or robotic toys when terrain is essentially 2D.

Automating human thought processes for a UAV forced landing

This paper describes the current status of a program to develop an automated forced landing system for a fixed-wing Unmanned Aerial Vehicle (UAV). This automated system seeks to emulate human pilot thought processes when planning for and conducting an engine-off emergency landing. Firstly, a path planning algorithm that extends Dubins curves to 3D space is presented. This planning element is then combined with a nonlinear guidance and control logic, and simulated test results demonstrate the robustness of this approach to strong winds during a glided descent. The average path deviation errors incurred are comparable to or even better than that of manned, powered aircraft. Secondly, a study into suitable multi-criteria decision making approaches and the problems that confront the decision-maker is presented. From this study, it is believed that decision processes that utilize human expert knowledge and fuzzy logic reasoning are most suited to the problem at hand, and further investigations will be conducted to identify the particular technique/s to be implemented in simulations and field tests. The automated UAV forced landing approach presented in this paper is promising, and will allow the progression of this technology from the development and simulation stages through to a prototype system

Microassembly fabrication of tissue engineering scaffolds with customized design

Browse > Journals> Automation Science and Enginee ...> Volume: 5 Issue: 3 Microassembly Fabrication of Tissue Engineering Scaffolds With Customized Design 4468741 abstract Han Zhang; Burdet, E.; Poo, A.N.; Hutmacher, D.W.; GE Global Res. Center Ltd., Shanghai This paper appears in: Automation Science and Engineering, IEEE Transactions on Issue Date: July 2008 Volume: 5 Issue:3 On page(s): 446 - 456 ISSN: 1545-5955 Digital Object Identifier: 10.1109/TASE.2008.917011 Date of Current Version: 02 July 2008 Sponsored by: IEEE Robotics and Automation Society Abstract This paper presents a novel technique to fabricate scaffold/cell constructs for tissue engineering by robotic assembly of microscopic building blocks (of volume 0.5$,times,$0.5$,times,$0.2 ${hbox{mm}}^{3}$ and 60 $mu {hbox{m}}$ thickness). In this way, it becomes possible to build scaffolds with freedom in the design of architecture, surface morphology, and chemistry. Biocompatible microparts with complex 3-D shapes were first designed and mass produced using MEMS techniques. Semi-automatic assembly was then realized using a robotic workstation with four degrees of freedom integrating a dedicated microgripper and two optical microscopes. Coarse movement of the gripper is determined by pattern matching in the microscopes images, while the operator controls fine positioning and accurate insertion of the microparts. Successful microassembly was demonstrated using SU-8 and acrylic resin microparts. Taking advantage of parts distortion and adhesion forces, which dominate at micro-level, the parts cleave together after assembly. In contrast to many current scaffold fabrication techniques, no heat, pressure, electrical effect, or toxic chemical reaction is involved, a critical condition for creating scaffolds with biological agents.

Image synthesis based on a model of human vision

Modern computer graphics systems are able to construct renderings of such high quality that viewers are deceived into regarding the images as coming from a photographic source. Large amounts of computing resources are expended in this rendering process, using complex mathematical models of lighting and shading. However, psychophysical experiments have revealed that viewers only regard certain informative regions within a presented image. Furthermore, it has been shown that these visually important regions contain low-level visual feature differences that attract the attention of the viewer. This thesis will present a new approach to image synthesis that exploits these experimental findings by modulating the spatial quality of image regions by their visual importance. Efficiency gains are therefore reaped, without sacrificing much of the perceived quality of the image. Two tasks must be undertaken to achieve this goal. Firstly, the design of an appropriate region-based model of visual importance, and secondly, the modification of progressive rendering techniques to effect an importance-based rendering approach. A rule-based fuzzy logic model is presented that computes, using spatial feature differences, the relative visual importance of regions in an image. This model improves upon previous work by incorporating threshold effects induced by global feature difference distributions and by using texture concentration measures. A modified approach to progressive ray-tracing is also presented. This new approach uses the visual importance model to guide the progressive refinement of an image. In addition, this concept of visual importance has been incorporated into supersampling, texture mapping and computer animation techniques. Experimental results are presented, illustrating the efficiency gains reaped from using this method of progressive rendering. This visual importance-based rendering approach is expected to have applications in the entertainment industry, where image fidelity may be sacrificed for efficiency purposes, as long as the overall visual impression of the scene is maintained. Different aspects of the approach should find many other applications in image compression, image retrieval, progressive data transmission and active robotic vision.

Vision-based topological mapping and localisation

Competent navigation in an environment is a major requirement for an autonomous mobile robot to accomplish its mission. Nowadays, many successful systems for navigating a mobile robot use an internal map which represents the environment in a detailed geometric manner. However, building, maintaining and using such environment maps for navigation is difficult because of perceptual aliasing and measurement noise. Moreover, geometric maps require the processing of huge amounts of data which is computationally expensive. This thesis addresses the problem of vision-based topological mapping and localisation for mobile robot navigation. Topological maps are concise and graphical representations of environments that are scalable and amenable to symbolic manipulation. Thus, they are well-suited for basic robot navigation applications, and also provide a representational basis for the procedural and semantic information needed for higher-level robotic tasks. In order to make vision-based topological navigation suitable for inexpensive mobile robots for the mass market we propose to characterise key places of the environment based on their visual appearance through colour histograms. The approach for representing places using visual appearance is based on the fact that colour histograms change slowly as the field of vision sweeps the scene when a robot moves through an environment. Hence, a place represents a region of the environment rather than a single position. We demonstrate in experiments using an indoor data set, that a topological map in which places are characterised using visual appearance augmented with metric clues provides sufficient information to perform continuous metric localisation which is robust to the kidnapped robot problem. Many topological mapping methods build a topological map by clustering visual observations to places. However, due to perceptual aliasing observations from different places may be mapped to the same place representative in the topological map. A main contribution of this thesis is a novel approach for dealing with the perceptual aliasing problem in topological mapping. We propose to incorporate neighbourhood relations for disambiguating places which otherwise are indistinguishable. We present a constraint based stochastic local search method which integrates the approach for place disambiguation in order to induce a topological map. Experiments show that the proposed method is capable of mapping environments with a high degree of perceptual aliasing, and that a small map is found quickly. Moreover, the method of using neighbourhood information for place disambiguation is integrated into a framework for topological off-line simultaneous localisation and mapping which does not require an initial categorisation of visual observations. Experiments on an indoor data set demonstrate the suitability of our method to reliably localise the robot while building a topological map.

Dragline automation: experimental evaluation through productivity trial

Since 1993 we have been working on the automation of dragline excavators, the largest earthmoving machines that exist. Recently we completed a large-scale experimental program where the automation system was used for production purposes over a two week period and moved over 200,000 tonnes of overburden. This is a landmark achievement in the history of automated excavation. In this paper we briefly describe the robotic system and how it works cooperatively with the machine operator. We then describe our methodology for gauging machine performance, analyze results from the production trial and comment on the effectiveness of the system that we have created. © Springer-Verlag Berlin Heidelberg 2006.

Adaptive depth estimation in image based visual servo control of dynamic systems

This paper considers the question of designing a fully image based visual servo control for a dynamic system. The work is motivated by the ongoing development of image based visual servo control of small aerial robotic vehicles. The observed targets considered are coloured blobs on a flat surface to which the normal direction is known. The theoretical framework is directly applicable to the case of markings on a horizontal floor or landing field. The image features used are a first order spherical moment for position and an image flow measurement for velocity. A fully non-linear adaptive control design is provided that ensures global stability of the closed-loop system. © 2005 IEEE.

Autonomous control of underground mining vehicles using reactive navigation

Describes how many of the navigation techniques developed by the robotics research community over the last decade may be applied to a class of underground mining vehicles (LHDs and haul trucks). We review the current state-of-the-art in this area and conclude that there are essentially two basic methods of navigation applicable. We describe an implementation of a reactive navigation system on a 30 tonne LHD which has achieved full-speed operation at a production mine.

Spatial cognition for robots

The paper discusses robot navigation from biological inspiration. The authors sought to build a model of the rodent brain that is suitable for practical robot navigation. The core model, dubbed RatSLAM, has been demonstrated to have exactly the same advantages described earlier: it can build, maintain, and use maps simultaneously over extended periods of time and can construct maps of large and complex areas from very weak geometric information. The work contrasts with other efforts to embody models of rat brains in robots. The article describes the key elements of the known biology of the rat brain in relation to navigation and how the RatSLAM model captures the ideas from biology in a fashion suitable for implementation on a robotic platform. The paper then outline RatSLAM's performance in two difficult robot navigation challenges, demonstrating how a cognitive robotics approach to navigation can produce results that rival other state of the art approaches in robotics.