Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Knowing

Directed by Alex Proyas, the Knowing is an action-packed science-fiction disaster movie. A well-known Australian director working in Hollywood, Proyas has developed an international reputation for stylised fantasy and science-fiction movies, including the neo-gothic movie The Crow (1994), the complex science-fiction film Dark City (1998), and the adaptation of Isaac Asimov’s sci-fi classic I, Robot (2004) which earned almost US$350 million theatrically worldwide. Knowing was produced for US$50 million and relies heavily upon special effects (including a visually impressive sequence of the world being destroyed) and high-octane action sequences (including a notable plane crash). Knowing’s cast included Australian actors, Rose Byrne and Ben Mendelsohn, and American actor Nicolas Cage. While Knowing received typically poor critical reviews, the movie performed well at the box-office earning over US$183 million worldwide.

Autonomous hot metal carrier

This paper reports work involved with the automation of a Hot Metal Carrier — a 20 tonne forklift-type vehicle used to move molten metal in aluminium smelters. To achieve efficient vehicle operation, issues of autonomous navigation and materials handling must be addressed. We present our complete system and experiments demontrating reliable operation. One of the most significant experiments was five-hours of continuous operation where the vehicle travelled over 8 km and conducted 60 load handling operations. We also describe an experiment where the vehicle and autonomous operation were supervised from the other side of the world via a satellite phone network.

Is the sun too bright in Queensland? An approach to robust outdoor colour beacon detection

Using cameras onboard a robot for detecting a coloured stationary target outdoors is a difficult task. Apart from the complexity of separating the target from the background scenery over different ranges, there are also the inconsistencies with direct and reflected illumination from the sun,clouds, moving and stationary objects. They can vary both the illumination on the target and its colour as perceived by the camera. In this paper, we analyse the effect of environment conditions, range to target, camera settings and image processing on the reported colours of various targets. The analysis indicates the colour space and camera configuration that provide the most consistent colour values over varying environment conditions and ranges. This information is used to develop a detection system that provides range and bearing to detected targets. The system is evaluated over various lighting conditions from bright sunlight, shadows and overcast days and demonstrates robust performance. The accuracy of the system is compared against a laser beacon detector with preliminary results indicating it to be a valuable asset for long-range coloured target detection.

Evaluation of internal navigation sensor suites for underground mining vehicle navigation

This paper describes a series of trials that were done at an underground mine in New South Wales, Australia. Experimental results are presented from the data obtained during the field trials and suitable sensor suites for an autonomous mining vehicle navigation system are evaluated.

Robotics for the mining industry

The mining industry is highly suitable for the application of robotics and automation technology since the work is both arduous and dangerous. However, while the industry makes extensive use of mechanisation it has shown a slow uptake of automation. A major cause of this is the complexity of the task, and the limitations of existing automation technology which is predicated on a structured and time invariant working environment. Here we discuss the topic of mining automation from a robotics and computer vision perspective — as a problem in sensor based robot control, an issue which the robotics community has been studying for nearly two decades. We then describe two of our current mining automation projects to demonstrate what is possible for both open-pit and underground mining operations.

Autonomous helicopter hover using an artificial neural network

Details the developments to date of an unmanned air vehicle (UAV) based on a standard size 60 model helicopter. The design goal is to have the helicopter achieve stable hover with the aid of an INS and stereo vision. The focus of the paper is on the development of an artificial neural network (ANN) that makes use of only the INS data to generate hover commands, which are used to directly manipulate the flight servos. Current results show that networks incorporating some form of recurrency (state history) offer little advantage over those without. At this stage, the ANN has partially maintained periods of hover even with misaligned sensors.

A real-time software architecture for robotics and automation

This paper describes a software architecture for real-world robotic applications. We discuss issues of software reliability, testing and realistic off-line simulation that allows the majority of the automation system to be tested off-line in the laboratory before deployment in the field. A recent project, the automation of a very large mining machine is used to illustrate the discussion.

3D perception for mining robotics

This paper discusses some of the sensing technologies available for guiding robot manipulators for a class of underground mining tasks including drilling jumbos, bolting arms, shotcreters or explosive chargers. Data acquired with such sensors, in the laboratory and underground, is presented.

Dragline modelling and control

Electric walking draglines are physically large and powerful machines used in the mining industry. However with the addition of suitable sensors and a controller a dragline can be considered as a numerically controlled machine or robot which can then perform parts of the operating cycle automatically. This paper presents an analysis of the electromechanical system necessary precursor to automatic control

Dragline swing automation

Draglines are very large machines that are used to remove overburden in open-cut coal mines. This paper outlines the design of a computer control system to implement an automated swing cycle on a production dragline. Subsystems and sensors have been developed to satisfy the constraints imposed by the task, the harsh operating environment and the mine's production requirements.

Vision-based control for mining automation

The mining industry is highly suitable for the application of robotics and automation technology since the work is both arduous and dangerous. Visual servoing is a means of integrating noncontact visual sensing with machine control to augment or replace operator based control. This article describes two of our current mining automation projects in order to demonstrate some, perhaps unusual, applications of visual servoing, and also to illustrate some very real problems with robust computer vision

Modeling and control of a robotic power line inspection vehicle

Power line inspection is a vital function for electricity supply companies but it involves labor-intensive and expensive procedures which are tedious and error-prone for humans to perform. A possible solution is to use an unmanned aerial vehicle (UAV) equipped with video surveillance equipment to perform the inspection. This paper considers how a small, electrically driven rotorcraft conceived for this application could be controlled by visually tracking the overhead supply lines. A dynamic model for a ducted-fan rotorcraft is presented and used to control the action of an Air Vehicle Simulator (AVS), consisting of a cable-array robot. Results show how visual data can be used to determine, and hence regulate in closed loop, the simulated vehicle’s position relative to the overhead lines.

Rubbing shoulders with mobile service robots

This paper is not about the details of yet another robot control system, but rather the issues surrounding realworld robotic implementation. It is a fact that in order to realise a future where robots co-exist with people in everyday places, we have to pass through a developmental phase that involves some risk. Putting a “Keep Out, Experiment in Progress” sign on the door is no longer possible since we are now at a level of capability that requires testing over long periods of time in complex realistic environments that contain people. We all know that controlling the risk is important – a serious accident could set the field back globally – but just as important is convincing others that the risks are known and controlled. In this article, we describe our experience going down this path and we show that mobile robotics research health and safety assessment is still unexplored territory in universities and is often ignored. We hope that the article will make robotics research labs in universities around the world take note of these issues rather than operating under the radar to prevent any catastrophic accidents.