Toward an object-based semantic memory for long-term operation of mobile service robots

Throughout a lifetime of operation, a mobile service robot needs to acquire, store and update its knowledge of a working environment. This includes the ability to identify and track objects in different places, as well as using this information for interaction with humans. This paper introduces a long-term updating mechanism, inspired by the modal model of human memory, to enable a mobile robot to maintain its knowledge of a changing environment. The memory model is integrated with a hybrid map that represents the global topology and local geometry of the environment, as well as the respective 3D location of objects. We aim to enable the robot to use this knowledge to help humans by suggesting the most likely locations of specific objects in its map. An experiment using omni-directional vision demonstrates the ability to track the movements of several objects in a dynamic environment over an extended period of time.

Simultaneous localization and planning on multiple map hypotheses

This paper presents a novel method to rank map hypotheses by the quality of localization they afford. The highest ranked hypothesis at any moment becomes the active representation that is used to guide the robot to its goal location. A single static representation is insufficient for navigation in dynamic environments where paths can be blocked periodically, a common scenario which poses significant challenges for typical planners. In our approach we simultaneously rank multiple map hypotheses by the influence that localization in each of them has on locally accurate odometry. This is done online for the current locally accurate window by formulating a factor graph of odometry relaxed by localization constraints. Comparison of the resulting perturbed odometry of each hypothesis with the original odometry yields a score that can be used to rank map hypotheses by their utility. We deploy the proposed approach on a real robot navigating a structurally noisy office environment. The configuration of the environment is physically altered outside the robots sensory horizon during navigation tasks to demonstrate the proposed approach of hypothesis selection.

Topological localization using optical flow descriptors

We propose a topological localization method based on optical flow information. We analyse the statistical characteristics of the optical flow signal and demonstrate that the flow vectors can be used to identify and describe key locations in the environment. The key locations (nodes) correspond to significant scene changes and depth discontinuities. Since optical flow vectors contain position, magnitude and angle information, for each node, we extract low and high order statistical moments of the vectors and use them as descriptors for that node. Once a database of nodes and their corresponding optical flow features is created, the robot can perform topological localization by using the Mahalanobis distance between the current frame and the database. This is supported by field trials, which illustrate the repeatability of the proposed method for detecting and describing key locations in indoor and outdoor environments in challenging and diverse lighting conditions.

Robotic explosive charging in mining and construction applications

Describes the development and testing of a robotic system for charging blast holes in underground mining. The automation system supports four main tactical functions: detection of blast holes; teleoperated arm pose control; automatic arm pose control; and human-in-the-loop visual servoing. We present the system architecture, and analyse the major components, Hole detection is crucial for automating the process, and we discuss theoretical and practical aspects in detail. The sensors used are laser range finders and cameras installed in the end effector. For automatic insertion, we consider image processing techniques to support visual servoing the tool to the hole. We also discuss issues surrounding the control of heavy-duty mining manipulators, in particular, friction, stiction, and actuator saturation.

Practical visual odometry for car-like vehicles

A method for calculating visual odometry for ground vehicles with car-like kinematic motion constraints similar to Ackerman's steering model is presented. By taking advantage of this non-holonomic driving constraint we show a simple and practical solution to the odometry calculation by clever placement of a single camera. The method has been implemented successfully on a large industrial forklift and a Toyota Prado SUV. Results from our industrial test site is presented demonstrating the applicability of this method as a replacement for wheel encoder-based odometry for these vehicles.

Deletions and other pleasures : Episode 1: The pleasures of science fiction

"Christy Dena’s online-remix-narrative takes iconic images of popular culture and builds with them a strange world where the human fallibility is programmatically deleted. Both dystopic and playful, Dena’s work is an ironic reimagining of pleasure as a state of robotic flatlining, using tropes of science fiction to critique processes of social normalisation and increasing alienation from emotionality." This creative response began as a completely different story and form. What excited me in the end was the concept of deletion and how it could be an interesting mechanic: where the only thing you can do in the world is delete. I thought about deleting parts of robots to make them better. Healing comes from taking away, from removing things. Memories of Joseph Weizenbaum’s chatbot ELIZA came flooding back: where the (human) player is a patient talking to a Rogerian psychotherapist. But in this work I’m switching the roles and making the player the doctor, a doctor to robots…a doctor that can only prescribe deletions. I conceived of the work as a branching narrative, and started writing it in Twine. With every robot patient, the player chose one of many deletions. But when I realised I wouldn’t be able to arrange an artist and sound designer I looked for another option. I played with Zeega and felt that I could get the mood I was after with that platform. So the piece transformed into a work where the player/viewer is imprisoned in the decisions of the deleting protagonist…which has its own effect on the experience and meaning.

Shared autonomy for close-quarters navigation and control of a VTOL platform

This thesis presents an approach for a vertical infrastructure inspection using a vertical take-off and landing (VTOL) unmanned aerial vehicle and shared autonomy. Inspecting vertical structure such as light and power distribution poles is a difficult task. There are challenges involved with developing such an inspection system, such as flying in close proximity to a target while maintaining a fixed stand-off distance from it. The contributions of this thesis fall into three main areas. Firstly, an approach to vehicle dynamic modeling is evaluated in simulation and experiments. Secondly, EKF-based state estimators are demonstrated, as well as estimator-free approaches such as image based visual servoing (IBVS) validated with motion capture ground truth data. Thirdly, an integrated pole inspection system comprising a VTOL platform with human-in-the-loop control, (shared autonomy) is demonstrated. These contributions are comprehensively explained through a series of published papers.

Inspection of pole-like structures using a vision-controlled VTOL UAV and shared autonomy

We present an approach for the inspection of vertical pole-like infrastructure using a vertical take-off and landing (VTOL) unmanned aerial vehicle and shared autonomy. Inspecting vertical structures, such as light and power distribution poles, is a time consuming, dangerous and expensive task with high operator workload. To address these issues, we propose a VTOL platform that can operate at close-quarters, whilst maintaining a safe stand-off distance and rejecting environmental disturbances. We adopt an Image based Visual Servoing (IBVS) technique using only two line features to stabilise the vehicle with respect to a pole. Visual, inertial and sonar data are used, making the approach suitable for indoor or GPS-denied environments. Results from simulation and outdoor flight experiments demonstrate the system is able to successfully inspect and circumnavigate a pole.

Text recognition approaches for indoor robotics : a comparison

This paper evaluates the performance of different text recognition techniques for a mobile robot in an indoor (university campus) environment. We compared four different methods: our own approach using existing text detection methods (Minimally Stable Extremal Regions detector and Stroke Width Transform) combined with a convolutional neural network, two modes of the open source program Tesseract, and the experimental mobile app Google Goggles. The results show that a convolutional neural network combined with the Stroke Width Transform gives the best performance in correctly matched text on images with single characters whereas Google Goggles gives the best performance on images with multiple words. The dataset used for this work is released as well.

Force controlled hexapod walking

This thesis is a study on controlling methods for six-legged robots. The study is based on mathematical modeling and simulation. A new joint controller is proposed and tested in simulation that uses joint angles and leg reaction force as inputs to generate a torque, and a method to optimise this controller is formulated and validated. Simulation shows that hexapod can walk on flat ground based on PID controllers with just four target configurations and a set of leg coordination rules, which provided the basis for the design of the new controller.

Towards a spatial language for mobile robots

We present a framework and first set of simulations for evolving a language for communicating about space. The framework comprises two components: (1) An established mobile robot platform, RatSLAM, which has a "brain" architecture based on rodent hippocampus with the ability to integrate visual and odometric cues to create internal maps of its environment. (2) A language learning system based on a neural network architecture that has been designed and implemented with the ability to evolve generalizable languages which can be learned by naive learners. A study using visual scenes and internal maps streamed from the simulated world of the robots to evolve languages is presented. This study investigated the structure of the evolved languages showing that with these inputs, expressive languages can effectively categorize the world. Ongoing studies are extending these investigations to evolve languages that use the full power of the robots representations in populations of agents.

Towards coordinated multirobot missions for lunar sample collection in an unknown environment

In this paper, the recent results of the space project IMPERA are presented. The goal of IMPERA is the development of a multirobot planning and plan execution architecture with a focus on a lunar sample collection scenario in an unknown environment. We describe the implementation and verification of different modules that are integrated into a distributed system architecture. The modules include a mission planning approach for a multirobot system and modules for task and skill execution within a lunar use-case scenario. The skills needed for the test scenario include cooperative exploration and mapping strategies for an unknown environment, the localization and classification of sample containers using a novel approach of semantic perception, and the skill of transporting sample containers to a collection point using a mobile manipulation robot. Additionally, we present our approach of a reliable communication framework that can deal with communication loss during the mission. Several modules are tested within several experiments in the domain of planning and plan execution, communication, coordinated exploration, perception, and object transportation. An overall system integration is tested on a mission scenario experiment using three robots.

Simultaneous underwater visibility assessment, enhancement and improved stereo

Vision-based underwater navigation and obstacle avoidance demands robust computer vision algorithms, particularly for operation in turbid water with reduced visibility. This paper describes a novel method for the simultaneous underwater image quality assessment, visibility enhancement and disparity computation to increase stereo range resolution under dynamic, natural lighting and turbid conditions. The technique estimates the visibility properties from a sparse 3D map of the original degraded image using a physical underwater light attenuation model. Firstly, an iterated distance-adaptive image contrast enhancement enables a dense disparity computation and visibility estimation. Secondly, using a light attenuation model for ocean water, a color corrected stereo underwater image is obtained along with a visibility distance estimate. Experimental results in shallow, naturally lit, high-turbidity coastal environments show the proposed technique improves range estimation over the original images as well as image quality and color for habitat classification. Furthermore, the recursiveness and robustness of the technique allows implementation onboard an Autonomous Underwater Vehicle for improving navigation and obstacle avoidance performance.

Towards coordinated vision-based docking using an Autonomous Surface Vehicle

This paper describes the development and experimental evaluation of a novel vision-based Autonomous Surface Vehicle with the purpose of performing coordinated docking manoeuvres with a target, such as an Autonomous Underwater Vehicle, on the water’s surface. The system architecture integrates two small processor units; the first performs vehicle control and implements a virtual force obstacle avoidance and docking strategy, with the second performing vision-based target segmentation and tracking. Furthermore, the architecture utilises wireless sensor network technology allowing the vehicle to be observed by, and even integrated within an ad-hoc sensor network. The system performance is demonstrated through real-world experiments.