Designing a Computer Vision System for Underwater Robotic Interventions

This thesis deals with the challenging problem of designing systems able to perceive objects in underwater environments. In the last few decades research activities in robotics have advanced the state of art regarding intervention capabilities of autonomous systems. State of art in fields such as localization and navigation, real time perception and cognition, safe action and manipulation capabilities, applied to ground environments (both indoor and outdoor) has now reached such a readiness level that it allows high level autonomous operations. On the opposite side, the underwater environment remains a very difficult one for autonomous robots. Water influences the mechanical and electrical design of systems, interferes with sensors by limiting their capabilities, heavily impacts on data transmissions, and generally requires systems with low power consumption in order to enable reasonable mission duration. Interest in underwater applications is driven by needs of exploring and intervening in environments in which human capabilities are very limited. Nowadays, most underwater field operations are carried out by manned or remotely operated vehicles, deployed for explorations and limited intervention missions. Manned vehicles, directly on-board controlled, expose human operators to risks related to the stay in field of the mission, within a hostile environment. Remotely Operated Vehicles (ROV) currently represent the most advanced technology for underwater intervention services available on the market. These vehicles can be remotely operated for long time but they need support from an oceanographic vessel with multiple teams of highly specialized pilots. Vehicles equipped with multiple state-of-art sensors and capable to autonomously plan missions have been deployed in the last ten years and exploited as observers for underwater fauna, seabed, ship wrecks, and so on. On the other hand, underwater operations like object recovery and equipment maintenance are still challenging tasks to be conducted without human supervision since they require object perception and localization with much higher accuracy and robustness, to a degree seldom available in Autonomous Underwater Vehicles (AUV). This thesis reports the study, from design to deployment and evaluation, of a general purpose and configurable platform dedicated to stereo-vision perception in underwater environments. Several aspects related to the peculiar environment characteristics have been taken into account during all stages of system design and evaluation: depth of operation and light conditions, together with water turbidity and external weather, heavily impact on perception capabilities. The vision platform proposed in this work is a modular system comprising off-the-shelf components for both the imaging sensors and the computational unit, linked by a high performance ethernet network bus. The adopted design philosophy aims at achieving high flexibility in terms of feasible perception applications, that should not be as limited as in case of a special-purpose and dedicated hardware. Flexibility is required by the variability of underwater environments, with water conditions ranging from clear to turbid, light backscattering varying with daylight and depth, strong color distortion, and other environmental factors. Furthermore, the proposed modular design ensures an easier maintenance and update of the system over time. Performance of the proposed system, in terms of perception capabilities, has been evaluated in several underwater contexts taking advantage of the opportunity offered by the MARIS national project. Design issues like energy power consumption, heat dissipation and network capabilities have been evaluated in different scenarios. Finally, real-world experiments, conducted in multiple and variable underwater contexts, including open sea waters, have led to the collection of several datasets that have been publicly released to the scientific community. The vision system has been integrated in a state of the art AUV equipped with a robotic arm and gripper, and has been exploited in the robot control loop to successfully perform underwater grasping operations.

The application of robotics to the turbine blade encapsulation process

A survey of the existing state-of-the-art of turbine blade manufacture highlights two operations that have not been automated namely that of loading of a turbine blade into an encapsulation die, and that of removing a machined blade from the encapsulation block. The automation of blade decapsulation has not been pursued. In order to develop a system to automate the loading of an encapsulation die a prototype mechanical handling robot has been designed together with a computer controlled encapsulation die. The robot has been designed as a mechanical handling robot of cylindrical geometry, suitable for use in a circular work cell. It is the prototype for a production model to be called `The Cybermate'. The prototype robot is mechanically complete but due to unforeseen circumstances the robot control system is not available (the development of the control system did not form a part of this project), hence it has not been possible to fully test and assess the robot mechanical design. Robot loading of the encapsulation die has thus been simulated. The research work with regard to the encapsulation die has focused on the development of computer controlled, hydraulically actuated, location pins. Such pins compensate for the inherent positional inaccuracy of the loading robot and reproduce the dexterity of the human operator. Each pin comprises a miniature hydraulic cylinder, controlled by a standard bidirectional flow control valve. The precision positional control is obtained through pulsing of the valves under software control, with positional feedback from an 8-bit transducer. A test-rig comprising one hydraulic location pin together with an opposing spring loaded pin has demonstrated that such a pin arrangement can be controlled with a repeatability of +/-.00045'. In addition this test-rig has demonstrated that such a pin arrangement can be used to gauge and compensate for the dimensional error of the component held between the pins, by offsetting the pin datum positions to allow for the component error. A gauging repeatability of +/- 0.00015' was demonstrated. This work has led to the design and manufacture of an encapsulation die comprising ten such pins and the associated computer software. All aspects of the control software except blade gauging and positional data storage have been demonstrated. Work is now required to achieve the accuracy of control demonstrated by the single pin test-rig, with each of the ten pins in the encapsulation die. This would allow trials of the complete loading cycle to take place.

Uma interface para o controle de robôs móveis por intermédio de gestos

The main objective of this work was to enable the recognition of human gestures through the development of a computer program. The program created captures the gestures executed by the user through a camera attached to the computer and sends it to the robot command referring to the gesture. They were interpreted in total ve gestures made by human hand. The software (developed in C ++) widely used the computer vision concepts and open source library OpenCV that directly impact the overall e ciency of the control of mobile robots. The computer vision concepts take into account the use of lters to smooth/blur the image noise reduction, color space to better suit the developer's desktop as well as useful information for manipulating digital images. The OpenCV library was essential in creating the project because it was possible to use various functions/procedures for complete control lters, image borders, image area, the geometric center of borders, exchange of color spaces, convex hull and convexity defect, plus all the necessary means for the characterization of imaged features. During the development of the software was the appearance of several problems, as false positives (noise), underperforming the insertion of various lters with sizes oversized masks, as well as problems arising from the choice of color space for processing human skin tones. However, after the development of seven versions of the control software, it was possible to minimize the occurrence of false positives due to a better use of lters combined with a well-dimensioned mask size (tested at run time) all associated with a programming logic that has been perfected over the construction of the seven versions. After all the development is managed software that met the established requirements. After the completion of the control software, it was observed that the overall e ectiveness of the various programs, highlighting in particular the V programs: 84.75 %, with VI: 93.00 % and VII with: 94.67 % showed that the nal program performed well in interpreting gestures, proving that it was possible the mobile robot control through human gestures without the need for external accessories to give it a better mobility and cost savings for maintain such a system. The great merit of the program was to assist capacity in demystifying the man set/machine therefore uses an easy and intuitive interface for control of mobile robots. Another important feature observed is that to control the mobile robot is not necessary to be close to the same, as to control the equipment is necessary to receive only the address that the Robotino passes to the program via network or Wi-Fi.

Improved joint control using a genetic algorithm for a humanoid robot

This paper describes experiments conducted in order to simultaneously tune 15 joints of a humanoid robot. Two Genetic Algorithm (GA) based tuning methods were developed and compared against a hand-tuned solution. The system was tuned in order to minimise tracking error while at the same time achieve smooth joint motion. Joint smoothness is crucial for the accurate calculation of online ZMP estimation, a prerequisite for a closedloop dynamically stable humanoid walking gait. Results in both simulation and on a real robot are presented, demonstrating the superior smoothness performance of the GA based methods.

Fuzzy associative memory for humanoid robot joint control

Traditional approaches to joint control required accurate modelling of the system dynamic of the plant in question. Fuzzy Associative Memory (FAM) control schemes allow adequate control without a model of the system to be controlled. This paper presents a FAM based joint controller implemented on a humanoid robot. An empirically tuned PI velocity control loop is augmented with this feed forward FAM, with considerable reduction in joint position error achieved online and with minimal additional computational overhead.

Biologically inspired joint control for a humanoid robot

The GuRoo is a 1.2 m tall, 23 degree of freedom humanoid constructed at the University of Queensland for research into humanoid robotics. The key challenge being addressed by the GuRoo project is the development of appropriate learning strategies for control and coordination of the robot's many joints. The development of learning strategies is seen as a way to side-step the inherent intricacy of modeling a multi-DOF biped robot. This paper outlines the approach taken to generate an appropriate control scheme for the joints of the GuRoo. The paper demonstrates the determination of local feedback control parameters using a genetic algorithm. The feedback loop is then augmented by a predictive modulator that learns a form of feed-forward control to overcome the irregular loads experienced at each joint during the gait cycle. The predictive modulator is based on the CMAC architecture. Results from tests on the GuRoo platform show that both systems provide improvements in stability and tracking of joint control.

Modelling and control of a quad-rotor robot

To date, most quad-rotor aerial robots have been based on flying toys. Although such systems can be used as prototypes, they are not sufficiently robust to serve as experimental robotics platforms. We have developed the X-4 Flyer, a quad-rotor robot using custom-built chassis and avionics with off-the-shelf motors and batteries, to be a highly reliable experimental platform. The vehicle uses tuned plant dynamics with an onboard embedded attitude controller to stabilise flight. A linear SISO controller was designed to regulate flyer attitude.

Cerebellar augmented joint control for a humanoid robot

The joints of a humanoid robot experience disturbances of markedly different magnitudes during the course of a walking gait. Consequently, simple feedback control techniques poorly track desired joint trajectories. This paper explores the addition of a control system inspired by the architecture of the cerebellum to improve system response. This system learns to compensate the changes in load that occur during a cycle of motion. The joint compensation scheme, called Trajectory Error Learning, augments the existing feedback control loop on a humanoid robot. The results from tests on the GuRoo platform show an improvement in system response for the system when augmented with the cerebellar compensator.

Distributed control of gait for a humanoid robot

This paper describes a walking gait for a humanoid robot with a distributed control system. The motion for the robot is calculated in real time on a central controller, and sent over CAN bus to the distributed control system. The distributed control system loosely follows the motion patterns from the central controller, while also acting to maintain stability and balance. There is no global feedback control system; the system maintains its balance by the interaction between central gait and soft control of the actuators. The paper illustrates a straight line walking gait and shows the interaction between gait generation and the control system. The analysis of the data shows that successful walking can be achieved without maintaining strict local joint control, and without explicit global balance coordination.

Learning multidimensional joint control of a robot using receding horizon locally weighted regression

In this paper we explore the ability of a recent model-based learning technique Receding Horizon Locally Weighted Regression (RH-LWR) useful for learning temporally dependent systems. In particular this paper investigates the application of RH-LWR to learn control of Multiple-input Multiple-output robot systems. RH-LWR is demonstrated through learning joint velocity and position control of a three Degree of Freedom (DoF) rigid body robot.

Modeling and control of a 3500 tonne mining robot

Draglines are extremely large machines that are widely used in open-cut coal mines for overburden stripping. Since 1994 we have been working toward the development of a computer control system capable of automatically driving a dragline for a large portion of its operating cycle. This has necessitated the development and experimental evaluation of sensor systems, machines models, closed-loop control controllers, and an operator interface. This paper describes our steps toward the goal through scale-model and full-scale field experimentation.

Adaptive compliance control of a multi‐legged stair‐climbing robot based on proprioceptive data

Purpose – The purpose of this paper is to describe an innovative compliance control architecture for hybrid multi‐legged robots. The approach was verified on the hybrid legged‐wheeled robot ASGUARD, which was inspired by quadruped animals. The adaptive compliance controller allows the system to cope with a variety of stairs, very rough terrain, and is also able to move with high velocity on flat ground without changing the control parameters. Design/methodology/approach – The paper shows how this adaptivity results in a versatile controller for hybrid legged‐wheeled robots. For the locomotion control we use an adaptive model of motion pattern generators. The control approach takes into account the proprioceptive information of the torques, which are applied on the legs. The controller itself is embedded on a FPGA‐based, custom designed motor control board. An additional proprioceptive inclination feedback is used to make the same controller more robust in terms of stair‐climbing capabilities. Findings – The robot is well suited for disaster mitigation as well as for urban search and rescue missions, where it is often necessary to place sensors or cameras into dangerous or inaccessible areas to get a better situation awareness for the rescue personnel, before they enter a possibly dangerous area. A rugged, waterproof and dust‐proof corpus and the ability to swim are additional features of the robot. Originality/value – Contrary to existing approaches, a pre‐defined walking pattern for stair‐climbing was not used, but an adaptive approach based only on internal sensor information. In contrast to many other walking pattern based robots, the direct proprioceptive feedback was used in order to modify the internal control loop, thus adapting the compliance of each leg on‐line.

Design and control of MIRA: A lightweight climbing robot for ship inspection

The inspection of marine vessels is currently performed manually. Inspectors use tools (e.g. cameras and devices for non-destructive testing) to detect damaged areas, cracks, and corrosion in large cargo holds, tanks, and other parts of a ship. Due to the size and complex geometry of most ships, ship inspection is time-consuming and expensive. The EU-funded project INCASS develops concepts for a marine inspection robotic assistant system to improve and automate ship inspections. In this paper, we introduce our magnetic wall–climbing robot: Marine Inspection Robotic Assistant (MIRA). This semiautonomous lightweight system is able to climb a vessels steel frame to deliver on-line visual inspection data. In addition, we describe the design of the robot and its building subsystems as well as its hardware and software components.

Component-based approach to the holonic control of a robot assembly cell

The aim of this paper is to describe the implementation of a new approach for the introduction of so called 'holonic manufacturing' principles into existing production control systems. Such an approach is intended to improve the reconfigurability of the control system to cope with the increasing requirements of production change. A conceptual architecture is described and implemented in a robot assembly cell to demonstrate that this approach can lead to a manufacturing control system which can adapt relatively simply to long-term change. A design methodology and migration strategy for achieving these solutions using conventional hardware is proposed to develop execution level of manufacturing control systems.

Software de control desarrollado en ROS para teleoperar de manera eficiente el robot Kuka Lightweight mediante el háptico Phantom Omni

La teleoperación o telerobótica es un campo de la robótica que se basa en el control remoto de robots esclavo por parte de un usuario encargado de gobernar, mediante un dispositivo maestro, la fuerza y movimiento del robot. Sobre dicho usuario recaen también las tareas de percepción del entorno, planificación y manipulación compleja. Concretamente se pretende desarrollar el control software necesario para teleoperar un manipulador esclavo, Kuka Lightweigh mediante un dispositivo háptico Phamton Omni, que se comporta como maestro, sin que afecten las diferencias dinámicas y estructurales existentes entre ambos dispositivos, aportando información adicional al operador para facilitar la operación. La principal motivación de la evolución de esta tecnología se debe a la necesidad de realizar trabajos en entornos hostiles, de difícil acceso, o perjudiciales para la salud del usuario.