Human-robot interaction for telemanipulation in large workspaces

Independientemente de la existencia de técnicas altamente sofisticadas y capacidades de cómputo cada vez más elevadas, los problemas asociados a los robots que interactúan con entornos no estructurados siguen siendo un desafío abierto en robótica. A pesar de los grandes avances de los sistemas robóticos autónomos, hay algunas situaciones en las que una persona en el bucle sigue siendo necesaria. Ejemplos de esto son, tareas en entornos de fusión nuclear, misiones espaciales, operaciones submarinas y cirugía robótica. Esta necesidad se debe a que las tecnologías actuales no pueden realizar de forma fiable y autónoma cualquier tipo de tarea. Esta tesis presenta métodos para la teleoperación de robots abarcando distintos niveles de abstracción que van desde el control supervisado, en el que un operador da instrucciones de alto nivel en la forma de acciones, hasta el control bilateral, donde los comandos toman la forma de señales de control de bajo nivel. En primer lugar, se presenta un enfoque para llevar a cabo la teleoperación supervisada de robots humanoides. El objetivo es controlar robots terrestres capaces de ejecutar tareas complejas en entornos de búsqueda y rescate utilizando enlaces de comunicación limitados. Esta propuesta incorpora comportamientos autónomos que el operador puede utilizar para realizar tareas de navegación y manipulación mientras se permite cubrir grandes áreas de entornos remotos diseñados para el acceso de personas. Los resultados experimentales demuestran la eficacia de los métodos propuestos. En segundo lugar, se investiga el uso de dispositivos rentables para telemanipulación guiada. Se presenta una aplicación que involucra un robot humanoide bimanual y un traje de captura de movimiento basado en sensores inerciales. En esta aplicación, se estudian las capacidades de adaptación introducidas por el factor humano y cómo estas pueden compensar la falta de sistemas robóticos de alta precisión. Este trabajo es el resultado de una colaboración entre investigadores del Biorobotics Laboratory de la Universidad de Harvard y el Centro de Automática y Robótica UPM-CSIC. En tercer lugar, se presenta un nuevo controlador háptico que combina velocidad y posición. Este controlador bilateral híbrido hace frente a los problemas relacionados con la teleoperación de un robot esclavo con un gran espacio de trabajo usando un dispositivo háptico pequeño como maestro. Se pueden cubrir amplias áreas de trabajo al cambiar automáticamente entre los modos de control de velocidad y posición. Este controlador háptico es ideal para sistemas maestro-esclavo con cinemáticas diferentes, donde los comandos se transmiten en el espacio de la tarea del entorno remoto. El método es validado para realizar telemanipulación hábil de objetos con un robot industrial. Por último, se introducen dos contribuciones en el campo de la manipulación robótica. Por un lado, se presenta un nuevo algoritmo de cinemática inversa, llamado método iterativo de desacoplamiento cinemático. Este método se ha desarrollado para resolver el problema cinemático inverso de un tipo de robot de seis grados de libertad donde una solución cerrada no está disponible. La eficacia del método se compara con métodos numéricos convencionales. Además, se ha diseñado una taxonomía robusta de agarres que permite controlar diferentes manos robóticas utilizando una correspondencia, basada en gestos, entre los espacios de trabajo de la mano humana y de la mano robótica. El gesto de la mano humana se identifica mediante la lectura de los movimientos relativos del índice, el pulgar y el dedo medio del usuario durante las primeras etapas del agarre. ABSTRACT Regardless of the availability of highly sophisticated techniques and ever increasing computing capabilities, the problems associated with robots interacting with unstructured environments remains an open challenge. Despite great advances in autonomous robotics, there are some situations where a humanin- the-loop is still required, such as, nuclear, space, subsea and robotic surgery operations. This is because the current technologies cannot reliably perform all kinds of task autonomously. This thesis presents methods for robot teleoperation strategies at different levels of abstraction ranging from supervisory control, where the operator gives high-level task actions, to bilateral teleoperation, where the commands take the form of low-level control inputs. These strategies contribute to improve the current human-robot interfaces specially in the case of slave robots deployed at large workspaces. First, an approach to perform supervisory teleoperation of humanoid robots is presented. The goal is to control ground robots capable of executing complex tasks in disaster relief environments under constrained communication links. This proposal incorporates autonomous behaviors that the operator can use to perform navigation and manipulation tasks which allow covering large human engineered areas of the remote environment. The experimental results demonstrate the efficiency of the proposed methods. Second, the use of cost-effective devices for guided telemanipulation is investigated. A case study involving a bimanual humanoid robot and an Inertial Measurement Unit (IMU) Motion Capture (MoCap) suit is introduced. Herein, it is corroborated how the adaptation capabilities offered by the human-in-the-loop factor can compensate for the lack of high-precision robotic systems. This work is the result of collaboration between researchers from the Harvard Biorobotics Laboratory and the Centre for Automation and Robotics UPM-CSIC. Thirdly, a new haptic rate-position controller is presented. This hybrid bilateral controller copes with the problems related to the teleoperation of a slave robot with large workspace using a small haptic device as master. Large workspaces can be covered by automatically switching between rate and position control modes. This haptic controller is ideal to couple kinematic dissimilar master-slave systems where the commands are transmitted in the task space of the remote environment. The method is validated to perform dexterous telemanipulation of objects with a robotic manipulator. Finally, two contributions for robotic manipulation are introduced. First, a new algorithm, the Iterative Kinematic Decoupling method, is presented. It is a numeric method developed to solve the Inverse Kinematics (IK) problem of a type of six-DoF robotic arms where a close-form solution is not available. The effectiveness of this IK method is compared against conventional numerical methods. Second, a robust grasp mapping has been conceived. It allows to control a wide range of different robotic hands using a gesture based correspondence between the human hand space and the robotic hand space. The human hand gesture is identified by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of grasping.

A Motor-less and Gear-less Bio-mimetic Robotic Fish Design

In this paper, we describe our current work on bio-inspired locomotion systems using a deformable structure and smart materials, concretely Shape Memory Alloys, exploring the possibility of building motor-less and gear-less robots. A swimming underwater robot has been developed whose movements are generated using such actuators, used for bending the backbone of the fish, which in turn causes a change on the curvature of the body. This paper focuses on how standard swimming patterns can be reproduced with the proposed design, using an actuation dynamics model identified in prior work.

Cooperative Task Execution between Modular Robots Based on Tight-Loose Cooperation Strategies

The complexity in the execution of cooperative tasks is high due to the fact that a robot team requires movement coordination at the beginning of the mission and continuous coordination during the execution of the task. A variety of techniques have been proposed to give a solution to this problem assuming standard mobile robots. This work focuses on presenting the execution of a cooperative task by a modular robot team. The complexity of the task execution increases due to the fact that each robot is composed of modules which have to be coordinated in a proper way to successfully work. A combined tight and loose cooperation strategy is presented and a bar-pushing example is used as a cooperative task to show the performance of this type of system.

A waypoint-based mission planner for a farmland coverage with an aerial robot - a precision farming tool

Remote sensing (RS) with aerial robots is becoming more usual in every day time in Precision Agriculture (PA) practices, do to their advantages over conventional methods. Usually, available commercial platforms providing off-the-shelf waypoint navigation are adopted to perform visual surveys over crop fields, with the purpose to acquire specific image samples. The way in which a waypoint list is computed and dispatched to the aerial robot when mapping non empty agricultural workspaces has not been yet discussed. In this paper we propose an offline mission planner approach that computes an efficient coverage path subject to some constraints by decomposing the environment approximately into cells. Therefore, the aim of this work is contributing with a feasible waypoints-based tool to support PA practices

Visual Servoing for the Robotenis System: a Strategy for a 3 DOF Parallel Robot to Hit a Ping-Pong Ball

This article describes a new visual servo control and strategies that are used to carry out dynamic tasks by the Robotenis platform. This platform is basically a parallel robot that is equipped with an acquisition and processing system of visual information, its main feature is that it has a completely open architecture control, and planned in order to design, implement, test and compare control strategies and algorithms (visual and actuated joint controllers). Following sections describe a new visual control strategy specially designed to track and intercept objects in 3D space. The results are compared with a controller shown in previous woks, where the end effector of the robot keeps a constant distance from the tracked object. In this work, the controller is specially designed in order to allow changes in the tracking reference. Changes in the tracking reference can be used to grip an object that is under movement, or as in this case, hitting a hanging Ping-Pong ball. Lyapunov stability is taken into account in the controller design.

A method for Kinematic Calibration of a Parallel Robot by using one camera in hand and a spherical object

The main purpose of robot calibration is the correction of the possible errors in the robot parameters. This paper presents a method for a kinematic calibration of a parallel robot that is equipped with one camera in hand. In order to preserve the mechanical configuration of the robot, the camera is utilized to acquire incremental positions of the end effector from a spherical object that is fixed in the word reference frame. The positions of the end effector are related to incremental positions of resolvers of the motors of the robot, and a kinematic model of the robot is used to find a new group of parameters which minimizes errors in the kinematic equations. Additionally, properties of the spherical object and intrinsic camera parameters are utilized to model the projection of the object in the image and improving spatial measurements. Finally, the robotic system is designed to carry out tracking tasks and the calibration of the robot is validated by means of integrating the errors of the visual controller.

Analysis and 2D Simulation of a Hexapod Robot Leg for Remote Exploration

A walking machine is a wheeled rover alternative, well suited for work in an unstructured environment and specially in abrupt terrain. They have some drawback like speed and power consumption, but they can achieve complex movements and protrude very little the environment they are working on. The locomotion system is determined by the terrain conditions and, in our case, this legged design has been chosen based in a working area like Rio Tinto in the South of Spain, which is a river area with abrupt terrain. A walking robot with so many degrees of freedom can be a challenge when dealing with the analysis and simulations of the legs. This paper shows how to deal with the kinematical analysis of the equations of a hexapod robot based on a design developed by the Center of Astrobiology INTA-CSIC following the classical formulation of equations

Efficiency and automation in ATM

Sustainable growth of aviation whilst respecting the environment. • ANSPs and Aircraft Operators (AO) have to improve the way they operate

HALA!'s White Paper. Towards Higher Levels of Automation in ATM

• Objectives of HALA! • Main Activities • HALA! Magement Team • Participants • Intended Audience • Heritage in ATM and Automation • The new paradigm shift in Automation in ATM • Overall system performance as main driver for ATM Automation • The three interdependent dimensions for the paradigm change. • New roles assignment based on : • “best time” • “decision place” • “best player” • HALA! main research areas

Using the Own Flexibility of a Climbing Robot as a Double Force Sensor

Force sensors are used when interaction tasks are carried out by robots in general, and by climbing robots in particular. If the mechanics and electronics systems are contained inside the own robot, the robot becomes portable without external control. Commercial force sensors cannot be used due to limited space and weight. By selecting the links material with appropriate stiffness and placing strain gauges on the structure, the own robot flexibility can be used such as force sensor. Thus, forces applied on the robot tip can be measured without additional external devices. Only gauges and small internal electronic converters are necessary. This paper illustrates the proposed algorithm to achieve these measurements. Additionally, experimental results are presented.

ROAD: Domestic Assistant and Rehabilitation Robot

This study introduces the concept design and analysis of a robotic system for the assistance and rehabilitation of disabled people. Based on the statistical data of the most common types of disabilities in Spain and other industrialized countries, the different tasks that the device must be able to perform have been determined. In this study, different robots for rehabilitation and assistance previously introduced have been reviewed. This survey is focused on those robots that assist with gait, balance and standing up. The structure of the ROAD robot presents various advantages over these robots, we discuss some of them. The performance of the proposed architecture is analyzed when it performs the sit to stand activity.

Biomechanics of smart wings in a bat robot: morphing wings using SMA actuators

This paper presents the design of a bat-like micro aerial vehicle with actuated morphing wings. NiTi shape memory alloys (SMAs) acting as artificial biceps and triceps muscles are used for mimicking the morphing wing mechanism of the bat flight apparatus. Our objective is twofold. Firstly, we have implemented a control architecture that allows an accurate and fast SMA actuation. This control makes use of the electrical resistance measurements of SMAs to adjust morphing wing motions. Secondly, the feasibility of using SMA actuation technology is evaluated for the application at hand. To this purpose, experiments are conducted to analyze the control performance in terms of nominal and overloaded operation modes of the SMAs. This analysis includes: (i) inertial forces regarding the stretchable wing membrane and aerodynamic loads, and (ii) uncertainties due to impact of airflow conditions over the resistance–motion relationship of SMAs. With the proposed control, morphing actuation speed can be increased up to 2.5 Hz, being sufficient to generate lift forces at a cruising speed of 5ms−1.

On the biomimetic design of agile-robot legs

The development of functional legged robots has encountered its limits in human-made actuation technology. This paper describes research on the biomimetic design of legs for agile quadrupeds. A biomimetic leg concept that extracts key principles from horse legs which are responsible for the agile and powerful locomotion of these animals is presented. The proposed biomimetic leg model defines the effective leg length, leg kinematics, limb mass distribution, actuator power, and elastic energy recovery as determinants of agile locomotion, and values for these five key elements are given. The transfer of the extracted principles to technological instantiations is analyzed in detail, considering the availability of current materials, structures and actuators. A real leg prototype has been developed following the biomimetic leg concept proposed. The actuation system is based on the hybrid use of series elasticity and magneto-rheological dampers which provides variable compliance for natural motion. From the experimental evaluation of this prototype, conclusions on the current technological barriers to achieve real functional legged robots to walk dynamically in agile locomotion are presented.

BaTboT: a biologically inspired flapping and morphing bat robot actuated by SMA-based artificial muscles

Bats are animals that posses high maneuvering capabilities. Their wings contain dozens of articulations that allow the animal to perform aggressive maneuvers by means of controlling the wing shape during flight (morphing-wings). There is no other flying creature in nature with this level of wing dexterity and there is biological evidence that the inertial forces produced by the wings have a key role in the attitude movements of the animal. This can inspire the design of highly articulated morphing-wing micro air vehicles (not necessarily bat-like) with a significant wing-to-body mass ratio. This thesis presents the development of a novel bat-like micro air vehicle (BaTboT) inspired by the morphing-wing mechanism of bats. BaTboT’s morphology is alike in proportion compared to its biological counterpart Cynopterus brachyotis, which provides the biological foundations for developing accurate mathematical models and methods that allow for mimicking bat flight. In nature bats can achieve an amazing level of maneuverability by combining flapping and morphing wingstrokes. Attempting to reproduce the biological wing actuation system that provides that kind of motion using an artificial counterpart requires the analysis of alternative actuation technologies more likely muscle fiber arrays instead of standard servomotor actuators. Thus, NiTinol Shape Memory Alloys (SMAs) acting as artificial biceps and triceps muscles are used for mimicking the morphing wing mechanism of the bat flight apparatus. This antagonistic configuration of SMA-muscles response to an electrical heating power signal to operate. This heating power is regulated by a proper controller that allows for accurate and fast SMA actuation. Morphing-wings will enable to change wings geometry with the unique purpose of enhancing aerodynamics performance. During the downstroke phase of the wingbeat motion both wings are fully extended aimed at increasing the area surface to properly generate lift forces. Contrary during the upstroke phase of the wingbeat motion both wings are retracted to minimize the area and thus reducing drag forces. Morphing-wings do not only improve on aerodynamics but also on the inertial forces that are key to maneuver. Thus, a modeling framework is introduced for analyzing how BaTboT should maneuver by means of changing wing morphology. This allows the definition of requirements for achieving forward and turning flight according to the kinematics of the wing modulation. Motivated by the biological fact about the influence of wing inertia on the production of body accelerations, an attitude controller is proposed. The attitude control law incorporates wing inertia information to produce desired roll (φ) and pitch (θ) acceleration commands. This novel flight control approach is aimed at incrementing net body forces (Fnet) that generate propulsion. Mimicking the way how bats take advantage of inertial and aerodynamical forces produced by the wings in order to both increase lift and maneuver is a promising way to design more efficient flapping/morphing wings MAVs. The novel wing modulation strategy and attitude control methodology proposed in this thesis provide a totally new way of controlling flying robots, that eliminates the need of appendices such as flaps and rudders, and would allow performing more efficient maneuvers, especially useful in confined spaces. As a whole, the BaTboT project consists of five major stages of development: - Study and analysis of biological bat flight data reported in specialized literature aimed at defining design and control criteria. - Formulation of mathematical models for: i) wing kinematics, ii) dynamics, iii) aerodynamics, and iv) SMA muscle-like actuation. It is aimed at modeling the effects of modulating wing inertia into the production of net body forces for maneuvering. - Bio-inspired design and fabrication of: i) skeletal structure of wings and body, ii) SMA muscle-like mechanisms, iii) the wing-membrane, and iv) electronics onboard. It is aimed at developing the bat-like platform (BaTboT) that allows for testing the methods proposed. - The flight controller: i) control of SMA-muscles (morphing-wing modulation) and ii) flight control (attitude regulation). It is aimed at formulating the proper control methods that allow for the proper modulation of BaTboT’s wings. - Experiments: it is aimed at quantifying the effects of properly wing modulation into aerodynamics and inertial production for maneuvering. It is also aimed at demonstrating and validating the hypothesis of improving flight efficiency thanks to the novel control methods presented in this thesis. This thesis introduces the challenges and methods to address these stages. Windtunnel experiments will be oriented to discuss and demonstrate how the wings can considerably affect the dynamics/aerodynamics of flight and how to take advantage of wing inertia modulation that the morphing-wings enable to properly change wings’ geometry during flapping. Resumen: Los murciélagos son mamíferos con una alta capacidad de maniobra. Sus alas están conformadas por docenas de articulaciones que permiten al animal maniobrar gracias al cambio geométrico de las alas durante el vuelo. Esta característica es conocida como (alas mórficas). En la naturaleza, no existe ningún especimen volador con semejante grado de dexteridad de vuelo, y se ha demostrado, que las fuerzas inerciales producidas por el batir de las alas juega un papel fundamental en los movimientos que orientan al animal en vuelo. Estas características pueden inspirar el diseño de un micro vehículo aéreo compuesto por alas mórficas con redundantes grados de libertad, y cuya proporción entre la masa de sus alas y el cuerpo del robot sea significativa. Esta tesis doctoral presenta el desarrollo de un novedoso robot aéreo inspirado en el mecanismo de ala mórfica de los murciélagos. El robot, llamado BaTboT, ha sido diseñado con parámetros morfológicos muy similares a los descritos por su símil biológico Cynopterus brachyotis. El estudio biológico de este especimen ha permitido la definición de criterios de diseño y modelos matemáticos que representan el comportamiento del robot, con el objetivo de imitar lo mejor posible la biomecánica de vuelo de los murciélagos. La biomecánica de vuelo está definida por dos tipos de movimiento de las alas: aleteo y cambio de forma. Intentar imitar como los murciélagos cambian la forma de sus alas con un prototipo artificial, requiere el análisis de métodos alternativos de actuación que se asemejen a la biomecánica de los músculos que actúan las alas, y evitar el uso de sistemas convencionales de actuación como servomotores ó motores DC. En este sentido, las aleaciones con memoria de forma, ó por sus siglas en inglés (SMA), las cuales son fibras de NiTinol que se contraen y expanden ante estímulos térmicos, han sido usados en este proyecto como músculos artificiales que actúan como bíceps y tríceps de las alas, proporcionando la funcionalidad de ala mórfica previamente descrita. De esta manera, los músculos de SMA son mecánicamente posicionados en una configuración antagonista que permite la rotación de las articulaciones del robot. Los actuadores son accionados mediante una señal de potencia la cual es regulada por un sistema de control encargado que los músculos de SMA respondan con la precisión y velocidad deseada. Este sistema de control mórfico de las alas permitirá al robot cambiar la forma de las mismas con el único propósito de mejorar el desempeño aerodinámico. Durante la fase de bajada del aleteo, las alas deben estar extendidas para incrementar la producción de fuerzas de sustentación. Al contrario, durante el ciclo de subida del aleteo, las alas deben contraerse para minimizar el área y reducir las fuerzas de fricción aerodinámica. El control de alas mórficas no solo mejora el desempeño aerodinámico, también impacta la generación de fuerzas inerciales las cuales son esenciales para maniobrar durante el vuelo. Con el objetivo de analizar como el cambio de geometría de las alas influye en la definición de maniobras y su efecto en la producción de fuerzas netas, simulaciones y experimentos han sido llevados a cabo para medir cómo distintos patrones de modulación de las alas influyen en la producción de aceleraciones lineales y angulares. Gracias a estas mediciones, se propone un control de vuelo, ó control de actitud, el cual incorpora información inercial de las alas para la definición de referencias de aceleración angular. El objetivo de esta novedosa estrategia de control radica en el incremento de fuerzas netas para la adecuada generación de movimiento (Fnet). Imitar como los murciélagos ajustan sus alas con el propósito de incrementar las fuerzas de sustentación y mejorar la maniobra en vuelo es definitivamente un tópico de mucho interés para el diseño de robots aéros mas eficientes. La propuesta de control de vuelo definida en este trabajo de investigación podría dar paso a una nueva forma de control de vuelo de robots aéreos que no necesitan del uso de partes mecánicas tales como alerones, etc. Este control también permitiría el desarrollo de vehículos con mayor capacidad de maniobra. El desarrollo de esta investigación se centra en cinco etapas: - Estudiar y analizar el vuelo de los murciélagos con el propósito de definir criterios de diseño y control. - Formular modelos matemáticos que describan la: i) cinemática de las alas, ii) dinámica, iii) aerodinámica, y iv) actuación usando SMA. Estos modelos permiten estimar la influencia de modular las alas en la producción de fuerzas netas. - Diseño y fabricación de BaTboT: i) estructura de las alas y el cuerpo, ii) mecanismo de actuación mórfico basado en SMA, iii) membrana de las alas, y iv) electrónica abordo. - Contro de vuelo compuesto por: i) control de la SMA (modulación de las alas) y ii) regulación de maniobra (actitud). - Experimentos: están enfocados en poder cuantificar cuales son los efectos que ejercen distintos perfiles de modulación del ala en el comportamiento aerodinámico e inercial. El objetivo es demostrar y validar la hipótesis planteada al inicio de esta investigación: mejorar eficiencia de vuelo gracias al novedoso control de orientación (actitud) propuesto en este trabajo. A lo largo del desarrollo de cada una de las cinco etapas, se irán presentando los retos, problemáticas y soluciones a abordar. Los experimentos son realizados utilizando un túnel de viento con la instrumentación necesaria para llevar a cabo las mediciones de desempeño respectivas. En los resultados se discutirá y demostrará que la inercia producida por las alas juega un papel considerable en el comportamiento dinámico y aerodinámico del sistema y como poder tomar ventaja de dicha característica para regular patrones de modulación de las alas que conduzcan a mejorar la eficiencia del robot en futuros vuelos.

Game Theory Models for Multi-Robot Patrolling of Infraestructures

Abstract This work is focused on the problem of performing multi‐robot patrolling for infrastructure security applications in order to protect a known environment at critical facilities. Thus, given a set of robots and a set of points of interest, the patrolling task consists of constantly visiting these points at irregular time intervals for security purposes. Current existing solutions for these types of applications are predictable and inflexible. Moreover, most of the previous centralized and deterministic solutions and only few efforts have been made to integrate dynamic methods. Therefore, the development of new dynamic and decentralized collaborative approaches in order to solve the aforementioned problem by implementing learning models from Game Theory. The model selected in this work that includes belief‐based and reinforcement models as special cases is called Experience‐Weighted Attraction. The problem has been defined using concepts of Graph Theory to represent the environment in order to work with such Game Theory techniques. Finally, the proposed methods have been evaluated experimentally by using a patrolling simulator. The results obtained have been compared with previous available