Human-robot interaction for telemanipulation in large workspaces

Independientemente de la existencia de técnicas altamente sofisticadas y capacidades de cómputo cada vez más elevadas, los problemas asociados a los robots que interactúan con entornos no estructurados siguen siendo un desafío abierto en robótica. A pesar de los grandes avances de los sistemas robóticos autónomos, hay algunas situaciones en las que una persona en el bucle sigue siendo necesaria. Ejemplos de esto son, tareas en entornos de fusión nuclear, misiones espaciales, operaciones submarinas y cirugía robótica. Esta necesidad se debe a que las tecnologías actuales no pueden realizar de forma fiable y autónoma cualquier tipo de tarea. Esta tesis presenta métodos para la teleoperación de robots abarcando distintos niveles de abstracción que van desde el control supervisado, en el que un operador da instrucciones de alto nivel en la forma de acciones, hasta el control bilateral, donde los comandos toman la forma de señales de control de bajo nivel. En primer lugar, se presenta un enfoque para llevar a cabo la teleoperación supervisada de robots humanoides. El objetivo es controlar robots terrestres capaces de ejecutar tareas complejas en entornos de búsqueda y rescate utilizando enlaces de comunicación limitados. Esta propuesta incorpora comportamientos autónomos que el operador puede utilizar para realizar tareas de navegación y manipulación mientras se permite cubrir grandes áreas de entornos remotos diseñados para el acceso de personas. Los resultados experimentales demuestran la eficacia de los métodos propuestos. En segundo lugar, se investiga el uso de dispositivos rentables para telemanipulación guiada. Se presenta una aplicación que involucra un robot humanoide bimanual y un traje de captura de movimiento basado en sensores inerciales. En esta aplicación, se estudian las capacidades de adaptación introducidas por el factor humano y cómo estas pueden compensar la falta de sistemas robóticos de alta precisión. Este trabajo es el resultado de una colaboración entre investigadores del Biorobotics Laboratory de la Universidad de Harvard y el Centro de Automática y Robótica UPM-CSIC. En tercer lugar, se presenta un nuevo controlador háptico que combina velocidad y posición. Este controlador bilateral híbrido hace frente a los problemas relacionados con la teleoperación de un robot esclavo con un gran espacio de trabajo usando un dispositivo háptico pequeño como maestro. Se pueden cubrir amplias áreas de trabajo al cambiar automáticamente entre los modos de control de velocidad y posición. Este controlador háptico es ideal para sistemas maestro-esclavo con cinemáticas diferentes, donde los comandos se transmiten en el espacio de la tarea del entorno remoto. El método es validado para realizar telemanipulación hábil de objetos con un robot industrial. Por último, se introducen dos contribuciones en el campo de la manipulación robótica. Por un lado, se presenta un nuevo algoritmo de cinemática inversa, llamado método iterativo de desacoplamiento cinemático. Este método se ha desarrollado para resolver el problema cinemático inverso de un tipo de robot de seis grados de libertad donde una solución cerrada no está disponible. La eficacia del método se compara con métodos numéricos convencionales. Además, se ha diseñado una taxonomía robusta de agarres que permite controlar diferentes manos robóticas utilizando una correspondencia, basada en gestos, entre los espacios de trabajo de la mano humana y de la mano robótica. El gesto de la mano humana se identifica mediante la lectura de los movimientos relativos del índice, el pulgar y el dedo medio del usuario durante las primeras etapas del agarre. ABSTRACT Regardless of the availability of highly sophisticated techniques and ever increasing computing capabilities, the problems associated with robots interacting with unstructured environments remains an open challenge. Despite great advances in autonomous robotics, there are some situations where a humanin- the-loop is still required, such as, nuclear, space, subsea and robotic surgery operations. This is because the current technologies cannot reliably perform all kinds of task autonomously. This thesis presents methods for robot teleoperation strategies at different levels of abstraction ranging from supervisory control, where the operator gives high-level task actions, to bilateral teleoperation, where the commands take the form of low-level control inputs. These strategies contribute to improve the current human-robot interfaces specially in the case of slave robots deployed at large workspaces. First, an approach to perform supervisory teleoperation of humanoid robots is presented. The goal is to control ground robots capable of executing complex tasks in disaster relief environments under constrained communication links. This proposal incorporates autonomous behaviors that the operator can use to perform navigation and manipulation tasks which allow covering large human engineered areas of the remote environment. The experimental results demonstrate the efficiency of the proposed methods. Second, the use of cost-effective devices for guided telemanipulation is investigated. A case study involving a bimanual humanoid robot and an Inertial Measurement Unit (IMU) Motion Capture (MoCap) suit is introduced. Herein, it is corroborated how the adaptation capabilities offered by the human-in-the-loop factor can compensate for the lack of high-precision robotic systems. This work is the result of collaboration between researchers from the Harvard Biorobotics Laboratory and the Centre for Automation and Robotics UPM-CSIC. Thirdly, a new haptic rate-position controller is presented. This hybrid bilateral controller copes with the problems related to the teleoperation of a slave robot with large workspace using a small haptic device as master. Large workspaces can be covered by automatically switching between rate and position control modes. This haptic controller is ideal to couple kinematic dissimilar master-slave systems where the commands are transmitted in the task space of the remote environment. The method is validated to perform dexterous telemanipulation of objects with a robotic manipulator. Finally, two contributions for robotic manipulation are introduced. First, a new algorithm, the Iterative Kinematic Decoupling method, is presented. It is a numeric method developed to solve the Inverse Kinematics (IK) problem of a type of six-DoF robotic arms where a close-form solution is not available. The effectiveness of this IK method is compared against conventional numerical methods. Second, a robust grasp mapping has been conceived. It allows to control a wide range of different robotic hands using a gesture based correspondence between the human hand space and the robotic hand space. The human hand gesture is identified by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of grasping.

Integration of low-cost supervisory mobile robots in domestic wireless sensor networks

This paper presents a communication interface between supervisory low-cost mobile robots and domestic Wireless Sensor Network (WSN) based on the Zig Bee protocol from different manufacturers. The communication interface allows control and communication with other network devices using the same protocol. The robot can receive information from sensor devices (temperature, humidity, luminosity) and send commands to actuator devices (lights, shutters, thermostats) from different manufacturers. The architecture of the system, the interfaces and devices needed to establish the communication are described in the paper.

Hammer: an Android based application for end-user industrial robot programming

This paper presents a novel tablet based end-user interface for industrial robot programming (called Hammer). This application makes easier to program tasks for industrial robots like polishing, milling or grinding. It is based on the Scratch programming language, but specifically design and created for Android OS. It is a visual programming concept that allows non-skilled programmer operators to create programs. The application also allows to monitor the tasks while it is being executed by overlapping real time information through augmented reality. The application includes a teach pendant screen that can be customized according to the operator needs at every moment.

Transmitter-side antennas correlation in SVD-assisted MIMO systems

MIMO techniques allow increasing wireless channel performance by decreasing the BER and increasing the channel throughput and in consequence are included in current mobile communication standards. MIMO techniques are based on benefiting the existence of multipath in wireless communications and the application of appropriate signal processing techniques. The singular value decomposition (SVD) is a popular signal processing technique which, based on the perfect channel state information (PCSI) knowledge at both the transmitter and receiver sides, removes inter-antenna interferences and improves channel performance. Nevertheless, the proximity of the multiple antennas at each front-end produces the so called antennas correlation effect due to the similarity of the various physical paths. In consequence, antennas correlation drops the MIMO channel performance. This investigation focuses on the analysis of a MIMO channel under transmitter-side antennas correlation conditions. First, antennas correlation is analyzed and characterized by the correlation coefficients. The analysis describes the relation between antennas correlation and the appearance of predominant layers which significantly affect the channel performance. Then, based on the SVD, pre- and post-processing is applied to remove inter-antenna interferences. Finally, bit- and power allocation strategies are applied to reach the best performance. The resulting BER reveals that antennas correlation effect diminishes the channel performance and that not necessarily all MIMO layers must be activated to obtain the best performance.

Multisensor data fusion for accurate modelling of mobile objects

In the last decade, multi-sensor data fusion has become a broadly demanded discipline to achieve advanced solutions that can be applied in many real world situations, either civil or military. In Defence,accurate detection of all target objects is fundamental to maintaining situational awareness, to locating threats in the battlefield and to identifying and protecting strategically own forces. Civil applications, such as traffic monitoring, have similar requirements in terms of object detection and reliable identification of incidents in order to ensure safety of road users. Thanks to the appropriate data fusion technique, we can give these systems the power to exploit automatically all relevant information from multiple sources to face for instance mission needs or assess daily supervision operations. This paper focuses on its application to active vehicle monitoring in a particular area of high density traffic, and how it is redirecting the research activities being carried out in the computer vision, signal processing and machine learning fields for improving the effectiveness of detection and tracking in ground surveillance scenarios in general. Specifically, our system proposes fusion of data at a feature level which is extracted from a video camera and a laser scanner. In addition, a stochastic-based tracking which introduces some particle filters into the model to deal with uncertainty due to occlusions and improve the previous detection output is presented in this paper. It has been shown that this computer vision tracker contributes to detect objects even under poor visual information. Finally, in the same way that humans are able to analyze both temporal and spatial relations among items in the scene to associate them a meaning, once the targets objects have been correctly detected and tracked, it is desired that machines can provide a trustworthy description of what is happening in the scene under surveillance. Accomplishing so ambitious task requires a machine learning-based hierarchic architecture able to extract and analyse behaviours at different abstraction levels. A real experimental testbed has been implemented for the evaluation of the proposed modular system. Such scenario is a closed circuit where real traffic situations can be simulated. First results have shown the strength of the proposed system.

Study on heuristic usability evaluation for mobile applications

Usability guidelines are a useful tool for the developers to improve interaction with systems. It includes knowledge of different disciplines related to usability and provides solutions and best practices to achieve the objectives of usability. Heuristic evaluation is one of the methods most widely used to evaluate and user interfaces. The objective of this study is to enrich the process of heuristic evaluation with the design guidelines focusing it on the evaluation of applications for mobile devices. As well as generate a homogeneous classification of guidelines content, in order to help that from design and development process, be included solutions and good practices provided by the guidelines. In order to achieve the objectives of this work, it is provides a method for generating heuristics for mobile applications, with which four applications were evaluated, and a web tool has also been developed that allows access to the content of the guidelines using the homogeneous classification of guidelines content. The results showed the ease and utility of performing heuristic evaluations using a set of heuristics focused on mobile applications.

Explicit context matching in content-based publish/subscribe systems

Although context could be exploited to improve performance, elasticity and adaptation in most distributed systems that adopt the publish/subscribe (P/S) communication model, only a few researchers have focused on the area of context-aware matching in P/S systems and have explored its implications in domains with highly dynamic context like wireless sensor networks (WSNs) and IoT-enabled applications. Most adopted P/S models are context agnostic or do not differentiate context from the other application data. In this article, we present a novel context-aware P/S model. SilboPS manages context explicitly, focusing on the minimization of network overhead in domains with recurrent context changes related, for example, to mobile ad hoc networks (MANETs). Our approach represents a solution that helps to efficiently share and use sensor data coming from ubiquitous WSNs across a plethora of applications intent on using these data to build context awareness. Specifically, we empirically demonstrate that decoupling a subscription from the changing context in which it is produced and leveraging contextual scoping in the filtering process notably reduces (un)subscription cost per node, while improving the global performance/throughput of the network of brokers without fltering the cost of SIENA-like topology changes.

Advanced Control Strategies for a 6 DoF Hydraulic Parallel Robot Based on the Dynamic Model

Nowadays robots have made their way into real applications that were prohibitive and unthinkable thirty years ago. This is mainly due to the increase in power computations and the evolution in the theoretical field of robotics and control. Even though there is plenty of information in the current literature on this topics, it is not easy to find clear concepts of how to proceed in order to design and implement a controller for a robot. In general, the design of a controller requires of a complete understanding and knowledge of the system to be controlled. Therefore, for advanced control techniques the systems must be first identified. Once again this particular objective is cumbersome and is never straight forward requiring of great expertise and some criteria must be adopted. On the other hand, the particular problem of designing a controller is even more complex when dealing with Parallel Manipulators (PM), since their closed-loop structures give rise to a highly nonlinear system. Under this basis the current work is developed, which intends to resume and gather all the concepts and experiences involve for the control of an Hydraulic Parallel Manipulator. The main objective of this thesis is to provide a guide remarking all the steps involve in the designing of advanced control technique for PMs. The analysis of the PM under study is minced up to the core of the mechanism: the hydraulic actuators. The actuators are modeled and experimental identified. Additionally, some consideration regarding traditional PID controllers are presented and an adaptive controller is finally implemented. From a macro perspective the kinematic and dynamic model of the PM are presented. Based on the model of the system and extending the adaptive controller of the actuator, a control strategy for the PM is developed and its performance is analyzed with simulation.

Analysis and optimization for a hexapod walking robot for planetary missions

La tecnología de las máquinas móviles autónomas ha sido objeto de una gran investigación y desarrollo en las últimas décadas. En muchas actividades y entornos, los robots pueden realizar operaciones que son duras, peligrosas o simplemente imposibles para los humanos. La exploración planetaria es un buen ejemplo de un entorno donde los robots son necesarios para realizar las tareas requeridas por los científicos. La reciente exploración de Marte con robots autónomos nos ha mostrado la capacidad de las nuevas tecnologías. Desde la invención de la rueda, que esta acertadamente considerado como el mayor invento en la historia del transporte humano, casi todos los vehículos para exploración planetaria han empleado las ruedas para su desplazamiento. Las nuevas misiones planetarias demandan maquinas cada vez mas complejas. En esta Tesis se propone un nuevo diseño de un robot con patas o maquina andante que ofrecerá claras ventajas en entornos extremos. Se demostrara que puede desplazarse en los terrenos donde los robots con ruedas son ineficientes, convirtiéndolo en una elección perfecta para misiones planetarias. Se presenta una reseña histórica de los principales misiones espaciales, en particular aquellos dirigidos a la exploración planetaria. A través de este estudio será posible analizar las desventajas de los robots con ruedas utilizados en misiones anteriores. El diseño propuesto de robot con patas será presentado como una alternativa para aquellas misiones donde los robots con ruedas puedan no ser la mejor opción. En esta tesis se presenta el diseño mecánico de un robot de seis patas capaz de soportar las grandes fuerzas y momentos derivadas del movimiento de avance. Una vez concluido el diseño mecánico es necesario realizar un análisis que permita entender el movimiento y comportamiento de una maquina de esta complejidad. Las ecuaciones de movimiento del robot serán validadas por dos métodos: cinemático y dinámico. Dos códigos Matlab® han sido desarrollados para resolver dichos sistemas de ecuaciones y han sido verificados por un tercer método, un modelo de elementos finitos, que también verifica el diseño mecánico. El robot con patas presentado, ha sido diseñado para la exploración planetaria en Marte. El comportamiento del robot durante sus desplazamientos será probado mediante un código de Matlab®, desarrollado para esta tesis, que permite modificar las trayectorias, el tipo de terreno, y el número y altura de los obstáculos. Estos terrenos y requisitos iniciales no han sido elegidos de forma aleatoria, si no que están basados en mi experiencia como miembro del equipo de MSL-NASA que opera un instrumento a bordo del rover Curiosity en Marte. El robot con patas desarrollado y fabricado por el Centro de Astrobiología (INTA-CSIC), esta basado en el diseño mecánico y análisis presentados en esta tesis. ABSTRACT The autonomous machines technology has undergone a major research and development during the last decades. In many activities and environments, robots can perform operations that are tought, dangerous or simply imposible to humans. Planetary exploration is a good example of such environment where robots are needed to perform the tasks required by the scientits. Recent Mars exploration based on autonomous vehicles has shown us the capacity of the new technologies. From the invention of the wheel, which is rightly regarded as the greatest invention in the history of human transportation, nearly all-planetary vehicles are based in wheeled locomotion, but new missions demand new types of machines due to the complex tasks needed to be performed. It will be proposed in this thesis a new design of a legged robot or walking machine, which may offer clear advantages in tough environments. This Thesis will show that the proposed walking machine can travel, were terrain difficulties make wheeled vehicles ineffective, making it a perfect choice for planetary mission. A historical background of the main space missions, in particular those aimed at planetary exploration will be presented. From this study the disadvantages found in the existing wheel rovers will be analysed. The legged robot designed will be introduced as an alternative were wheeled rovers could be no longer the best option for planetary exploration. This thesis introduces the mechanical design of a six-leg robot capable of withstanding high forces and moments due to the walking motion. Once the mechanical design is concluded, and in order to analyse a machine of this complexity an understanding of its movement and behaviour is mandatory. This movement equation will be validated by two methods: kinematics and dynamics. Two Matlab® codes have been developed to solve the systems of equations and validated by a third method, a finite element model, which also verifies the mechanical design. The legged robot presented has been designed for a Mars planetary exploration. The movement behaviour of the robot will be tested in a Matlab® code developed that allows to modify the trajectories, the type of terrain, number and height of obstacles. These terrains and initial requirements have not been chosen randomly, those are based on my experience as a member of the MSL NASA team, which operates an instrument on-board of the Curiosity rover in Mars. The walking robot developed and manufactured by the Center of Astrobiology (CAB) is based in the mechanical design and analysis that will be presented in this thesis.

A Reliable Open-Source System Architecture for the Fast Designing and Prototyping of Autonomous Multi-UAV Systems: Simulation and Experimentation

During the process of design and development of an autonomous Multi-UAV System, two main problems appear. The first one is the difficulty of designing all the modules and behaviors of the aerial multi-robot system. The second one is the difficulty of having an autonomous prototype of the system for the developers that allows to test the performance of each module even in an early stage of the project. These two problems motivate this paper. A multipurpose system architecture for autonomous multi-UAV platforms is presented. This versatile system architecture can be used by the system designers as a template when developing their own systems. The proposed system architecture is general enough to be used in a wide range of applications, as demonstrated in the paper. This system architecture aims to be a reference for all designers. Additionally, to allow for the fast prototyping of autonomous multi-aerial systems, an Open Source framework based on the previously defined system architecture is introduced. It allows developers to have a flight proven multi-aerial system ready to use, so that they can test their algorithms even in an early stage of the project. The implementation of this framework, introduced in the paper with the name of “CVG Quadrotor Swarm”, which has also the advantages of being modular and compatible with different aerial platforms, can be found at https://​github.​com/​Vision4UAV/​cvg_​quadrotor_​swarm with a consistent catalog of available modules. The good performance of this framework is demonstrated in the paper by choosing a basic instance of it and carrying out simulation and experimental tests whose results are summarized and discussed in this paper.

A Vision-based Quadrotor Multi-robot Solution for the Indoor Autonomy Challenge of the 2013 International Micro Air Vehicle Competition

This paper presents a completely autonomous solution to participate in the Indoor Challenge of the 2013 International Micro Air Vehicle Competition (IMAV 2013). Our proposal is a multi-robot system with no centralized coordination whose robotic agents share their position estimates. The capability of each agent to navigate avoiding collisions is a consequence of the resulting emergent behavior. Each agent consists of a ground station running an instance of the proposed architecture that communicates over WiFi with an AR Drone 2.0 quadrotor. Visual markers are employed to sense and map obstacles and to improve the pose estimation based on Inertial Measurement Unit (IMU) and ground optical flow data. Based on our architecture, each robotic agent can navigate avoiding obstacles and other members of the multi-robot system. The solution is demonstrated and the achieved navigation performance is evaluated by means of experimental flights. This work also analyzes the capabilities of the presented solution in simulated flights of the IMAV 2013 Indoor Challenge. The performance of the CVG UPM team was awarded with the First Prize in the Indoor Autonomy Challenge of the IMAV 2013 competition.

Guidance of robot arms using depth data from RGB-D camera

Image Based Visual Servoing (IBVS) is a robotic control scheme based on vision. This scheme uses only the visual information obtained from a camera to guide a robot from any robot pose to a desired one. However, IBVS requires the estimation of different parameters that cannot be obtained directly from the image. These parameters range from the intrinsic camera parameters (which can be obtained from a previous camera calibration), to the measured distance on the optical axis between the camera and visual features, it is the depth. This paper presents a comparative study of the performance of D-IBVS estimating the depth from three different ways using a low cost RGB-D sensor like Kinect. The visual servoing system has been developed over ROS (Robot Operating System), which is a meta-operating system for robots. The experiments prove that the computation of the depth value for each visual feature improves the system performance.

Active Visual Features Based on Events to Guide Robot Manipulators in Tracking Tasks

Traditional visual servoing systems do not deal with the topic of moving objects tracking. When these systems are employed to track a moving object, depending on the object velocity, visual features can go out of the image, causing the fail of the tracking task. This occurs specially when the object and the robot are both stopped and then the object starts the movement. In this work, we have employed a retina camera based on Address Event Representation (AER) in order to use events as input in the visual servoing system. The events launched by the camera indicate a pixel movement. Event visual information is processed only at the moment it occurs, reducing the response time of visual servoing systems when they are used to track moving objects.

RGBD Human-Hand recognition for the Interaction with Robot-Hand

New low cost sensors and the new open free libraries for 3D image processing are permitting to achieve important advances for robot vision applications such as tridimensional object recognition, semantic mapping, navigation and localization of robots, human detection and/or gesture recognition for human-machine interaction. In this paper, a method to recognize the human hand and to track the fingers is proposed. This new method is based on point clouds from range images, RGBD. It does not require visual marks, camera calibration, environment knowledge and complex expensive acquisition systems. Furthermore, this method has been implemented to create a human interface in order to move a robot hand. The human hand is recognized and the movement of the fingers is analyzed. Afterwards, it is imitated from a Barret hand, using communication events programmed from ROS.

3DCOMET: 3D compression methods test dataset

The use of 3D data in mobile robotics applications provides valuable information about the robot’s environment. However usually the huge amount of 3D information is difficult to manage due to the fact that the robot storage system and computing capabilities are insufficient. Therefore, a data compression method is necessary to store and process this information while preserving as much information as possible. A few methods have been proposed to compress 3D information. Nevertheless, there does not exist a consistent public benchmark for comparing the results (compression level, distance reconstructed error, etc.) obtained with different methods. In this paper, we propose a dataset composed of a set of 3D point clouds with different structure and texture variability to evaluate the results obtained from 3D data compression methods. We also provide useful tools for comparing compression methods, using as a baseline the results obtained by existing relevant compression methods.