Virtual force feedback teleoperation of the InsectBot using optical flow

This paper proposes the use of optical flow from a moving robot to provide force feedback to an operator’s joystick to facilitate collision free teleoperation. Optical flow is measured by a pair of wide angle cameras on board the vehicle and used to generate a virtual environmental force that is reflected to the user through the joystick, as well as feeding back into the control of the vehicle. We show that the proposed control is dissipative and prevents the vehicle colliding with the environment as well as providing the operator with a natural feel for the remote environment. Experimental results are provided on the InsectBot holonomic vehicle platform.

A new framework for force feedback teleoperation of robotic vehicles based on optical flow

This paper proposes the use of optical flow from a moving robot to provide force feedback to an operator's joystick to facilitate collision free teleoperation. Optic flow is measured by wide angle cameras on board the vehicle and used to generate a virtual environmental force that is reflected to the user through the joystick, as well as feeding back into the control of the vehicle. The coupling between optical flow (velocity) and force is modelled as an impedance - in this case an optical impedance. We show that the proposed control is dissipative and prevents the vehicle colliding with the environment as well as providing the operator with a natural feel for the remote environment. The paper focuses on applications to aerial robotics vehicles, however, the ideas apply directly to other force actuated vehicles such as submersibles or space vehicles, and the authors believe the approach has potential for control of terrestrial vehicles and even teleoperation of manipulators. Experimental results are provided for a simulated aerial robot in a virtual environment controlled by a haptic joystick.

Intercontinental haptic teleoperation of a flying vehicle : a step towards real-time applications

This paper describes the theory and practice for a stable haptic teleoperation of a flying vehicle. It extends passivity-based control framework for haptic teleoperation of aerial vehicles in the longest intercontinental setting that presents great challenges. The practicality of the control architecture has been shown in maneuvering and obstacle-avoidance tasks over the internet with the presence of significant time-varying delays and packet losses. Experimental results are presented for teleoperation of a slave quadrotor in Australia from a master station in the Netherlands. The results show that the remote operator is able to safely maneuver the flying vehicle through a structure using haptic feedback of the state of the slave and the perceived obstacles.

Teleoperation of a 7 DOF humanoid robot arm using human arm accelerations and EMG signals

We present our work on tele-operating a complex humanoid robot with the help of bio-signals collected from the operator. The frameworks (for robot vision, collision avoidance and machine learning), developed in our lab, allow for a safe interaction with the environment, when combined. This even works with noisy control signals, such as, the operator’s hand acceleration and their electromyography (EMG) signals. These bio-signals are used to execute equivalent actions (such as, reaching and grasping of objects) on the 7 DOF arm.

Vision-Based Assisted Teleoperation for Inspection Tasks with a Small ROV

It is well-known that ROVs require human intervention to guarantee the success of their assignment, as well as the equipment safety. However, as its teleoperation is quite complex to perform, there is a need for assisted teleoperation. This study aims to take on this challenge by developing vision-based assisted teleoperation maneuvers, since a standard camera is present in any ROV. The proposed approach is a visual servoing solution, that allows the user to select between several standard image processing methods and is applied to a 3-DOF ROV. The most interesting characteristic of the presented system is the exclusive use of the camera data to improve the teleoperation of an underactuated ROV. It is demonstrated through the comparison and evaluation of standard implementations of different vision methods and the execution of simple maneuvers to acquire experimental results, that the teleoperation of a small ROV can be drastically improved without the need to install additional sensors.

A novel scheme for human-friendly and time-delays robust neuropredictive teleoperation

A novel Neuropredictive Teleoperation (NPT) Scheme is presented. The design results from two key ideas: the exploitation of the measured or estimated neural input to the human arm or its electromyograph (EMG) as the system input and the employment of a predictor of the arm movement, based on this neural signal and an arm model, to compensate for time delays in the system. Although a multitude of such models, as well as measuring devices for the neural signals and the EMG, have been proposed, current telemanipulator research has only been considering highly simplified arm models. In the present design, the bilateral constraint that the master and slave are simultaneously compliant to each other's state (equal positions and forces) is abandoned, thus obtaining a simple to analyzesuccession of only locally controlled modules, and a robustness to time delays of up to 500 ms. The proposed designs were inspired by well established physiological evidence that the brain, rather than controlling the movement on-line, programs the arm with an action plan of a complete movement, which is then executed largely in open loop, regulated only by local reflex loops. As a model of the human arm the well-established Stark model is employed, whose mathematical representation is modified to make it suitable for an engineering application. The proposed scheme is however valid for any arm model. BIBO-stability and passivity results for a variety of local control laws are reported. Simulation results and comparisons with traditional designs also highlight the advantages of the proposed design.

Human-computer cooperative teleoperation with time delay

Presents a method for model based bilateral control of master-slave arm with time delay between master and slave arms, where the system supports cooperative action between manual and automatic modes. The method realises efficiencies in master-slave arm control with the simplicities of a computer and the flexibility of a skilled human operator.

Assisted remote viewing of a teleoperation work cell

Intelligent viewing systems are required if efficient and productive teleoperation is to be applied to dynamic manufacturing environments. These systems must automatically provide remote views to an operator which assist in the completion of the task. This assistance increases the productivity of the teleoperation task if the robot controller is responsive to the unpredictable dynamic evolution of the workcell. Behavioral controllers can be utilized to give reactive 'intelligence.' The inherent complex structure of current systems, however, places considerable time overheads on any redesign of the emergent behavior. In industry, where the remote environment and task frequently change, this continual redesign process becomes inefficient. We introduce a novel behavioral controller, based on an 'ego-behavior' architecture, to command an active camera (a camera mounted on a robot) within a remote workcell. Using this ego-behavioral architecture the responses from individual behaviors are rapidly combined to produce an 'intelligent' responsive viewing system. The architecture is single-layered, each behavior being autonomous with no explicit knowledge of the number, description or activity of other behaviors present (if any). This lack of imposed structure decreases the development time as it allows each behavior to be designed and tested independently before insertion into the architecture. The fusion mechanism for the behaviors provides the ability for each behavior to compete and/or co-operate with other behaviors for full or partial control of the viewing active camera. Each behavior continually reassesses this degree of competition or co-operation by measuring its own success in controlling the active camera against pre-defined constraints. The ego-behavioral architecture is demonstrated through simulation and experimentation.

Gesture-based teleoperation using a holonomic robot

This work presents the development and integration of an user interface (UI) framework based on various current input devices that take advantage of our ergonomics. The purpose is to teleoperate a holonomic robot using upper member gestures and postures for studying the suitable of such interfaces when programming and interacting with a mobile robot. As performance vary from UI to UI the framework is focused to be used as a complementary industrial or didactic tool thus, changing how inexperience users tackle their first impressions when working with mobile robots while performing simple gesture-based teleoperation tasks. © 2012 ICROS.

Cooperative teleoperation systems

This thesis deals with the studies on the Cooperative Teleoperation Systems. The literature on cooperative teleoperation did not take into account control architectures composed of pairs of wave-based bilateral teleoperators operating in a shared environment. In this work The author two cooperative control schemes based on wave variables by considering two pairs of single-master/single-slave devices collaborating to carry out operations in a shared remote environment are proposed. Such architectures have been validated both with simulations and experimental tests. Ch. 2 introduces a description of the two control architectures proposed and presents some simulation results where the cooperative teleoperation systems evolve in free space and in contact with a stiff wall. In the Ch. 3 some experimental results which confirm the positive results of the control schemes are illustred. Such results have been achieved by using a prototype custom built at Laboratory of Automaiton and Robotics of University of Bologna, which is also illustrated in this chapter. In Ch. 4 the problem of defining proper tools and procedures for an analysis, and possibly a comparison, of the performances of cooperative teleoperation systems is addressed. In particular, a novel generalization of criteria adopted for classical (i.e. one master-one slave) teleoperators is presented and illustrated on the basis of the force-position and the position-position cooperative control schemes proposed in Ch. 2, both from a transparency and stability point of view, and by assuming a null time delay in the communication channel.

Haptic teleoperation of mobile robots for augmentation of operator perception in environments with low-wireless signal

Wireless teleoperation of field robots for maintenance, inspection and rescue missions is often performed in environments with low wireless connectivity, caused by signal losses from the environment and distance from the wireless transmitters. Various studies from the literature have addressed these problems with time-delay robust control systems and multi-hop wireless relay networks. However, such approaches do not solve the issue of how to present wireless data to the operator to avoid losing control of the robot. Despite the fact that teleoperation for maintenance often already involves haptic devices, no studies look at the possibility of using this existing feedback to aid operators in navigating within areas of variable wireless connectivity. We propose a method to incorporate haptic information into the velocity control of an omnidirectional robot to augment the operators perception of wireless signal strength in the remote environment. In this paper we introduce a mapping between wireless signal strength from multiple receivers to the force feedback of a 6 Degree of Freedom haptic master and evaluate the proposed approach using experimental data and randomly generated wireless maps

Time domain passivity control for delayed teleoperation

La telepesencia combina diferentes modalidades sensoriales, incluyendo, entre otras, la visual y la del tacto, para producir una sensación de presencia remota en el operador. Un elemento clave en la implementación de sistemas de telepresencia para permitir una telemanipulación del entorno remoto es el retorno de fuerza. Durante una telemanipulación, la energía mecánica es transferida entre el operador humano y el entorno remoto. En general, la energía es una propiedad de los objetos físicos, fundamental en su mutual interacción. En esta interacción, la energía se puede transmitir entre los objetos, puede cambiar de forma pero no puede crearse ni destruirse. En esta tesis, se aplica este principio fundamental para derivar un nuevo método de control bilateral que permite el diseño de sistemas de teleoperación estables para cualquier arquitectura concebible. El razonamiento parte del hecho de que la energía mecánica insertada por el operador humano en el sistema debe transferirse hacia el entorno remoto y viceversa. Tal como se verá, el uso de la energía como variable de control permite un tratamiento más general del sistema que el control convencional basado en variables específicas del sistema. Mediante el concepto de Red de Potencia de Retardo Temporal (RPRT), el problema de definir los flujos de energía en un sistema de teleoperación es solucionado con independencia de la arquitectura de comunicación. Como se verá, los retardos temporales son la principal causa de generación de energía virtual. Este hecho se observa con retardos a partir de 1 milisegundo. Esta energía virtual es añadida al sistema de forma intrínseca y representa la causa principal de inestabilidad. Se demuestra que las RPRTs son transportadoras de la energía deseada intercambiada entre maestro y esclavo pero a la vez generadoras de energía virtual debido al retardo temporal. Una vez estas redes son identificadas, el método de Control de Pasividad en el Dominio Temporal para RPRTs se propone como mecanismo de control para asegurar la pasividad del sistema, y as__ la estabilidad. El método se basa en el simple hecho de que esta energía virtual debido al retardo debe transformarse en disipación. As__ el sistema se aproxima al sistema deseado, donde solo la energía insertada desde un extremo es transferida hacia el otro. El sistema resultante presenta dos cualidades: por un lado la estabilidad del sistema queda garantizada con independencia de la arquitectura del sistema y del canal de comunicación; por el otro, el rendimiento es maximizado en términos de fidelidad de transmisión energética. Los métodos propuestos se sustentan con sistemas experimentales con diferentes arquitecturas de control y retardos entre 2 y 900 ms. La tesis concluye con un experimento que incluye una comunicación espacial basada en el satélite geoestacionario ASTRA. ABSTRACT Telepresence combines different sensorial modalities, including vision and touch, to produce a feeling of being present in a remote location. The key element to successfully implement a telepresence system and thus to allow telemanipulation of a remote environment is force feedback. In a telemanipulation, mechanical energy must convey from the human operator to the manipulated object found in the remote environment. In general, energy is a property of all physical objects, fundamental to their mutual interactions in which the energy can be transferred among the objects and can change form but cannot be created or destroyed. In this thesis, we exploit this fundamental principle to derive a novel bilateral control mechanism that allows designing stable teleoperation systems with any conceivable communication architecture. The rationale starts from the fact that the mechanical energy injected by a human operator into the system must be conveyed to the remote environment and Vice Versa. As will be seen, setting energy as the control variable allows a more general treatment of the controlled system in contrast to the more conventional control of specific systems variables. Through the Time Delay Power Network (TDPN) concept, the issue of defining the energy flows involved in a teleoperation system is solved with independence of the communication architecture. In particular, communication time delays are found to be a source of virtual energy. This fact is observed with delays starting from 1 millisecond. Since this energy is added, the resulting teleoperation system can be non-passive and thus become unstable. The Time Delay Power Networks are found to be carriers of the desired exchanged energy but also generators of virtual energy due to the time delay. Once these networks are identified, the Time Domain Passivity Control approach for TDPNs is proposed as a control mechanism to ensure system passivity and therefore, system stability. The proposed method is based on the simple fact that this intrinsically added energy due to the communication must be transformed into dissipation. Then the system becomes closer to the ambitioned one, where only the energy injected from one end of the system is conveyed to the other one. The resulting system presents two benefits: On one hand, system stability is guaranteed through passivity independently from the chosen control architecture and communication channel; on the other, performance is maximized in terms of energy transfer faithfulness. The proposed methods are sustained with a set of experimental implementations using different control architectures and communication delays ranging from 2 to 900 milliseconds. An experiment that includes a communication Space link based on the geostationary satellite ASTRA concludes this thesis.

State Convergence Theory Applied to a Delayed Teleoperation System

State convergence is a control strategy that was proposed in the early 2000s to ensure stability and transparency in a teleoperation system under specific control gains values. This control strategy has been implemented for a linear system with or without time delay. This paper represents the first attempt at demonstrating, theoretically and experimentantally, that this control strategy can also be applied to a nonlinear teleoperation system with n degrees of freedom and delay in the communication channel. It is assumed that the human operator applies a constant force on the local manipulator during the teleoperation. In addition, the interaction between the remote manipulator and the environment is considered passive. Communication between the local and remote sites is made by means of a communication channel with variable time delay. In this article the theory of Lyapunov-Krasovskii was used to demonstrate that the local-remote teleoperation system is asymptotically stable.