Design and control of multi-finger haptic devices for dexterous manipulation

En la interacción con el entorno que nos rodea durante nuestra vida diaria (utilizar un cepillo de dientes, abrir puertas, utilizar el teléfono móvil, etc.) y en situaciones profesionales (intervenciones médicas, procesos de producción, etc.), típicamente realizamos manipulaciones avanzadas que incluyen la utilización de los dedos de ambas manos. De esta forma el desarrollo de métodos de interacción háptica multi-dedo dan lugar a interfaces hombre-máquina más naturales y realistas. No obstante, la mayoría de interfaces hápticas disponibles en el mercado están basadas en interacciones con un solo punto de contacto; esto puede ser suficiente para la exploración o palpación del entorno pero no permite la realización de tareas más avanzadas como agarres. En esta tesis, se investiga el diseño mecánico, control y aplicaciones de dispositivos hápticos modulares con capacidad de reflexión de fuerzas en los dedos índice, corazón y pulgar del usuario. El diseño mecánico de la interfaz diseñada, ha sido optimizado con funciones multi-objetivo para conseguir una baja inercia, un amplio espacio de trabajo, alta manipulabilidad y reflexión de fuerzas superiores a 3 N en el espacio de trabajo. El ancho de banda y la rigidez del dispositivo se han evaluado mediante simulación y experimentación real. Una de las áreas más importantes en el diseño de estos dispositivos es el efector final, ya que es la parte que está en contacto con el usuario. Durante este trabajo se ha diseñado un dedal de bajo peso, adaptable a diferentes usuarios que, mediante la incorporación de sensores de contacto, permite estimar fuerzas normales y tangenciales durante la interacción con entornos reales y virtuales. Para el diseño de la arquitectura de control, se estudiaron los principales requisitos para estos dispositivos. Entre estos, cabe destacar la adquisición, procesado e intercambio a través de internet de numerosas señales de control e instrumentación; la computación de equaciones matemáticas incluyendo la cinemática directa e inversa, jacobiana, algoritmos de detección de agarres, etc. Todos estos componentes deben calcularse en tiempo real garantizando una frecuencia mínima de 1 KHz. Además, se describen sistemas para manipulación de precisión virtual y remota; así como el diseño de un método denominado "desacoplo cinemático iterativo" para computar la cinemática inversa de robots y la comparación con otros métodos actuales. Para entender la importancia de la interacción multimodal, se ha llevado a cabo un estudio para comprobar qué estímulos sensoriales se correlacionan con tiempos de respuesta más rápidos y de mayor precisión. Estos experimentos se desarrollaron en colaboración con neurocientíficos del instituto Technion Israel Institute of Technology. Comparando los tiempos de respuesta en la interacción unimodal (auditiva, visual y háptica) con combinaciones bimodales y trimodales de los mismos, se demuestra que el movimiento sincronizado de los dedos para generar respuestas de agarre se basa principalmente en la percepción háptica. La ventaja en el tiempo de procesamiento de los estímulos hápticos, sugiere que los entornos virtuales que incluyen esta componente sensorial generan mejores contingencias motoras y mejoran la credibilidad de los eventos. Se concluye que, los sistemas que incluyen percepción háptica dotan a los usuarios de más tiempo en las etapas cognitivas para rellenar información de forma creativa y formar una experiencia más rica. Una aplicación interesante de los dispositivos hápticos es el diseño de nuevos simuladores que permitan entrenar habilidades manuales en el sector médico. En colaboración con fisioterapeutas de Griffith University en Australia, se desarrolló un simulador que permite realizar ejercicios de rehabilitación de la mano. Las propiedades de rigidez no lineales de la articulación metacarpofalange del dedo índice se estimaron mediante la utilización del efector final diseñado. Estos parámetros, se han implementado en un escenario que simula el comportamiento de la mano humana y que permite la interacción háptica a través de esta interfaz. Las aplicaciones potenciales de este simulador están relacionadas con entrenamiento y educación de estudiantes de fisioterapia. En esta tesis, se han desarrollado nuevos métodos que permiten el control simultáneo de robots y manos robóticas en la interacción con entornos reales. El espacio de trabajo alcanzable por el dispositivo háptico, se extiende mediante el cambio de modo de control automático entre posición y velocidad. Además, estos métodos permiten reconocer el gesto del usuario durante las primeras etapas de aproximación al objeto para su agarre. Mediante experimentos de manipulación avanzada de objetos con un manipulador y diferentes manos robóticas, se muestra que el tiempo en realizar una tarea se reduce y que el sistema permite la realización de la tarea con precisión. Este trabajo, es el resultado de una colaboración con investigadores de Harvard BioRobotics Laboratory. ABSTRACT When we interact with the environment in our daily life (using a toothbrush, opening doors, using cell-phones, etc.), or in professional situations (medical interventions, manufacturing processes, etc.) we typically perform dexterous manipulations that involve multiple fingers and palm for both hands. Therefore, multi-Finger haptic methods can provide a realistic and natural human-machine interface to enhance immersion when interacting with simulated or remote environments. Most commercial devices allow haptic interaction with only one contact point, which may be sufficient for some exploration or palpation tasks but are not enough to perform advanced object manipulations such as grasping. In this thesis, I investigate the mechanical design, control and applications of a modular haptic device that can provide force feedback to the index, thumb and middle fingers of the user. The designed mechanical device is optimized with a multi-objective design function to achieve a low inertia, a large workspace, manipulability, and force-feedback of up to 3 N within the workspace; the bandwidth and rigidity for the device is assessed through simulation and real experimentation. One of the most important areas when designing haptic devices is the end-effector, since it is in contact with the user. In this thesis the design and evaluation of a thimble-like, lightweight, user-adaptable, and cost-effective device that incorporates four contact force sensors is described. This design allows estimation of the forces applied by a user during manipulation of virtual and real objects. The design of a real-time, modular control architecture for multi-finger haptic interaction is described. Requirements for control of multi-finger haptic devices are explored. Moreover, a large number of signals have to be acquired, processed, sent over the network and mathematical computations such as device direct and inverse kinematics, jacobian, grasp detection algorithms, etc. have to be calculated in Real Time to assure the required high fidelity for the haptic interaction. The Hardware control architecture has different modules and consists of an FPGA for the low-level controller and a RT controller for managing all the complex calculations (jacobian, kinematics, etc.); this provides a compact and scalable solution for the required high computation capabilities assuring a correct frequency rate for the control loop of 1 kHz. A set-up for dexterous virtual and real manipulation is described. Moreover, a new algorithm named the iterative kinematic decoupling method was implemented to solve the inverse kinematics of a robotic manipulator. In order to understand the importance of multi-modal interaction including haptics, a subject study was carried out to look for sensory stimuli that correlate with fast response time and enhanced accuracy. This experiment was carried out in collaboration with neuro-scientists from Technion Israel Institute of Technology. By comparing the grasping response times in unimodal (auditory, visual, and haptic) events with the response times in events with bimodal and trimodal combinations. It is concluded that in grasping tasks the synchronized motion of the fingers to generate the grasping response relies on haptic cues. This processing-speed advantage of haptic cues suggests that multimodalhaptic virtual environments are superior in generating motor contingencies, enhancing the plausibility of events. Applications that include haptics provide users with more time at the cognitive stages to fill in missing information creatively and form a richer experience. A major application of haptic devices is the design of new simulators to train manual skills for the medical sector. In collaboration with physical therapists from Griffith University in Australia, we developed a simulator to allow hand rehabilitation manipulations. First, the non-linear stiffness properties of the metacarpophalangeal joint of the index finger were estimated by using the designed end-effector; these parameters are implemented in a scenario that simulates the behavior of the human hand and that allows haptic interaction through the designed haptic device. The potential application of this work is related to educational and medical training purposes. In this thesis, new methods to simultaneously control the position and orientation of a robotic manipulator and the grasp of a robotic hand when interacting with large real environments are studied. The reachable workspace is extended by automatically switching between rate and position control modes. Moreover, the human hand gesture is recognized by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of the approximation-to-the-object phase and then mapped to the robotic hand actuators. These methods are validated to perform dexterous manipulation of objects with a robotic manipulator, and different robotic hands. This work is the result of a research collaboration with researchers from the Harvard BioRobotics Laboratory. The developed experiments show that the overall task time is reduced and that the developed methods allow for full dexterity and correct completion of dexterous manipulations.

Human-robot interaction for telemanipulation in large workspaces

Independientemente de la existencia de técnicas altamente sofisticadas y capacidades de cómputo cada vez más elevadas, los problemas asociados a los robots que interactúan con entornos no estructurados siguen siendo un desafío abierto en robótica. A pesar de los grandes avances de los sistemas robóticos autónomos, hay algunas situaciones en las que una persona en el bucle sigue siendo necesaria. Ejemplos de esto son, tareas en entornos de fusión nuclear, misiones espaciales, operaciones submarinas y cirugía robótica. Esta necesidad se debe a que las tecnologías actuales no pueden realizar de forma fiable y autónoma cualquier tipo de tarea. Esta tesis presenta métodos para la teleoperación de robots abarcando distintos niveles de abstracción que van desde el control supervisado, en el que un operador da instrucciones de alto nivel en la forma de acciones, hasta el control bilateral, donde los comandos toman la forma de señales de control de bajo nivel. En primer lugar, se presenta un enfoque para llevar a cabo la teleoperación supervisada de robots humanoides. El objetivo es controlar robots terrestres capaces de ejecutar tareas complejas en entornos de búsqueda y rescate utilizando enlaces de comunicación limitados. Esta propuesta incorpora comportamientos autónomos que el operador puede utilizar para realizar tareas de navegación y manipulación mientras se permite cubrir grandes áreas de entornos remotos diseñados para el acceso de personas. Los resultados experimentales demuestran la eficacia de los métodos propuestos. En segundo lugar, se investiga el uso de dispositivos rentables para telemanipulación guiada. Se presenta una aplicación que involucra un robot humanoide bimanual y un traje de captura de movimiento basado en sensores inerciales. En esta aplicación, se estudian las capacidades de adaptación introducidas por el factor humano y cómo estas pueden compensar la falta de sistemas robóticos de alta precisión. Este trabajo es el resultado de una colaboración entre investigadores del Biorobotics Laboratory de la Universidad de Harvard y el Centro de Automática y Robótica UPM-CSIC. En tercer lugar, se presenta un nuevo controlador háptico que combina velocidad y posición. Este controlador bilateral híbrido hace frente a los problemas relacionados con la teleoperación de un robot esclavo con un gran espacio de trabajo usando un dispositivo háptico pequeño como maestro. Se pueden cubrir amplias áreas de trabajo al cambiar automáticamente entre los modos de control de velocidad y posición. Este controlador háptico es ideal para sistemas maestro-esclavo con cinemáticas diferentes, donde los comandos se transmiten en el espacio de la tarea del entorno remoto. El método es validado para realizar telemanipulación hábil de objetos con un robot industrial. Por último, se introducen dos contribuciones en el campo de la manipulación robótica. Por un lado, se presenta un nuevo algoritmo de cinemática inversa, llamado método iterativo de desacoplamiento cinemático. Este método se ha desarrollado para resolver el problema cinemático inverso de un tipo de robot de seis grados de libertad donde una solución cerrada no está disponible. La eficacia del método se compara con métodos numéricos convencionales. Además, se ha diseñado una taxonomía robusta de agarres que permite controlar diferentes manos robóticas utilizando una correspondencia, basada en gestos, entre los espacios de trabajo de la mano humana y de la mano robótica. El gesto de la mano humana se identifica mediante la lectura de los movimientos relativos del índice, el pulgar y el dedo medio del usuario durante las primeras etapas del agarre. ABSTRACT Regardless of the availability of highly sophisticated techniques and ever increasing computing capabilities, the problems associated with robots interacting with unstructured environments remains an open challenge. Despite great advances in autonomous robotics, there are some situations where a humanin- the-loop is still required, such as, nuclear, space, subsea and robotic surgery operations. This is because the current technologies cannot reliably perform all kinds of task autonomously. This thesis presents methods for robot teleoperation strategies at different levels of abstraction ranging from supervisory control, where the operator gives high-level task actions, to bilateral teleoperation, where the commands take the form of low-level control inputs. These strategies contribute to improve the current human-robot interfaces specially in the case of slave robots deployed at large workspaces. First, an approach to perform supervisory teleoperation of humanoid robots is presented. The goal is to control ground robots capable of executing complex tasks in disaster relief environments under constrained communication links. This proposal incorporates autonomous behaviors that the operator can use to perform navigation and manipulation tasks which allow covering large human engineered areas of the remote environment. The experimental results demonstrate the efficiency of the proposed methods. Second, the use of cost-effective devices for guided telemanipulation is investigated. A case study involving a bimanual humanoid robot and an Inertial Measurement Unit (IMU) Motion Capture (MoCap) suit is introduced. Herein, it is corroborated how the adaptation capabilities offered by the human-in-the-loop factor can compensate for the lack of high-precision robotic systems. This work is the result of collaboration between researchers from the Harvard Biorobotics Laboratory and the Centre for Automation and Robotics UPM-CSIC. Thirdly, a new haptic rate-position controller is presented. This hybrid bilateral controller copes with the problems related to the teleoperation of a slave robot with large workspace using a small haptic device as master. Large workspaces can be covered by automatically switching between rate and position control modes. This haptic controller is ideal to couple kinematic dissimilar master-slave systems where the commands are transmitted in the task space of the remote environment. The method is validated to perform dexterous telemanipulation of objects with a robotic manipulator. Finally, two contributions for robotic manipulation are introduced. First, a new algorithm, the Iterative Kinematic Decoupling method, is presented. It is a numeric method developed to solve the Inverse Kinematics (IK) problem of a type of six-DoF robotic arms where a close-form solution is not available. The effectiveness of this IK method is compared against conventional numerical methods. Second, a robust grasp mapping has been conceived. It allows to control a wide range of different robotic hands using a gesture based correspondence between the human hand space and the robotic hand space. The human hand gesture is identified by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of grasping.

Robotic multiwell planar patch-clamp for native and primary mammalian cells

Robotic multiwell planar patch-clamp has become common in drug development and safety programs because it enables efficient and systematic testing of compounds against ion channels during voltage-clamp. It has not, however, been adopted significantly in other important areas of ion channel research, where conventional patch-clamp remains the favored method. Here, we show the wider potential of the multiwell approach with the ability for efficient intracellular solution exchange, describing protocols and success rates for recording from a range of native and primary mammalian cells derived from blood vessels, arthritic joints and the immune and central nervous systems. The protocol involves preparing a suspension of single cells to be dispensed robotically into 4-8 microfluidic chambers each containing a glass chip with a small aperture. Under automated control, giga-seals and whole-cell access are achieved followed by preprogrammed routines of voltage paradigms and fast extracellular or intracellular solution exchange. Recording from 48 chambers usually takes 1-6 h depending on the experimental design and yields 16-33 cell recordings.

On homoclinic chaos and a control chaos strategy applied to a free-joint nonholonomic manipulator

In this work, the occurrence of chaos (homoclinic scene) is verified in a robotic system with two degrees of freedom by using Poincare-Mel'nikov method. The studied problem was based on experimental results of a two-joint planar manipulator-first joint actuated and the second joint free-that resides in a horizontal plane. This is the simplest model of nonholonomic free-joint manipulators. The purpose of the present study is to verify analytically those results and to suggest a control strategy.

Nonlinear state estimation and control applied to a manipulator robotic including drive motor

A robotic control design considering all the inherent nonlinearities of the robot engine configuration is developed. The interactions between the robot and joint motor drive mechanism are considered. The proposed control combines two strategies, one feedforward control in order to maintain the system in the desired coordinate, and feedback control system to take the system into a desired coordinate. The feedback control is obtained using State Dependent Riccati Equation (SDRE). For link positioning two cases are considered. Case 1: For control positioning, it is only used motor voltage; Case 2: For control positioning, it is used both motor voltage and torque between the links. Simulation results, including parametric uncertainties in control shows the feasibility of the proposed control for the considered system.

A geometric approach for determining inner and exterior boundaries of workspaces of planar manipulators

In this paper a method to determine the internal and external boundaries of planar workspaces, represented with an ordered set of points, is presented. The sequence of points are grouped and can be interpreted to form a sequence of curves. Three successive curves are used for determining the instantaneous center of rotation for the second one of them. The two extremal points on the curve with respect to the instantaneous center are recognized as singular points. The chronological ordering of these singular points is used to generate the two envelope curves, which are potentially intersecting. Methods have been presented in the paper for the determination of the workspace boundary from the envelope curves. Strategies to deal with the manipulators with joint limits and various degenerate situations have also been discussed. The computational steps being completely geometric, the method does not require the knowledge about the manipulator's kinematics. Hence, it can be used for the workspace of arbitrary planar manipulators. A number of illustrative examples demonstrate the efficacy of the proposed method.

Synthesis of workspaces of planar manipulators with arbitrary topology using shape representation and simulated annealing

This paper presents a general methodology for the synthesis of the external boundary of the workspaces of a planar manipulator with arbitrary topology. Both the desired workspace and the manipulator workspaces are identified by their boundaries and are treated as simple closed polygons. The paper introduces the concept of best match configuration and shows that the corresponding transformation can be obtained by using the concept of shape normalization available in image processing literature. Introduction of the concept of shape in workspace synthesis allows highly accurate synthesis with fewer numbers of design variables. This paper uses a new global property based vector representation for the shape of the workspaces which is computationally efficient because six out of the seven elements of this vector are obtained as a by-product of the shape normalization procedure. The synthesis of workspaces is formulated as an optimization problem where the distance between the shape vector of the desired workspace and that of the workspace of the manipulator at hand are minimized by changing the dimensional parameters of the manipulator. In view of the irregular nature of the error manifold, the statistical optimization procedure of simulated annealing has been used. A number of worked-out examples illustrate the generality and efficiency of the present method. (C) 1998 Elsevier Science Ltd. All rights reserved.

Whole arm manipulation using closed and hybrid planar chains: A kinematic study

This paper studies planar whole arm manipulation of a circular object using closed loop and hybrid manipulators. The manipulation is simple with fewer degrees of actuation than the task space. This is an useful operation if the initial and final positions of the object are on the same surface. Closed loop manipulator is a 4/5 bar mechanism. In hybrid manipulators a open loop manipulator with 3/4 links is attached to the floating link of 4/5 bar mechanism. The mobility analysis is carried out to find the connectivity of the object with reference to frame. The manipulation (forward kinematics) starts from a given configuration of the object and the manipulator. In hybrid manipulators determination of initial configuration involves inverse kinematics of open loop manipulator. The input joint velocities are used to demonstrate the manipulation. Conditions are specified for prehensile manipulation.

The Effect of Transmission Design on Force-Controlled Manipulator Performance

Previous research in force control has focused on the choice of appropriate servo implementation without corresponding regard to the choice of mechanical hardware. This report analyzes the effect of mechanical properties such as contact compliance, actuator-to-joint compliance, torque ripple, and highly nonlinear dry friction in the transmission mechanisms of a manipulator. A set of requisites for high performance then guides the development of mechanical-design and servo strategies for improved performance. A single-degree-of-freedom transmission testbed was constructed that confirms the predicted effect of Coulomb friction on robustness; design and construction of a cable-driven, four-degree-of- freedom, "whole-arm" manipulator illustrates the recommended design strategies.

An intelligent configuration-sampling based local motion planner for robotic manipulators

Manipulator motion planning is a task which relies heavily on the construction of a configuration space prior to path planning. However when fast real-time motion is needed, the full construction of the manipulator's high-dimensional configu-ration space can be too slow and expensive. Alternative planning methods, which avoid this full construction of the manipulator's configuration space are needed to solve this problem. Here, one such existing local planning method for manipulators based on configuration-sampling and subgoal-selection has been extended. Using a modified Artificial Potential Fields (APF) function, goal-configuration sampling and a novel subgoal selection method, it provides faster, more optimal paths than the previously proposed work. Simulation results show a decrease in both runtime and path lengths, along with a decrease in unexpected local minimum and crashing issues.

Manipulator trajectory planning using a MOEA

Generating manipulator trajectories considering multiple objectives and obstacle avoidance is a non-trivial optimization problem. In this paper a multi-objective genetic algorithm based technique is proposed to address this problem. Multiple criteria are optimized considering up to five simultaneous objectives. Simulation results are presented for robots with two and three degrees of freedom, considering two and five objectives optimization. A subsequent analysis of the spread and solutions distribution along the converged non-dominated Pareto front is carried out, in terms of the achieved diversity.

Design and Development of a Remotely Operated Underwater Multi-Robot Manipulator Controller

For the scientific and commercial utilization of Ocean resources, the role of intelligent underwater robotic systems are of great importance. Scientific activities like Marine Bio-technology, Hydrographic mapping, and commercial applications like Marine mining, Ocean energy, fishing, aquaculture, cable laying and pipe lining are a few utilization of ocean resources. As most of the deep undersea exploration are beyond the reachability of divers and also as the use of operator controlled and teleoperated Remotely Operated Vehicles (ROVs) and Diver Transport Vehicles (DTVs) turn out to be highly inefficient, it is essential to have a fully automated system capable providing stable control and communication links for the unstructured undersea environment.

The PHD: A Planar, Harmonic Drive Robot for Joint Torque Control

This thesis details the development of a model of a seven degree of freedom manipulator for position control. Then, it goes on to discuss the design and construction of a the PHD, a robot built to serve two purposes: first, to perform research on joint torque control schemes, and second, to determine the important dynamic characteristics of the Harmonic Drive. The PHD, is a planar, three degree of freedom arm with torque sensors integral to each joint. Preliminary testing has shown that a simple linear spring model of the Harmonic Drive's flexibility is suitable in many situations.

Design and Control of an Anthropomorphic Robotic Finger with Multi-point Tactile Sensation

The goal of this research is to develop the prototype of a tactile sensing platform for anthropomorphic manipulation research. We investigate this problem through the fabrication and simple control of a planar 2-DOF robotic finger inspired by anatomic consistency, self-containment, and adaptability. The robot is equipped with a tactile sensor array based on optical transducer technology whereby localized changes in light intensity within an illuminated foam substrate correspond to the distribution and magnitude of forces applied to the sensor surface plane. The integration of tactile perception is a key component in realizing robotic systems which organically interact with the world. Such natural behavior is characterized by compliant performance that can initiate internal, and respond to external, force application in a dynamic environment. However, most of the current manipulators that support some form of haptic feedback either solely derive proprioceptive sensation or only limit tactile sensors to the mechanical fingertips. These constraints are due to the technological challenges involved in high resolution, multi-point tactile perception. In this work, however, we take the opposite approach, emphasizing the role of full-finger tactile feedback in the refinement of manual capabilities. To this end, we propose and implement a control framework for sensorimotor coordination analogous to infant-level grasping and fixturing reflexes. This thesis details the mechanisms used to achieve these sensory, actuation, and control objectives, along with the design philosophies and biological influences behind them. The results of behavioral experiments with a simple tactilely-modulated control scheme are also described. The hope is to integrate the modular finger into an %engineered analog of the human hand with a complete haptic system.