Design and control of multi-finger haptic devices for dexterous manipulation

En la interacción con el entorno que nos rodea durante nuestra vida diaria (utilizar un cepillo de dientes, abrir puertas, utilizar el teléfono móvil, etc.) y en situaciones profesionales (intervenciones médicas, procesos de producción, etc.), típicamente realizamos manipulaciones avanzadas que incluyen la utilización de los dedos de ambas manos. De esta forma el desarrollo de métodos de interacción háptica multi-dedo dan lugar a interfaces hombre-máquina más naturales y realistas. No obstante, la mayoría de interfaces hápticas disponibles en el mercado están basadas en interacciones con un solo punto de contacto; esto puede ser suficiente para la exploración o palpación del entorno pero no permite la realización de tareas más avanzadas como agarres. En esta tesis, se investiga el diseño mecánico, control y aplicaciones de dispositivos hápticos modulares con capacidad de reflexión de fuerzas en los dedos índice, corazón y pulgar del usuario. El diseño mecánico de la interfaz diseñada, ha sido optimizado con funciones multi-objetivo para conseguir una baja inercia, un amplio espacio de trabajo, alta manipulabilidad y reflexión de fuerzas superiores a 3 N en el espacio de trabajo. El ancho de banda y la rigidez del dispositivo se han evaluado mediante simulación y experimentación real. Una de las áreas más importantes en el diseño de estos dispositivos es el efector final, ya que es la parte que está en contacto con el usuario. Durante este trabajo se ha diseñado un dedal de bajo peso, adaptable a diferentes usuarios que, mediante la incorporación de sensores de contacto, permite estimar fuerzas normales y tangenciales durante la interacción con entornos reales y virtuales. Para el diseño de la arquitectura de control, se estudiaron los principales requisitos para estos dispositivos. Entre estos, cabe destacar la adquisición, procesado e intercambio a través de internet de numerosas señales de control e instrumentación; la computación de equaciones matemáticas incluyendo la cinemática directa e inversa, jacobiana, algoritmos de detección de agarres, etc. Todos estos componentes deben calcularse en tiempo real garantizando una frecuencia mínima de 1 KHz. Además, se describen sistemas para manipulación de precisión virtual y remota; así como el diseño de un método denominado "desacoplo cinemático iterativo" para computar la cinemática inversa de robots y la comparación con otros métodos actuales. Para entender la importancia de la interacción multimodal, se ha llevado a cabo un estudio para comprobar qué estímulos sensoriales se correlacionan con tiempos de respuesta más rápidos y de mayor precisión. Estos experimentos se desarrollaron en colaboración con neurocientíficos del instituto Technion Israel Institute of Technology. Comparando los tiempos de respuesta en la interacción unimodal (auditiva, visual y háptica) con combinaciones bimodales y trimodales de los mismos, se demuestra que el movimiento sincronizado de los dedos para generar respuestas de agarre se basa principalmente en la percepción háptica. La ventaja en el tiempo de procesamiento de los estímulos hápticos, sugiere que los entornos virtuales que incluyen esta componente sensorial generan mejores contingencias motoras y mejoran la credibilidad de los eventos. Se concluye que, los sistemas que incluyen percepción háptica dotan a los usuarios de más tiempo en las etapas cognitivas para rellenar información de forma creativa y formar una experiencia más rica. Una aplicación interesante de los dispositivos hápticos es el diseño de nuevos simuladores que permitan entrenar habilidades manuales en el sector médico. En colaboración con fisioterapeutas de Griffith University en Australia, se desarrolló un simulador que permite realizar ejercicios de rehabilitación de la mano. Las propiedades de rigidez no lineales de la articulación metacarpofalange del dedo índice se estimaron mediante la utilización del efector final diseñado. Estos parámetros, se han implementado en un escenario que simula el comportamiento de la mano humana y que permite la interacción háptica a través de esta interfaz. Las aplicaciones potenciales de este simulador están relacionadas con entrenamiento y educación de estudiantes de fisioterapia. En esta tesis, se han desarrollado nuevos métodos que permiten el control simultáneo de robots y manos robóticas en la interacción con entornos reales. El espacio de trabajo alcanzable por el dispositivo háptico, se extiende mediante el cambio de modo de control automático entre posición y velocidad. Además, estos métodos permiten reconocer el gesto del usuario durante las primeras etapas de aproximación al objeto para su agarre. Mediante experimentos de manipulación avanzada de objetos con un manipulador y diferentes manos robóticas, se muestra que el tiempo en realizar una tarea se reduce y que el sistema permite la realización de la tarea con precisión. Este trabajo, es el resultado de una colaboración con investigadores de Harvard BioRobotics Laboratory. ABSTRACT When we interact with the environment in our daily life (using a toothbrush, opening doors, using cell-phones, etc.), or in professional situations (medical interventions, manufacturing processes, etc.) we typically perform dexterous manipulations that involve multiple fingers and palm for both hands. Therefore, multi-Finger haptic methods can provide a realistic and natural human-machine interface to enhance immersion when interacting with simulated or remote environments. Most commercial devices allow haptic interaction with only one contact point, which may be sufficient for some exploration or palpation tasks but are not enough to perform advanced object manipulations such as grasping. In this thesis, I investigate the mechanical design, control and applications of a modular haptic device that can provide force feedback to the index, thumb and middle fingers of the user. The designed mechanical device is optimized with a multi-objective design function to achieve a low inertia, a large workspace, manipulability, and force-feedback of up to 3 N within the workspace; the bandwidth and rigidity for the device is assessed through simulation and real experimentation. One of the most important areas when designing haptic devices is the end-effector, since it is in contact with the user. In this thesis the design and evaluation of a thimble-like, lightweight, user-adaptable, and cost-effective device that incorporates four contact force sensors is described. This design allows estimation of the forces applied by a user during manipulation of virtual and real objects. The design of a real-time, modular control architecture for multi-finger haptic interaction is described. Requirements for control of multi-finger haptic devices are explored. Moreover, a large number of signals have to be acquired, processed, sent over the network and mathematical computations such as device direct and inverse kinematics, jacobian, grasp detection algorithms, etc. have to be calculated in Real Time to assure the required high fidelity for the haptic interaction. The Hardware control architecture has different modules and consists of an FPGA for the low-level controller and a RT controller for managing all the complex calculations (jacobian, kinematics, etc.); this provides a compact and scalable solution for the required high computation capabilities assuring a correct frequency rate for the control loop of 1 kHz. A set-up for dexterous virtual and real manipulation is described. Moreover, a new algorithm named the iterative kinematic decoupling method was implemented to solve the inverse kinematics of a robotic manipulator. In order to understand the importance of multi-modal interaction including haptics, a subject study was carried out to look for sensory stimuli that correlate with fast response time and enhanced accuracy. This experiment was carried out in collaboration with neuro-scientists from Technion Israel Institute of Technology. By comparing the grasping response times in unimodal (auditory, visual, and haptic) events with the response times in events with bimodal and trimodal combinations. It is concluded that in grasping tasks the synchronized motion of the fingers to generate the grasping response relies on haptic cues. This processing-speed advantage of haptic cues suggests that multimodalhaptic virtual environments are superior in generating motor contingencies, enhancing the plausibility of events. Applications that include haptics provide users with more time at the cognitive stages to fill in missing information creatively and form a richer experience. A major application of haptic devices is the design of new simulators to train manual skills for the medical sector. In collaboration with physical therapists from Griffith University in Australia, we developed a simulator to allow hand rehabilitation manipulations. First, the non-linear stiffness properties of the metacarpophalangeal joint of the index finger were estimated by using the designed end-effector; these parameters are implemented in a scenario that simulates the behavior of the human hand and that allows haptic interaction through the designed haptic device. The potential application of this work is related to educational and medical training purposes. In this thesis, new methods to simultaneously control the position and orientation of a robotic manipulator and the grasp of a robotic hand when interacting with large real environments are studied. The reachable workspace is extended by automatically switching between rate and position control modes. Moreover, the human hand gesture is recognized by reading the relative movements of the index, thumb and middle fingers of the user during the early stages of the approximation-to-the-object phase and then mapped to the robotic hand actuators. These methods are validated to perform dexterous manipulation of objects with a robotic manipulator, and different robotic hands. This work is the result of a research collaboration with researchers from the Harvard BioRobotics Laboratory. The developed experiments show that the overall task time is reduced and that the developed methods allow for full dexterity and correct completion of dexterous manipulations.

Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors’ feedback

Tactile sensors play an important role in robotics manipulation to perform dexterous and complex tasks. This paper presents a novel control framework to perform dexterous manipulation with multi-fingered robotic hands using feedback data from tactile and visual sensors. This control framework permits the definition of new visual controllers which allow the path tracking of the object motion taking into account both the dynamics model of the robot hand and the grasping force of the fingertips under a hybrid control scheme. In addition, the proposed general method employs optimal control to obtain the desired behaviour in the joint space of the fingers based on an indicated cost function which determines how the control effort is distributed over the joints of the robotic hand. Finally, authors show experimental verifications on a real robotic manipulation system for some of the controllers derived from the control framework.

Integrated visual perception architecture for robotic clothes perception and manipulation

This thesis proposes a generic visual perception architecture for robotic clothes perception and manipulation. This proposed architecture is fully integrated with a stereo vision system and a dual-arm robot and is able to perform a number of autonomous laundering tasks. Clothes perception and manipulation is a novel research topic in robotics and has experienced rapid development in recent years. Compared to the task of perceiving and manipulating rigid objects, clothes perception and manipulation poses a greater challenge. This can be attributed to two reasons: firstly, deformable clothing requires precise (high-acuity) visual perception and dexterous manipulation; secondly, as clothing approximates a non-rigid 2-manifold in 3-space, that can adopt a quasi-infinite configuration space, the potential variability in the appearance of clothing items makes them difficult to understand, identify uniquely, and interact with by machine. From an applications perspective, and as part of EU CloPeMa project, the integrated visual perception architecture refines a pre-existing clothing manipulation pipeline by completing pre-wash clothes (category) sorting (using single-shot or interactive perception for garment categorisation and manipulation) and post-wash dual-arm flattening. To the best of the author’s knowledge, as investigated in this thesis, the autonomous clothing perception and manipulation solutions presented here were first proposed and reported by the author. All of the reported robot demonstrations in this work follow a perception-manipulation method- ology where visual and tactile feedback (in the form of surface wrinkledness captured by the high accuracy depth sensor i.e. CloPeMa stereo head or the predictive confidence modelled by Gaussian Processing) serve as the halting criteria in the flattening and sorting tasks, respectively. From scientific perspective, the proposed visual perception architecture addresses the above challenges by parsing and grouping 3D clothing configurations hierarchically from low-level curvatures, through mid-level surface shape representations (providing topological descriptions and 3D texture representations), to high-level semantic structures and statistical descriptions. A range of visual features such as Shape Index, Surface Topologies Analysis and Local Binary Patterns have been adapted within this work to parse clothing surfaces and textures and several novel features have been devised, including B-Spline Patches with Locality-Constrained Linear coding, and Topology Spatial Distance to describe and quantify generic landmarks (wrinkles and folds). The essence of this proposed architecture comprises 3D generic surface parsing and interpretation, which is critical to underpinning a number of laundering tasks and has the potential to be extended to other rigid and non-rigid object perception and manipulation tasks. The experimental results presented in this thesis demonstrate that: firstly, the proposed grasp- ing approach achieves on-average 84.7% accuracy; secondly, the proposed flattening approach is able to flatten towels, t-shirts and pants (shorts) within 9 iterations on-average; thirdly, the proposed clothes recognition pipeline can recognise clothes categories from highly wrinkled configurations and advances the state-of-the-art by 36% in terms of classification accuracy, achieving an 83.2% true-positive classification rate when discriminating between five categories of clothes; finally the Gaussian Process based interactive perception approach exhibits a substantial improvement over single-shot perception. Accordingly, this thesis has advanced the state-of-the-art of robot clothes perception and manipulation.

AMADEUS: advanced manipulation for deep underwater sampling

AMADEUS is a dexterous subsea robot hand incorporating force and slip contact sensing, using fluid filled tentacles for fingers. Hydraulic pressure variations in each of three flexible tubes (bellows) in each finger create a bending moment, and consequent motion or increase in contact force during grasping. Such fingers have inherent passive compliance, no moving parts, and are naturally depth pressure-compensated, making them ideal for reliable use in the deep ocean. In addition to the mechanical design, development of the hand has also considered closed loop finger position and force control, coordinated finger motion for grasping, force and slip sensor development/signal processing, and reactive world modeling/planning for supervisory `blind grasping¿. Initially, the application focus is for marine science tasks, but broader roles in offshore oil and gas, salvage, and military use are foreseen. Phase I of the project is complete, with the construction of a first prototype. Phase I1 is now underway, to deploy the hand from an underwater robot arm, and carry out wet trials with users.

AMADEUS: advanced manipulation for deep underwater sampling

AMADEUS is a dexterous subsea robot hand incorporating force and slip contact sensing, using fluid filled tentacles for fingers. Hydraulic pressure variations in each of three flexible tubes (bellows) in each finger create a bending moment, and consequent motion or increase in contact force during grasping. Such fingers have inherent passive compliance, no moving parts, and are naturally depth pressure-compensated, making them ideal for reliable use in the deep ocean. In addition to the mechanical design, development of the hand has also considered closed loop finger position and force control, coordinated finger motion for grasping, force and slip sensor development/signal processing, and reactive world modeling/planning for supervisory `blind grasping¿. Initially, the application focus is for marine science tasks, but broader roles in offshore oil and gas, salvage, and military use are foreseen. Phase I of the project is complete, with the construction of a first prototype. Phase I1 is now underway, to deploy the hand from an underwater robot arm, and carry out wet trials with users.

New control architecturebased on PXI for a 3-finger haptic device applied to virtual manipulation

To perform advanced manipulation of remote environments such as grasping, more than one finger is required implying higher requirements for the control architecture. This paper presents the design and control of a modular 3-finger haptic device that can be used to interact with virtual scenarios or to teleoperate dexterous remote hands. In a modular haptic device, each module allows the interaction with a scenario by using a single finger; hence, multi-finger interaction can be achieved by adding more modules. Control requirements for a multifinger haptic device are analyzed and new hardware/software architecture for these kinds of devices is proposed. The software architecture described in this paper is distributed and the different modules communicate to allow the remote manipulation. Moreover, an application in which this haptic device is used to interact with a virtual scenario is shown.

Manipulation of quantum state transfer in cold Rydberg atom collisions

We investigate the role of the dc Stark effect in multilevel pairwise interactions between cold Rydberg atoms. We have observed the decay of nD + nD quasi-molecules by detecting the products in the (n + 2) P state after pulsed excitation for 29 <= n <= 41. The decay rate can be manipulated with a dc electric field and requires a consideration of the multilevel nature of the process to explain the observations. The time dependence of the (n + 2) P signal is found to support a time-dependent picture of the dynamics.

Manipulation of rest period length induces different causes of fatigue in vertical jumping

The aim of this study was to directly compare the causes of fatigue after a short- and a long-rest interval between consecutive stretch-shortening cycle exercises. Eleven healthy males jumped with different resting period lengths (short = 6.1 +/- 1 s, long = 8.6 +/- 0.9 s), performing countermovement jumps at 95% of their maximal jump height until they were unable to sustain the target height. After short- and long-rest, the maximal voluntary isometric contraction knee extension torque decreased (-7%; p = 0.04), comparing to values obtained before exercise protocols. No change was seen from pre- to post-exercise, for either short- or long-rest, in biceps femoris coactivation (-1%; p = 0.95), peak-to-peak amplitude (1%; p = 0.95) and duration (-8%; p = 0.92) of the compound muscle action potential of the vastus lateralis. Evoked peak twitch torque reduced after both exercise protocols (short = -26%, long = -32%; p = 0.003) indicating peripheral fatigue. However, central fatigue occurred only after short-rest evidenced by a reduction in voluntary activation of the quadriceps muscle (-14%; p = 0.013) measured using the interpolated twitch technique. In conclusion, after Stretch-shortening cycle exercise using short rest period length, the cause of fatigue was central and peripheral, while after using long rest period length, the cause of fatigue was peripheral.

Monocular vision and increased distance reducing the effects of visual manipulation on body sway

The goal of this study was to examine the coupling between visual information and body sway with binocular and monocular vision at two distances from the front wall of a moving room. Ten participants stood as still as possible inside of a moving room facing the front wall in conditions that combined room movement with monocular/binocular vision and distance from the front wall (75 and 150cm). Visual information effect on body sway decreased with monocular vision and with increased distance from the front wall. In addition, the combination of monocular vision with the farther distance resulted in the smallest body sway response to the driving stimulus provided by the moving room. These results suggest that binocularvision near the front wall provides visual information of a better quality than the monocular vision far from the front wall. We discuss the results with respect to two modes of visual detection of body sway: ocular and extraocular. (C) 2009 Elsevier Ireland Ltd. All rights reserved.

Prospects for control of African trypanosomiasis by tsetse vector manipulation

The extensive antigenic variation phenomena African trypanosomes display in their mammalian host have hampered efforts to develop effective vaccines against trypanosomiasis. Human disease management aims largely to treat infected hosts by chemotherapy, whereas control of animal diseases relies on reducing tsetse populations as well as on drug therapy. The control strategies for animal diseases are carried out and financed by livestock owners, who have an obvious economic incentive. Sustaining largely insecticide-based control at a local level and relying on drugs for treatment of infected hosts for a disease for which there is no evidence of acquired immunity could prove extremely costly in the long run. It is more likely that a combination of several methods in an integrated, phased and area-wide approach would be more effective in controlling these diseases and subsequently improving agricultural output. New approaches that are environmentally acceptable, efficacious and affordable are clearly desirable for control of various medically and agriculturally important insects including tsetse. Here, Serap Aksoy and colleagues discuss molecular genetic approaches to modulate tsetse vector competence.

Control of vector-borne disease by genetic manipulation of insect populations: technological requirements and research priorities

The possibility of controlling vector-borne disease through the development and release of transgenic insect vectors has recently gained popular support and is being actively pursued by a number of research laboratories around the world. Several technical problems must be solved before such a strategy could be implemented: genes encoding refractory traits (traits that render the insect unable to transmit the pathogen) must be identified, a transformation system for important vector species has to be developed, and a strategy to spread the refractory trait into natural vector populations must be designed. Recent advances in this field of research make it seem likely that this technology will be available in the near future. In this paper we review recent progress in this area as well as argue that care should be taken in selecting the most appropriate disease system with which to first attempt this form of intervention. Much attention is currently being given to the application of this technology to the control of malaria, transmitted by Anopheles gambiae in Africa. While malaria is undoubtedly the most important vector-borne disease in the world and its control should remain an important goal, we maintain that the complex epidemiology of malaria together with the intense transmission rates in Africa may make it unsuitable for the first application of this technology. Diseases such as African trypanosomiasis, transmitted by the tsetse fly, or unstable malaria in India may provide more appropriate initial targets to evaluate the potential of this form of intervention.

Photolytic manipulation of [Ca2+](i) reveals slow kinetics of potassium channels underlying the afterhyperpolarization in hipppocampal pyramidal neurons

The identity of the potassium channel underlying the slow, apamin-insensitive component of the afterhyperpolarization current (sl(AHP)) remains unknown. We studied sl(AHP) in CA1 pyramidal neurons using simultaneous whole-cell recording, calcium fluorescence imaging, and flash photolysis of caged compounds. Intracellular calcium concentration ([Ca2+](i)) peaked earlier and decayed more rapidly than sl(AHP). Loading cells with low concentrations of the calcium chelator EGTA slowed the activation and decay of sl(AHP). In the presence of EGTA, intracellular calcium decayed with two time constants. When [Ca2+](i) was increased rapidly after photolysis of DM-Nitrophen, both apamin-sensitive and apamin-insensitive outward currents were activated. The apamin-sensitive current activated rapidly (<20 msec), whereas the apamin-insensitive current activated more slowly (180 msec). The apamin-insensitive current was reduced by application of serotonin and carbachol, confirming that it was caused by sl(AHP) channels. When [Ca2+](i) was decreased rapidly via photolysis of diazo-2, the decay of sl(AHP) was similar to control (1.7 sec). All results could be reproduced by a model potassium channel gated by calcium, suggesting that the channels underlying sl(AHP) have intrinsically slow kinetics because of their high affinity for calcium.

When the going gets rough: effect of maternal size manipulation on larval quality

Variation in larval size has been shown to be an important factor for the post-metamorphic performance of marine invertebrates but, despite its importance, few sources of this variation have been identified. For a range of taxa, offspring size is positively correlated with maternal size but the reasons for this correlation remain unclear. We halved the size of colonies in the bryozoan Bugula neritina 1 wk prior to reproduction (but during embryogenesis) to determine if larval size is a fixed or plastic trait. We manipulated colonies in such a way that the ratio of feeding zooids to reproductive zooids was constant between treatment and control colonies. We found that manipulating colony size strongly affects larval size; halved colonies produced larvae that were similar to13% smaller than those produced by intact colonies. We entered these data into a simple model based on previous work to estimate the likely post-metamorphic consequences of this reduction in larval size. The model predicted that larvae that came from manipulated colonies would suffer similar to300% higher post-metamorphic mortality and similar to50% lower fecundity as adults. Colonies that are faced with a stress appear to be trading off current offspring fitness to maximize their own long-term fitness and this may explain previous observations of compensatory growth in colonial organisms. This study demonstrates that larval size is a surprisingly dynamic trait and strong links exist between the maternal phenotype and the fitness of the offspring. The performance of settling larvae may be determined not only by their larval experience but also by the experience of their mothers.

Immediate and lasting improvements in weight distribution seen in baropodometry following a high-velocity, low-amplitude thrust manipulation of the sacroiliac joint

The biomechanics of the sacroiliac joint makes the pelvic segment responsible for proper weight distribution between lower extremities; however, it is known to be susceptible to altered mobility. The objective of this study was to analyze baropodometric responses following thrust manipulation on subjects with sacroiliac joint restrictions. Twenty asymptomatic subjects were submitted to computerized baropodometric analysis before, after, and seven days following sacroiliac manipulation. The variables peak pressure and contact area were obtained at each of these periods as the average of absolute values of the difference between the right and left foot based on three trials. Data revealed significant reduction only in peak pressure immediately after manipulation and at follow-up when compared to pre-manipulative values (p < 0.05). Strong correlation was found between the dominant foot and the foot with greater contact area (r - 0.978), as well as between the side of joint restriction and the foot with greater contact area (r = 0.884). Weak correlation was observed between the dominant foot and the foot with greater peak pressure (r = 0.501), as well as between the side of joint restriction and the foot with greater peak pressure (r = 0.694). The results suggest that sacroiliac joint manipulation can influence peak pressure distribution between feet, but contact area does not seem to be related to the biomechanical aspects addressed in this study. (C) 2011 Elsevier Ltd. All rights reserved.