Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors’ feedback

Tactile sensors play an important role in robotics manipulation to perform dexterous and complex tasks. This paper presents a novel control framework to perform dexterous manipulation with multi-fingered robotic hands using feedback data from tactile and visual sensors. This control framework permits the definition of new visual controllers which allow the path tracking of the object motion taking into account both the dynamics model of the robot hand and the grasping force of the fingertips under a hybrid control scheme. In addition, the proposed general method employs optimal control to obtain the desired behaviour in the joint space of the fingers based on an indicated cost function which determines how the control effort is distributed over the joints of the robotic hand. Finally, authors show experimental verifications on a real robotic manipulation system for some of the controllers derived from the control framework.

Microfabricated elastomer tactile sensors for robotic fingertip systems

Tactile sensing is an important aspect of robotic systems, and enables safe, dexterous robot-environment interaction. The design and implementation of tactile sensors on robots has been a topic of research over the past 30 years, and current challenges include mechanically flexible “sensing skins”, high dynamic range (DR) sensing (i.e.: high force range and fine force resolution), multi-axis sensing, and integration between the sensors and robot. This dissertation focuses on addressing some of these challenges through a novel manufacturing process that incorporates conductive and dielectric elastomers in a reusable, multilength-scale mold, and new sensor designs for multi-axis sensing that improve force range without sacrificing resolution. A single taxel was integrated into a 1 degree of freedom robotic gripper for closed-loop slip detection. Manufacturing involved casting a composite silicone rubber, polydimethylsiloxane (PDMS) filled with conductive particles such as carbon nanotubes, into a mold to produce microscale flexible features on the order of 10s of microns. Molds were produced via microfabrication of silicon wafers, but were limited in sensing area and were costly. An improved technique was developed that produced molds of acrylic using a computer numerical controlled (CNC) milling machine. This maintained the ability to produce microscale features, and increased the sensing area while reducing costs. New sensing skins had features as small as 20 microns over an area as large as a human hand. Sensor architectures capable of sensing both shear and normal force sensing with high dynamic range were produced. Using this architecture, two sensing modalities were developed: a capacitive approach and a contact resistive approach. The capacitive approach demonstrated better dynamic range, while the contact resistive approach used simpler circuitry. Using the contact resistive approach, normal force range and resolution were 8,000 mN and 1,000 mN, respectively, and shear force range and resolution were 450 mN and 100 mN, respectively. Using the capacitive approach, normal force range and resolution were 10,000 mN and 100 mN, respectively, and shear force range and resolution were 1,500 mN and 50 mN, respectively.

Tactile Sensing: new directions, new challenges

Lee M.H., ?Tactile Sensing: new directions, new challenges?, Int J. Robotics Research 19: 7, 636-643. July 2000.

Tactile Sensing for Mechatronics: A State of the Art Survey

Lee M.H. and Nicholls H.R., Tactile Sensing for Mechatronics: A State of the Art Survey, Mechatronics, 9, Jan 1999, pp1-31.

Design and Control of an Anthropomorphic Robotic Finger with Multi-point Tactile Sensation

The goal of this research is to develop the prototype of a tactile sensing platform for anthropomorphic manipulation research. We investigate this problem through the fabrication and simple control of a planar 2-DOF robotic finger inspired by anatomic consistency, self-containment, and adaptability. The robot is equipped with a tactile sensor array based on optical transducer technology whereby localized changes in light intensity within an illuminated foam substrate correspond to the distribution and magnitude of forces applied to the sensor surface plane. The integration of tactile perception is a key component in realizing robotic systems which organically interact with the world. Such natural behavior is characterized by compliant performance that can initiate internal, and respond to external, force application in a dynamic environment. However, most of the current manipulators that support some form of haptic feedback either solely derive proprioceptive sensation or only limit tactile sensors to the mechanical fingertips. These constraints are due to the technological challenges involved in high resolution, multi-point tactile perception. In this work, however, we take the opposite approach, emphasizing the role of full-finger tactile feedback in the refinement of manual capabilities. To this end, we propose and implement a control framework for sensorimotor coordination analogous to infant-level grasping and fixturing reflexes. This thesis details the mechanisms used to achieve these sensory, actuation, and control objectives, along with the design philosophies and biological influences behind them. The results of behavioral experiments with a simple tactilely-modulated control scheme are also described. The hope is to integrate the modular finger into an %engineered analog of the human hand with a complete haptic system.

Flexible tactile digital feedback for clinical applications

Trauma and damage to the delicate structures of the inner ear frequently occurs during insertion of electrode array into the cochlea. This is strongly related to the excessive manual insertion force of the surgeon without any tool/tissue interaction feedback. The research is examined tool-tissue interaction of large prototype scale (12.5:1) digit embedded with distributive tactile sensor based upon cochlear electrode and large prototype scale (4.5:1) cochlea phantom for simulating the human cochlear which could lead to small scale digit requirements. This flexible digit classified the tactile information from the digit-phantom interaction such as contact status, tip penetration, obstacles, relative shape and location, contact orientation and multiple contacts. The digit, distributive tactile sensors embedded with silicon-substrate is inserted into the cochlea phantom to measure any digit/phantom interaction and position of the digit in order to minimize tissue and trauma damage during the electrode cochlear insertion. The digit is pre-curved in cochlea shape so that the digit better conforms to the shape of the scala tympani to lightly hug the modiolar wall of a scala. The digit have provided information on the characteristics of touch, digit-phantom interaction during the digit insertion. The tests demonstrated that even devices of such a relative simple design with low cost have potential to improve cochlear implants surgery and other lumen mapping applications by providing tactile feedback information by controlling the insertion through sensing and control of the tip of the implant during the insertion. In that approach, the surgeon could minimize the tissue damage and potential damage to the delicate structures within the cochlear caused by current manual electrode insertion of the cochlear implantation. This approach also can be applied diagnosis and path navigation procedures. The digit is a large scale stage and could be miniaturized in future to include more realistic surgical procedures.

An optical fiber Bragg grating tactile sensor

Tactile sensors are needed for many emerging robotic and telepresence applications such as keyhole surgery and robot operation in unstructured environments. We have proposed and demonstrated a tactile sensor consisting of a fibre Bragg grating embedded in a polymer "finger". When the sensor is placed in contact with a surface and translated tangentially across it measurements on the changes in the reflectivity spectrum of the grating provide a measurement of the spatial distribution of forces perpendicular to the surface and thus, through the elasticity of the polymer material, to the surface roughness. Using a sensor fabricated from a Poly Siloxane polymer (Methyl Vinyl Silicone rubber) spherical cap 50 mm in diameter, 6 mm deep with an embedded 10 mm long Bragg grating we have characterised the first and second moment of the grating spectral response when scanned across triangular and semicircular periodic structures both with a modulation depth of 1 mm and a period of 2 mm. The results clearly distinguish the periodicity of the surface structure and the differences between the two different surface profiles. For the triangular structure a central wavelength modulation of 4 pm is observed and includes a fourth harmonic component, the spectral width is modulated by 25 pm. Although crude in comparison to human senses these results clearly shown the potential of such a sensor for tactile imaging and we expect that with further development in optimising both the grating and polymer "finger" properties a much increased sensitivity and spatial resolution is achievable.

Development of a method to measure upper eyelid pressure on the cornea

Purpose: The cornea is known to be susceptible to forces exerted by eyelids. There have been previous attempts to quantify eyelid pressure but the reliability of the results is unclear. The purpose of this study was to develop a technique using piezoresistive pressure sensors to measure upper eyelid pressure on the cornea. Methods: The technique was based on the use of thin (0.18 mm) tactile piezoresistive pressure sensors, which generate a signal related to the applied pressure. A range of factors that influence the response of this pressure sensor were investigated along with the optimal method of placing the sensor in the eye. Results: Curvature of the pressure sensor was found to impart force, so the sensor needed to remain flat during measurements. A large rigid contact lens was designed to have a flat region to which the sensor was attached. To stabilise the contact lens during measurement, an apparatus was designed to hold and position the sensor and contact lens combination on the eye. A calibration system was designed to apply even pressure to the sensor when attached to the contact lens, so the raw digital output could be converted to actual pressure units. Conclusions: Several novel procedures were developed to use tactile sensors to measure eyelid pressure. The quantification of eyelid pressure has a number of applications including eyelid reconstructive surgery and the design of soft and rigid contact lenses.

Many-Valued Logic and Qualitative Modelling of Electrical Circuits

Lee M.H., Many-Valued Logic and Qualitative Modelling of Electrical Circuits, in Proc. QR?2000, 14th Int. Workshop on Qualitative Reasoning, Morelia, Mexico June 3rd - 7th 2000.

Developing sensorized arm skin for an octopus inspired robot

Soft skin artefacts made of knitted nylon reinforced silicon rubber were fabricated mimicking octopus skin. A combination of ecoflex 0030 and 0010 were used as matrix of the composite to obtain the right stiffness for the skin artefacts. Material properties were characterised using static uniaxial tension and scissors cutting tests. Two types of tactile sensors were developed to detect normal contact; one used quantum tunnelling composite materials and the second was fabricated from silicone rubber and a conductive textile. Sensitivities of the sensors were tested by applying different modes of loading and the soft sensors were incorporated into the skin prototype. Passive suckers were developed and tested against squid suckers. An integrated skin prototype with embedded deformable sensors and attached suckers developed for the arm of an octopus inspired robot is also presented.

A three dimensional surface measurement and reconstruction system for respiratory function analysis

Respiration is a complex activity. If the relationship between all neurological and skeletomuscular interactions was perfectly understood, an accurate dynamic model of the respiratory system could be developed and the interaction between different inputs and outputs could be investigated in a straightforward fashion. Unfortunately, this is not the case and does not appear to be viable at this time. In addition, the provision of appropriate sensor signals for such a model would be a considerable invasive task. Useful quantitative information with respect to respiratory performance can be gained from non-invasive monitoring of chest and abdomen motion. Currently available devices are not well suited in application for spirometric measurement for ambulatory monitoring. A sensor matrix measurement technique is investigated to identify suitable sensing elements with which to base an upper body surface measurement device that monitors respiration. This thesis is divided into two main areas of investigation; model based and geometrical based surface plethysmography. In the first instance, chapter 2 deals with an array of tactile sensors that are used as progression of existing and previously investigated volumetric measurement schemes based on models of respiration. Chapter 3 details a non-model based geometrical approach to surface (and hence volumetric) profile measurement. Later sections of the thesis concentrate upon the development of a functioning prototype sensor array. To broaden the application area the study has been conducted as it would be fore a generically configured sensor array. In experimental form the system performance on group estimation compares favourably with existing system on volumetric performance. In addition provides continuous transient measurement of respiratory motion within an acceptable accuracy using approximately 20 sensing elements. Because of the potential size and complexity of the system it is possible to deploy it as a fully mobile ambulatory monitoring device, which may be used outside of the laboratory. It provides a means by which to isolate coupled physiological functions and thus allows individual contributions to be analysed separately. Thus facilitating greater understanding of respiratory physiology and diagnostic capabilities. The outcome of the study is the basis for a three-dimensional surface contour sensing system that is suitable for respiratory function monitoring and has the prospect with future development to be incorporated into a garment based clinical tool.

Por baixo da pele outra pele: conjunto de obras artísticas. Corpo, ecrã e interface para uma visualidade háptica interativa

A tese teórico-prática de doutoramento apresentada contempla a investigação, a criação, a produção e a conceptualização da instalação artística interativa intitulada Por baixo da pele outra pele. A Obra é constituída por três objetos tridimensionais concebidos à escala humana. Recorre a materiais flexíveis, como têxteis, convidando o público (interator), à envolvência física, numa relação corporal sensorial e sensual com a obra. Os objetos contêm dispositivos técnicos interativos e sensores tácteis, que, ao serem utilizados, desencadeiam estímulos multissensoriais no espectador. A instalação interativa focaliza a experiência háptica e íntima do interator considerando os seus mecanismos sensoriais e cognitivos como um potencial aparato na construção de experiências fenomenológicas, singulares e individuais. A autora considera a interatividade enquanto elemento potenciador da experiência estética visual háptica. Na argumentação conceitual da obra, reflete-se sobre o tema da visualidade háptica interativa a partirdos conceitos de ecrã, corpo e interface, assim como de endossensorialidade. Instrumentam-se metodologias de investigação em ação, experimentais e observacionais. Apresentam-se os processos investigativos, criativos e técnicos necessários ao desenvolvimento e à materialização da instalação artística. A investigação revela-se de grande interesse para o avanço da pesquisa de novas linguagens experimentais apresentando estratégias de criação artística que, ao privilegiarem o corpo físico e fenomenológico do interator, transpõe a experiência háptica interativa para um grau interno de imersão motoro-sensorial; Underneath the skin another skin: art installation. Body, screen and interface towards an interactive haptic visuality. Abstract: The theoretical-practical doctorate dissertation presents the research and conceptual framework behind, and the processes leading to, the creation and production of the interactive installation art piece Underneath the skin, another skin. The piece is presented in the shape of three human-scale tridimentional objects. It is made from flexible materials, such as textiles, inviting the (interacting) audience, to physically engage in a bodily sensorial, and sensuous, relationship with the artwork. The objects enclose interactive devices and tactile sensors that, when used, trigger in the interactor multiple sensorial stimuli. The interactive installation focuses on the interactor's intimate haptic experience taking in consideration his or hers sensorial and cognitive mechanisms as a potential apparatus in the construction of unique individual phenomenological experiences. The author understands interactivity as a triggering element into an haptic visual aesthetical experience. The supporting conceptual reasoning deals with thought and criticism on interactive haptic visuality applied to the concepts of screen, body and interface, as well as with that of endo-sensoriality. The dissertation describes the use of experimental and observation research methodologies. It also elaborates on the research, creative and technical processes at play in the installation's development and realization. The research at hand has shown great potential for the further development of new experimental languages, as it presents art-creation strategies privileging the interactor's physical and phenomenological body, and thus able to take the interactive haptic experience onto an greater inner level of motor-sensorial immersion.

Fibre Bragg grating sensors for distributive tactile sensing

Distributive tactile sensing is a method of tactile sensing in which a small number of sensors monitors the behaviour of a flexible substrate which is in contact with the object being sensed. This paper describes the first use of fibre Bragg grating sensors in such a system. Two systems are presented: the first is a one-dimensional metal strip with an array of four sensors, which is capable of detecting the magnitude and position of a contacting load. This system is favourably compared experimentally with a similar system using resistive strain gauges. The second system is a two-dimensional steel plate with nine sensors which is able to distinguish the position and shape of a contacting load, or the positions of two loads simultaneously. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact. Issues and limitations of the systems are discussed, along with proposed solutions to some of the difficulties.

Distributive tactile sensing using fibre Bragg grating sensors

Two distributive tactile sensing systems are presented, based on fibre Bragg grating sensors. The first is a onedimensional metal strip with an array of 4 sensors, which is capable of detecting the magnitude and position of a contacting load. This system is compared experimentally with a similar system using resistive strain gauges. The second is a two-dimensional steel plate with 9 sensors which is able to distinguish the position and shape of a contacting load. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact.