Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors’ feedback

Tactile sensors play an important role in robotics manipulation to perform dexterous and complex tasks. This paper presents a novel control framework to perform dexterous manipulation with multi-fingered robotic hands using feedback data from tactile and visual sensors. This control framework permits the definition of new visual controllers which allow the path tracking of the object motion taking into account both the dynamics model of the robot hand and the grasping force of the fingertips under a hybrid control scheme. In addition, the proposed general method employs optimal control to obtain the desired behaviour in the joint space of the fingers based on an indicated cost function which determines how the control effort is distributed over the joints of the robotic hand. Finally, authors show experimental verifications on a real robotic manipulation system for some of the controllers derived from the control framework.

Microfabricated elastomer tactile sensors for robotic fingertip systems

Tactile sensing is an important aspect of robotic systems, and enables safe, dexterous robot-environment interaction. The design and implementation of tactile sensors on robots has been a topic of research over the past 30 years, and current challenges include mechanically flexible “sensing skins”, high dynamic range (DR) sensing (i.e.: high force range and fine force resolution), multi-axis sensing, and integration between the sensors and robot. This dissertation focuses on addressing some of these challenges through a novel manufacturing process that incorporates conductive and dielectric elastomers in a reusable, multilength-scale mold, and new sensor designs for multi-axis sensing that improve force range without sacrificing resolution. A single taxel was integrated into a 1 degree of freedom robotic gripper for closed-loop slip detection. Manufacturing involved casting a composite silicone rubber, polydimethylsiloxane (PDMS) filled with conductive particles such as carbon nanotubes, into a mold to produce microscale flexible features on the order of 10s of microns. Molds were produced via microfabrication of silicon wafers, but were limited in sensing area and were costly. An improved technique was developed that produced molds of acrylic using a computer numerical controlled (CNC) milling machine. This maintained the ability to produce microscale features, and increased the sensing area while reducing costs. New sensing skins had features as small as 20 microns over an area as large as a human hand. Sensor architectures capable of sensing both shear and normal force sensing with high dynamic range were produced. Using this architecture, two sensing modalities were developed: a capacitive approach and a contact resistive approach. The capacitive approach demonstrated better dynamic range, while the contact resistive approach used simpler circuitry. Using the contact resistive approach, normal force range and resolution were 8,000 mN and 1,000 mN, respectively, and shear force range and resolution were 450 mN and 100 mN, respectively. Using the capacitive approach, normal force range and resolution were 10,000 mN and 100 mN, respectively, and shear force range and resolution were 1,500 mN and 50 mN, respectively.

Tactile-Proprioceptive Robotic Grasping

Robotic grasping has been studied increasingly for a few decades. While progress has been made in this field, robotic hands are still nowhere near the capability of human hands. However, in the past few years, the increase in computational power and the availability of commercial tactile sensors have made it easier to develop techniques that exploit the feedback from the hand itself, the sense of touch. The focus of this thesis lies in the use of this sense. The work described in this thesis focuses on robotic grasping from two different viewpoints: robotic systems and data-driven grasping. The robotic systems viewpoint describes a complete architecture for the act of grasping and, to a lesser extent, more general manipulation. Two central claims that the architecture was designed for are hardware independence and the use of sensors during grasping. These properties enables the use of multiple different robotic platforms within the architecture. Secondly, new data-driven methods are proposed that can be incorporated into the grasping process. The first of these methods is a novel way of learning grasp stability from the tactile and haptic feedback of the hand instead of analytically solving the stability from a set of known contacts between the hand and the object. By learning from the data directly, there is no need to know the properties of the hand, such as kinematics, enabling the method to be utilized with complex hands. The second novel method, probabilistic grasping, combines the fields of tactile exploration and grasp planning. By employing well-known statistical methods and pre-existing knowledge of an object, object properties, such as pose, can be inferred with related uncertainty. This uncertainty is utilized by a grasp planning process which plans for stable grasps under the inferred uncertainty.

Design and Control of an Anthropomorphic Robotic Finger with Multi-point Tactile Sensation

The goal of this research is to develop the prototype of a tactile sensing platform for anthropomorphic manipulation research. We investigate this problem through the fabrication and simple control of a planar 2-DOF robotic finger inspired by anatomic consistency, self-containment, and adaptability. The robot is equipped with a tactile sensor array based on optical transducer technology whereby localized changes in light intensity within an illuminated foam substrate correspond to the distribution and magnitude of forces applied to the sensor surface plane. The integration of tactile perception is a key component in realizing robotic systems which organically interact with the world. Such natural behavior is characterized by compliant performance that can initiate internal, and respond to external, force application in a dynamic environment. However, most of the current manipulators that support some form of haptic feedback either solely derive proprioceptive sensation or only limit tactile sensors to the mechanical fingertips. These constraints are due to the technological challenges involved in high resolution, multi-point tactile perception. In this work, however, we take the opposite approach, emphasizing the role of full-finger tactile feedback in the refinement of manual capabilities. To this end, we propose and implement a control framework for sensorimotor coordination analogous to infant-level grasping and fixturing reflexes. This thesis details the mechanisms used to achieve these sensory, actuation, and control objectives, along with the design philosophies and biological influences behind them. The results of behavioral experiments with a simple tactilely-modulated control scheme are also described. The hope is to integrate the modular finger into an %engineered analog of the human hand with a complete haptic system.

Flexible tactile digital feedback for clinical applications

Trauma and damage to the delicate structures of the inner ear frequently occurs during insertion of electrode array into the cochlea. This is strongly related to the excessive manual insertion force of the surgeon without any tool/tissue interaction feedback. The research is examined tool-tissue interaction of large prototype scale (12.5:1) digit embedded with distributive tactile sensor based upon cochlear electrode and large prototype scale (4.5:1) cochlea phantom for simulating the human cochlear which could lead to small scale digit requirements. This flexible digit classified the tactile information from the digit-phantom interaction such as contact status, tip penetration, obstacles, relative shape and location, contact orientation and multiple contacts. The digit, distributive tactile sensors embedded with silicon-substrate is inserted into the cochlea phantom to measure any digit/phantom interaction and position of the digit in order to minimize tissue and trauma damage during the electrode cochlear insertion. The digit is pre-curved in cochlea shape so that the digit better conforms to the shape of the scala tympani to lightly hug the modiolar wall of a scala. The digit have provided information on the characteristics of touch, digit-phantom interaction during the digit insertion. The tests demonstrated that even devices of such a relative simple design with low cost have potential to improve cochlear implants surgery and other lumen mapping applications by providing tactile feedback information by controlling the insertion through sensing and control of the tip of the implant during the insertion. In that approach, the surgeon could minimize the tissue damage and potential damage to the delicate structures within the cochlear caused by current manual electrode insertion of the cochlear implantation. This approach also can be applied diagnosis and path navigation procedures. The digit is a large scale stage and could be miniaturized in future to include more realistic surgical procedures.

An optical fiber Bragg grating tactile sensor

Tactile sensors are needed for many emerging robotic and telepresence applications such as keyhole surgery and robot operation in unstructured environments. We have proposed and demonstrated a tactile sensor consisting of a fibre Bragg grating embedded in a polymer "finger". When the sensor is placed in contact with a surface and translated tangentially across it measurements on the changes in the reflectivity spectrum of the grating provide a measurement of the spatial distribution of forces perpendicular to the surface and thus, through the elasticity of the polymer material, to the surface roughness. Using a sensor fabricated from a Poly Siloxane polymer (Methyl Vinyl Silicone rubber) spherical cap 50 mm in diameter, 6 mm deep with an embedded 10 mm long Bragg grating we have characterised the first and second moment of the grating spectral response when scanned across triangular and semicircular periodic structures both with a modulation depth of 1 mm and a period of 2 mm. The results clearly distinguish the periodicity of the surface structure and the differences between the two different surface profiles. For the triangular structure a central wavelength modulation of 4 pm is observed and includes a fourth harmonic component, the spectral width is modulated by 25 pm. Although crude in comparison to human senses these results clearly shown the potential of such a sensor for tactile imaging and we expect that with further development in optimising both the grating and polymer "finger" properties a much increased sensitivity and spatial resolution is achievable.

Developing sensorized arm skin for an octopus inspired robot

Soft skin artefacts made of knitted nylon reinforced silicon rubber were fabricated mimicking octopus skin. A combination of ecoflex 0030 and 0010 were used as matrix of the composite to obtain the right stiffness for the skin artefacts. Material properties were characterised using static uniaxial tension and scissors cutting tests. Two types of tactile sensors were developed to detect normal contact; one used quantum tunnelling composite materials and the second was fabricated from silicone rubber and a conductive textile. Sensitivities of the sensors were tested by applying different modes of loading and the soft sensors were incorporated into the skin prototype. Passive suckers were developed and tested against squid suckers. An integrated skin prototype with embedded deformable sensors and attached suckers developed for the arm of an octopus inspired robot is also presented.

A three dimensional surface measurement and reconstruction system for respiratory function analysis

Respiration is a complex activity. If the relationship between all neurological and skeletomuscular interactions was perfectly understood, an accurate dynamic model of the respiratory system could be developed and the interaction between different inputs and outputs could be investigated in a straightforward fashion. Unfortunately, this is not the case and does not appear to be viable at this time. In addition, the provision of appropriate sensor signals for such a model would be a considerable invasive task. Useful quantitative information with respect to respiratory performance can be gained from non-invasive monitoring of chest and abdomen motion. Currently available devices are not well suited in application for spirometric measurement for ambulatory monitoring. A sensor matrix measurement technique is investigated to identify suitable sensing elements with which to base an upper body surface measurement device that monitors respiration. This thesis is divided into two main areas of investigation; model based and geometrical based surface plethysmography. In the first instance, chapter 2 deals with an array of tactile sensors that are used as progression of existing and previously investigated volumetric measurement schemes based on models of respiration. Chapter 3 details a non-model based geometrical approach to surface (and hence volumetric) profile measurement. Later sections of the thesis concentrate upon the development of a functioning prototype sensor array. To broaden the application area the study has been conducted as it would be fore a generically configured sensor array. In experimental form the system performance on group estimation compares favourably with existing system on volumetric performance. In addition provides continuous transient measurement of respiratory motion within an acceptable accuracy using approximately 20 sensing elements. Because of the potential size and complexity of the system it is possible to deploy it as a fully mobile ambulatory monitoring device, which may be used outside of the laboratory. It provides a means by which to isolate coupled physiological functions and thus allows individual contributions to be analysed separately. Thus facilitating greater understanding of respiratory physiology and diagnostic capabilities. The outcome of the study is the basis for a three-dimensional surface contour sensing system that is suitable for respiratory function monitoring and has the prospect with future development to be incorporated into a garment based clinical tool.

Por baixo da pele outra pele: conjunto de obras artísticas. Corpo, ecrã e interface para uma visualidade háptica interativa

A tese teórico-prática de doutoramento apresentada contempla a investigação, a criação, a produção e a conceptualização da instalação artística interativa intitulada Por baixo da pele outra pele. A Obra é constituída por três objetos tridimensionais concebidos à escala humana. Recorre a materiais flexíveis, como têxteis, convidando o público (interator), à envolvência física, numa relação corporal sensorial e sensual com a obra. Os objetos contêm dispositivos técnicos interativos e sensores tácteis, que, ao serem utilizados, desencadeiam estímulos multissensoriais no espectador. A instalação interativa focaliza a experiência háptica e íntima do interator considerando os seus mecanismos sensoriais e cognitivos como um potencial aparato na construção de experiências fenomenológicas, singulares e individuais. A autora considera a interatividade enquanto elemento potenciador da experiência estética visual háptica. Na argumentação conceitual da obra, reflete-se sobre o tema da visualidade háptica interativa a partirdos conceitos de ecrã, corpo e interface, assim como de endossensorialidade. Instrumentam-se metodologias de investigação em ação, experimentais e observacionais. Apresentam-se os processos investigativos, criativos e técnicos necessários ao desenvolvimento e à materialização da instalação artística. A investigação revela-se de grande interesse para o avanço da pesquisa de novas linguagens experimentais apresentando estratégias de criação artística que, ao privilegiarem o corpo físico e fenomenológico do interator, transpõe a experiência háptica interativa para um grau interno de imersão motoro-sensorial; Underneath the skin another skin: art installation. Body, screen and interface towards an interactive haptic visuality. Abstract: The theoretical-practical doctorate dissertation presents the research and conceptual framework behind, and the processes leading to, the creation and production of the interactive installation art piece Underneath the skin, another skin. The piece is presented in the shape of three human-scale tridimentional objects. It is made from flexible materials, such as textiles, inviting the (interacting) audience, to physically engage in a bodily sensorial, and sensuous, relationship with the artwork. The objects enclose interactive devices and tactile sensors that, when used, trigger in the interactor multiple sensorial stimuli. The interactive installation focuses on the interactor's intimate haptic experience taking in consideration his or hers sensorial and cognitive mechanisms as a potential apparatus in the construction of unique individual phenomenological experiences. The author understands interactivity as a triggering element into an haptic visual aesthetical experience. The supporting conceptual reasoning deals with thought and criticism on interactive haptic visuality applied to the concepts of screen, body and interface, as well as with that of endo-sensoriality. The dissertation describes the use of experimental and observation research methodologies. It also elaborates on the research, creative and technical processes at play in the installation's development and realization. The research at hand has shown great potential for the further development of new experimental languages, as it presents art-creation strategies privileging the interactor's physical and phenomenological body, and thus able to take the interactive haptic experience onto an greater inner level of motor-sensorial immersion.

Nanostructured piezoelectric materials for the design and development of self-sensing composite materials and energy harvesting devices

The work activities reported in this PhD thesis regard the functionalization of composite materials and the realization of energy harvesting devices by using nanostructured piezoelectric materials, which can be integrated in the composite without affecting its mechanical properties. The self-sensing composite materials were fabricated by interleaving between the plies of the laminate the piezoelectric elements. The problem of negatively impacting on the mechanical properties of the hosting structure was addressed by shaping the piezoelectric materials in appropriate ways. In the case of polymeric piezoelectric materials, the electrospinning technique allowed to produce highly-porous nanofibrous membranes which can be immerged in the hosting matrix without inducing delamination risk. The flexibility of the polymers was exploited also for the production of flexible tactile sensors. The sensing performances of the specimens were evaluated also in terms of lifetime with fatigue tests. In the case of ceramic piezo-materials, the production and the interleaving of nanometric piezoelectric powder limitedly affected the impact resistance of the laminate, which showed enhanced sensing properties. In addition to this, a model was proposed to predict the piezoelectric response of the self-sensing composite materials as function of the amount of the piezo-phase within the laminate and to adapt its sensing functionalities also for quasi-static loads. Indeed, one final application of the work was to integrate the piezoelectric nanofibers in the sole of a prosthetic foot in order to detect the walking cycle, which has a period in the order of 1 second. In the end, the energy harvesting capabilities of the piezoelectric materials were investigated, with the aim to design wearable devices able to collect energy from the environment and from the body movements. The research activities focused both on the power transfer capability to an external load and the charging of an energy storage unit, like, e.g., a supercapacitor.

Fibre Bragg grating sensors for distributive tactile sensing

Distributive tactile sensing is a method of tactile sensing in which a small number of sensors monitors the behaviour of a flexible substrate which is in contact with the object being sensed. This paper describes the first use of fibre Bragg grating sensors in such a system. Two systems are presented: the first is a one-dimensional metal strip with an array of four sensors, which is capable of detecting the magnitude and position of a contacting load. This system is favourably compared experimentally with a similar system using resistive strain gauges. The second system is a two-dimensional steel plate with nine sensors which is able to distinguish the position and shape of a contacting load, or the positions of two loads simultaneously. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact. Issues and limitations of the systems are discussed, along with proposed solutions to some of the difficulties.

Distributive tactile sensing using fibre Bragg grating sensors

Two distributive tactile sensing systems are presented, based on fibre Bragg grating sensors. The first is a onedimensional metal strip with an array of 4 sensors, which is capable of detecting the magnitude and position of a contacting load. This system is compared experimentally with a similar system using resistive strain gauges. The second is a two-dimensional steel plate with 9 sensors which is able to distinguish the position and shape of a contacting load. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact.

Fibre Bragg grating sensors for distributive tactile sensing

Distributive tactile sensing is a method of tactile sensing in which a small number of sensors monitors the behaviour of a flexible substrate which is in contact with the object being sensed. This paper describes the first use of fibre Bragg grating sensors in such a system. Two systems are presented: the first is a one-dimensional metal strip with an array of four sensors, which is capable of detecting the magnitude and position of a contacting load. This system is favourably compared experimentally with a similar system using resistive strain gauges. The second system is a two-dimensional steel plate with nine sensors which is able to distinguish the position and shape of a contacting load, or the positions of two loads simultaneously. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact. Issues and limitations of the systems are discussed, along with proposed solutions to some of the difficulties. © 2007 IOP Publishing Ltd.

Distributive tactile sensing using fibre Bragg grating sensors for biomedical applications

Artificial tactile sensing systems using the distributive tactile sensing technique and fibre Bragg grating sensors are presented. A one-dimensional arrangement, with possible applications in an endoscope, is compared with a similar arrangement using conventional electronic sensors. A two-dimensional sensing surface is described, with potential applications in human balance and gait analysis, capable of detecting simultaneously the position and shape of an object placed upon it. It is believed that this work represents the first use of fibre Bragg grating sensors in a distributive sensing regime.

Distributive tactile sensing using fibre bragg grating sensors

Two distributive tactile sensing systems are presented, based on fibre Bragg grating sensors. The first is a one-dimensional metal strip with an array of 4 sensors, which is capable of detecting the magnitude and position of a contacting load. This system is compared experimentally with a similar system using resistive strain gauges. The second is a two-dimensional steel plate with 9 sensors which is able to distinguish the position and shape of a contacting load. This system is compared with a similar system using 16 infrared displacement sensors. Each system uses neural networks to process the sensor data to give information concerning the type of contact.