Head-controlled assistive telerobot with extended physiological proprioception capability

People with disabilities such as quadriplegia can use mouth-sticks and head-sticks as extension devices to perform desired manipulations. These extensions provide extended proprioception which allows users to directly feel forces and other perceptual cues such as texture present at the tip of the mouth-stick. Such devices are effective for two principle reasons: because of their close contact with the user's tactile and proprioceptive sensing abilities; and because they tend to be lightweight and very stiff, and can thus convey tactile and kinesthetic information with high-bandwidth. Unfortunately, traditional mouth-sticks and head-sticks are limited in workspace and in the mechanical power that can be transferred because of user mobility and strength limitations. We describe an alternative implementation of the head-stick device using the idea of a virtual head-stick: a head-controlled bilateral force-reflecting telerobot. In this system the end-effector of the slave robot moves as if it were at the tip of an imaginary extension of the user's head. The design goal is for the system is to have the same intuitive operation and extended proprioception as a regular mouth-stick effector but with augmentation of workspace volume and mechanical power. The input is through a specially modified six DOF master robot (a PerForceTM hand-controller) whose joints can be back-driven to apply forces at the user's head. The manipulation tasks in the environment are performed by a six degree-of-freedom slave robot (the Zebra-ZEROTM) with a built-in force sensor. We describe the prototype hardware/software implementation of the system, control system design, safety/disability issues, and initial evaluation tasks.

Visuotactile learning and body representation: an ERP study with rubber hands and rubber objects

We studied how the integration of seen and felt tactile stimulation modulates somatosensory processing, and investigated whether visuotactile integration depends on temporal contiguity of stimulation, and its coherence with a pre-existing body representation. During training, participants viewed a rubber hand or a rubber object that was tapped either synchronously with stimulation of their own hand, or in an uncorrelated fashion. In a subsequent test phase, somatosensory event-related potentials (ERPs) were recorded to tactile stimulation of the left or right hand, to assess how tactile processing was affected by previous visuotactile experience during training. An enhanced somatosensory N140 component was elicited after synchronous, compared with uncorrelated, visuotactile training, irrespective of whether participants viewed a rubber hand or rubber object. This early effect of visuotactile integration on somatosensory processing is interpreted as a candidate electrophysiological correlate of the rubber hand illusion that is determined by temporal contiguity, but not by pre-existing body representations. ERPmodulations were observed beyond 200msec post-stimulus, suggesting an attentional bias induced by visuotactile training. These late modulations were absent when the stimulation of a rubber hand and the participant’s own hand was uncorrelated during training, suggesting that pre-existing body representations may affect later stages of tactile processing.

The brain’s fingers and hands

The brain keeps track of the changing positions of body parts in space using a spatial body schema. When subjects localise a tactile stimulus on the skin, they might either use a somatotopic body map, or use a body schema to identify the location of the stimulation in external space. Healthy subjects were touched on the fingertips, with the hands in one of two postures: either the right hand was vertically above the left, or the fingers of both hands were interwoven. Subjects made speeded verbal responses to identify either the finger or the hand that was touched. Interweaving the fingers significantly impaired hand identification across several experiments, but had no effect on finger identification. Our results suggest that identification of fingers occurs in a somatotopic representation or finger schema. Identification of hands uses a general body schema, and is influenced by external spatial location. This dissociation implies that touches on the finger can only be identified with a particular hand after a process of assigning fingers to hands. This assignment is based on external spatial location. Our results suggest a role of the body schema in the identification of structural body parts from touch.

Visual enhancement of touch in spatial body representation

Perception of our own bodies is based on integration of visual and tactile inputs, notably by neurons in the brain’s parietal lobes. Here we report a behavioural consequence of this integration process. Simply viewing the arm can speed up reactions to an invisible tactile stimulus on the arm. We observed this visual enhancement effect only when a tactile task required spatial computation within a topographic map of the body surface and the judgements made were close to the limits of performance. This effect of viewing the body surface was absent or reversed in tasks that either did not require a spatial computation or in which judgements were well above performance limits. We consider possible mechanisms by which vision may influence tactile processing.

Peripersonal space: a multisensory interface for body-object interactions

Research in the last four decades has brought a considerable advance in our understanding of how the brain synthesizes information arising from different sensory modalities. Indeed, many cortical and subcortical areas, beyond those traditionally considered to be ‘associative,’ have been shown to be involved in multisensory interaction and integration (Ghazanfar and Schroeder 2006). Visuo-tactile interaction is of particular interest, because of the prominent role played by vision in guiding our actions and anticipating their tactile consequences in everyday life. In this chapter, we focus on the functional role that visuo-tactile processing may play in driving two types of body-object interactions: avoidance and approach. We will first review some basic features of visuo-tactile interactions, as revealed by electrophysiological studies in monkeys. These will prove to be relevant for interpreting the subsequent evidence arising from human studies. A crucial point that will be stressed is that these visuo-tactile mechanisms have not only sensory, but also motor-related activity that qualifies them as multisensory-motor interfaces. Evidence will then be presented for the existence of functionally homologous processing in the human brain, both from neuropsychological research in brain-damaged patients and in healthy participants. The final part of the chapter will focus on some recent studies in humans showing that the human motor system is provided with a multisensory interface that allows for continuous monitoring of the space near the body (i.e., peripersonal space). We further demonstrate that multisensory processing can be modulated on-line as a consequence of interacting with objects. This indicates that, far from being passive, the monitoring of peripersonal space is an active process subserving actions between our body and objects located in the space around us.

A novel human-machine interface using subdermal magnetic implants

This paper explores a novel tactile human-machine interface based on the controlled stimulation of mechanoreceptors by a subdermal magnetic implant manipulated through an external electromagnet. The selection of a suitable implant magnet and implant site is discussed and an external interface for manipulating the implant is described. The paper also reports on the basic properties of such an interface, including magnetic field strength sensitivity and frequency sensitivity obtained through experimentation on two participants. Finally, the paper presents two practical application scenarios for the interface.

Developing sensorized arm skin for an octopus inspired robot

Soft skin artefacts made of knitted nylon reinforced silicon rubber were fabricated mimicking octopus skin. A combination of ecoflex 0030 and 0010 were used as matrix of the composite to obtain the right stiffness for the skin artefacts. Material properties were characterised using static uniaxial tension and scissors cutting tests. Two types of tactile sensors were developed to detect normal contact; one used quantum tunnelling composite materials and the second was fabricated from silicone rubber and a conductive textile. Sensitivities of the sensors were tested by applying different modes of loading and the soft sensors were incorporated into the skin prototype. Passive suckers were developed and tested against squid suckers. An integrated skin prototype with embedded deformable sensors and attached suckers developed for the arm of an octopus inspired robot is also presented.

The contribution of primary and secondary somatosensory cortices to the representation of body parts and body sides: an fMRI adaptation study

Although the somatosensory homunculus is a classically used description of the way somatosensory inputs are processed in the brain, the actual contributions of primary (SI) and secondary (SII) somatosensory cortices to the spatial coding of touch remain poorly understood. We studied adaptation of the fMRI BOLD response in the somatosensory cortex by delivering pairs of vibrotactile stimuli to the finger tips of the index and middle fingers. The first stimulus (adaptor) was delivered either to the index or to the middle finger of the right or left hand, whereas the second stimulus (test) was always administered to the left index finger. The overall BOLD response evoked by the stimulation was primarily contralateral in SI and was more bilateral in SII. However, our fMRI adaptation approach also revealed that both somatosensory cortices were sensitive to ipsilateral as well as to contralateral inputs. SI and SII adapted more after subsequent stimulation of homologous as compared with nonhomologous fingers, showing a distinction between different fingers. Most importantly, for both somatosensory cortices, this finger-specific adaptation occurred irrespective of whether the tactile stimulus was delivered to the same or to different hands. This result implies integration of contralateral and ipsilateral somatosensory inputs in SI as well as in SII. Our findings suggest that SI is more than a simple relay for sensory information and that both SI and SII contribute to the spatial coding of touch by discriminating between body parts (fingers) and by integrating the somatosensory input from the two sides of the body (hands).

Vision of the body and the differentiation of perceived body side in touch

Although tactile representations of the two body sides are initially segregated into opposite hemispheres of the brain, behavioural interactions between body sides exist and can be revealed under conditions of tactile double simultaneous stimulation (DSS) at the hands. Here we examined to what extent vision can affect body side segregation in touch. To this aim, we changed hand-related visual input while participants performed a go/no-go task to detect a tactile stimulus delivered to one target finger (e.g., right index), stimulated alone or with a concurrent non-target finger either on the same hand (e.g., right middle finger) or on the other hand (e.g., left index finger = homologous; left middle finger = non-homologous). Across experiments, the two hands were visible or occluded from view (Experiment 1), images of the two hands were either merged using a morphing technique (Experiment 2), or were shown in a compatible vs incompatible position with respect to the actual posture (Experiment 3). Overall, the results showed reliable interference effects of DSS, as compared to target-only stimulation. This interference varied as a function of which non-target finger was stimulated, and emerged both within and between hands. These results imply that the competition between tactile events is not clearly segregated across body sides. Crucially, non-informative vision of the hand affected overall tactile performance only when a visual/proprioceptive conflict was present, while neither congruent nor morphed hand vision affected tactile DSS interference. This suggests that DSS operates at a tactile processing stage in which interactions between body sides can occur regardless of the available visual input from the body.

Spatial coding of touch at the fingers: Insights from double simultaneous stimulation within and between hands

We studied the effect of tactile double simultaneous stimulation (DSS) within and between hands to examine spatial coding of touch at the fingers. Participants performed a go/no-go task to detect a tactile stimulus delivered to one target finger (e.g., right index), stimulated alone or with a concurrent non-target finger, either on the same hand (e.g., right middle finger) or on the other hand (e.g., left index finger=homologous; left middle finger=non-homologous). Across blocks we also changed the unseen hands posture (both hands palm down, or one hand rotated palm-up). When both hands were palm-down DSS interference effects emerged both within and between hands, but only when the non-homologous finger served as non-target. This suggests a clear segregation between the fingers of each hand, regardless of finger side. By contrast, when one hand was palm-up interference effects emerged only within hand, whereas between hands DSS interference was considerably reduced or absent. Thus, between hands interference was clearly affected by changes in hands posture. Taken together, these findings provide behavioral evidence in humans for multiple spatial coding of touch during tactile DSS at the fingers. In particular, they confirm the existence of representational stages of touch that distinguish between body-regions more than body-sides. Moreover, they show that the availability of tactile stimulation side becomes prominent when postural update is required.

High sensitivity recording of afferent nerve activity using ultra-compliant microchannel electrodes: an acutein vivovalidation

Neuroprostheses interfaced with transected peripheral nerves are technological routes to control robotic limbs as well as convey sensory feedback to patients suffering from traumatic neural injuries or degenerative diseases. To maximize the wealth of data obtained in recordings, interfacing devices are required to have intrafascicular resolution and provide high signal-to-noise ratio (SNR) recordings. In this paper, we focus on a possible building block of a three-dimensional regenerative implant: a polydimethylsiloxane (PDMS) microchannel electrode capable of highly sensitive recordings in vivo. The PDMS 'micro-cuff' consists of a 3.5 mm long (100 µm × 70 µm cross section) microfluidic channel equipped with five evaporated Ti/Au/Ti electrodes of sub-100 nm thickness. Individual electrodes have average impedance of 640 ± 30 kΩ with a phase angle of −58 ± 1 degrees at 1 kHz and survive demanding mechanical handling such as twisting and bending. In proof-of-principle acute implantation experiments in rats, surgically teased afferent nerve strands from the L5 dorsal root were threaded through the microchannel. Tactile stimulation of the skin was reliably monitored with the three inner electrodes in the device, simultaneously recording signal amplitudes of up to 50 µV under saline immersion. The overall SNR was approximately 4. A small but consistent time lag between the signals arriving at the three electrodes was observed and yields a fibre conduction velocity of 30 m s−1. The fidelity of the recordings was verified by placing the same nerve strand in oil and recording activity with hook electrodes. Our results show that PDMS microchannel electrodes open a promising technological path to 3D regenerative interfaces.

Within, but not between hands interactions in vibrotactile detection thresholds reflect somatosensory receptive field organization

Detection of a tactile stimulus on one finger is impaired when a concurrent stimulus (masker) is presented on an additional finger of the same or the opposite hand. This phenomenon is known to be finger-specific at the within-hand level. However, whether this specificity is also maintained at the between-hand level is not known. In four experiments, we addressed this issue by combining a Bayesian adaptive staircase procedure (QUEST) with a two-interval forced choice (2IFC) design in order to establish threshold for detecting 200ms, 100Hz sinusoidal vibrations applied to the index or little fingertip of either hand (targets). We systematically varied the masker finger (index, middle, ring, or little finger of either hand), while controlling the spatial location of the target and masker stimuli. Detection thresholds varied consistently as a function of the masker finger when the latter was on the same hand (Experiments 1 and 2), but not when on different hands (Experiments 3 and 4). Within the hand, detection thresholds increased for masker fingers closest to the target finger (i.e., middle>ring when the target was index). Between the hands, detection thresholds were higher only when the masker was present on any finger as compared to when the target was presented in isolation. The within hand effect of masker finger is consistent with the segregation of different fingers at the early stages of somatosensory processing, from the periphery to the primary somatosensory cortex (SI). We propose that detection is finger-specific and reflects the organisation of somatosensory receptive fields in SI within, but not between the hands.

Sensory enhancement, a pilot perceptual study of subdermal magnetic implants

Subdermal magnetic implants originated as an art form in the world of body modification. To date an in depth scientific analysis of the benefits of this implant has yet to be established. This research explores the concept of sensory extension of the tactile sense utilising this form of implantation. This relatively simple procedure enables the tactile sense to respond to static and alternating magnetic fields. This is not to say that the underlying biology of the system has changed; i.e. the concept does not increase our tactile frequency response range or sensitivity to pressure, but now does invoke a perceptual response to a stimulus that is not innately available to humans. Within this research two social surveys have been conducted in order to ascertain one, the social acceptance of the general notion of human enhancement, and two the perceptual experiences of individuals with the magnetic implants themselves. In terms of acceptance to the notion of sensory improvement (via implantation) ~39% of the general population questioned responded positively with a further ~25% of the respondents answering with the indecisive response. Thus with careful dissemination a large proportion of individuals may adopt this technology much like this if it were to become available for consumers. Interestingly of the responses collected from the magnetic implants survey ~60% of the respondents actually underwent the implant for magnetic vision purposes. The main contribution of this research however comes from a series of psychophysical testing. In which 7 subjects with subdermal magnetic implants, were cross compared with 7 subjects that had similar magnets superficially attached to their dermis. The experimentation examined multiple psychometric thresholds of the candidates including intensity, frequency and temporal. Whilst relatively simple, the experimental setup for the perceptual experimentation conducted was novel in that custom hardware and protocols were created in order to determine the subjective thresholds of the individuals. Abstract iv The overall purpose of this research is to utilise this concept in high stress scenarios, such as driving or piloting; whereby alerts and warnings could be relayed to an operator without intruding upon their other (typically overloaded) exterior senses (i.e. the auditory and visual senses). Hence each of the thresholding experiments were designed with the intention of utilising the results in the design of signals for information transfer. The findings from the study show that the implanted group of subjects significantly outperformed the superficial group in the absolute intensity threshold experiment, i.e. the implanted group required significantly less force than the superficial group in order to perceive the stimulus. The results for the frequency difference threshold showed no significant difference in the two groups tested. Interestingly however at low frequencies, i.e. 20 and 50 Hz, the ability of the subjects tested to discriminate frequencies significantly increased with more complex waveforms i.e. square and sawtooth, when compared against the typically used sinewave. Furthermore a novel protocol for establishing the temporal gap detection threshold during a temporal numerosity study has been established in this thesis. This experiment measured the subjects’ capability to correctly determine the number of concatenated signals presented to them whilst the time between the signals, referred to as pulses, tended to zero. A significant finding was that when altering the length of, the frequency of, and the number of cycles of the pulses, the time between pulses for correct recognition altered. This finding will ultimately aid in the design of the tactile alerts for this method of information transfer. Preliminary development work for the use of this method of input to the body, in an automotive scenario, is also presented within this thesis in the form of a driving simulation. The overall goal of which is to present warning alerts to a driver, such as rear-to-end collision, or excessive speeds on roads, in order to prevent incidents and penalties from occurring. Discussion on the broader utility of this implant has been presented, reflecting on its potential use as a basis for vibrotactile, and sensory substitution, devices. This discussion furthers with postulations on its use as a human machine interface, as well as how a similar implant could be used within the ear as a hearing aid device.

Effects of prenatal and postnatal depression, and maternal stroking, at the glucocorticoid receptor gene

In animal models, prenatal and postnatal stress is associated with elevated hypothalamic–pituitary axis (HPA) reactivity mediated via altered glucocorticoid receptor (GR) gene expression. Postnatal tactile stimulation is associated with reduced HPA reactivity mediated via increased GR gene expression. In this first study in humans to examine the joint effects of prenatal and postnatal environmental exposures, we report that GR gene (NR3C1) 1-F promoter methylation in infants is elevated in the presence of increased maternal postnatal depression following low prenatal depression, and that this effect is reversed by self-reported stroking of the infants by their mothers over the first weeks of life.

Maternal antenatal anxiety, postnatal stroking and emotional problems in children: outcomes predicted from pre and postnatal programming hypotheses

Background Mothers' self-reported stroking of their infants over the first weeks of life modifies the association between prenatal depression and physiological and emotional reactivity at 7 months, consistent with animal studies of the effects of tactile stimulation. We now investigate whether the effects of maternal stroking persist to 2.5 years. Given animal and human evidence for sex differences in the effects of prenatal stress we compare associations in boys and girls. Method From a general population sample of 1233 first-time mothers recruited at 20 weeks gestation we drew a random sample of 316 for assessment at 32 weeks, stratified by reported inter-partner psychological abuse, a risk indicator for child development. Of these mothers, 243 reported at 5 and 9 weeks how often they stroked their infants, and completed the Child Behavior Checklist (CBCL) at 2.5 years post-delivery. Results There was a significant interaction between prenatal anxiety and maternal stroking in the prediction of CBCL internalizing (p = 0.001) and anxious/depressed scores (p < 0.001). The effects were stronger in females than males, and the three-way interaction prenatal anxiety × maternal stroking × sex of infant was significant for internalizing symptoms (p = 0.003). The interactions arose from an association between prenatal anxiety and internalizing symptoms only in the presence of low maternal stroking. Conclusions The findings are consistent with stable epigenetic effects, many sex specific, reported in animal studies. While epigenetic mechanisms may be underlying the associations, it remains to be established whether stroking affects gene expression in humans.