Sensor Fusion of Proprioception, Force and Vision in Estimation and Robot

Sensor-based robot control allows manipulation in dynamic environments with uncertainties. Vision is a versatile low-cost sensory modality, but low sample rate, high sensor delay and uncertain measurements limit its usability, especially in strongly dynamic environments. Force is a complementary sensory modality allowing accurate measurements of local object shape when a tooltip is in contact with the object. In multimodal sensor fusion, several sensors measuring different modalities are combined to give a more accurate estimate of the environment. As force and vision are fundamentally different sensory modalities not sharing a common representation, combining the information from these sensors is not straightforward. In this thesis, methods for fusing proprioception, force and vision together are proposed. Making assumptions of object shape and modeling the uncertainties of the sensors, the measurements can be fused together in an extended Kalman filter. The fusion of force and visual measurements makes it possible to estimate the pose of a moving target with an end-effector mounted moving camera at high rate and accuracy. The proposed approach takes the latency of the vision system into account explicitly, to provide high sample rate estimates. The estimates also allow a smooth transition from vision-based motion control to force control. The velocity of the end-effector can be controlled by estimating the distance to the target by vision and determining the velocity profile giving rapid approach and minimal force overshoot. Experiments with a 5-degree-of-freedom parallel hydraulic manipulator and a 6-degree-of-freedom serial manipulator show that integration of several sensor modalities can increase the accuracy of the measurements significantly.

Quand l’action surpasse la perception : rôle de la vision et de la proprioception dans la perception et le contrôle en temps réel de l’orientation spatiale de la main

Cette recherche a pour but d’évaluer le rôle de la vision et de la proprioception pour la perception et le contrôle de l’orientation spatiale de la main chez l’humain. L’orientation spatiale de la main est une composante importante des mouvements d’atteinte et de saisie. Toutefois, peu d’attention a été portée à l’étude de l’orientation spatiale de la main dans la littérature. À notre connaissance, cette étude est la première à évaluer spécifiquement l’influence des informations sensorielles et de l’expérience visuelle pour la perception et le contrôle en temps réel de l'orientation spatiale de la main pendant le mouvement d’atteinte naturel vers une cible stationnaire. Le premier objectif était d’étudier la contribution de la vision et de la proprioception dans des tâches de perception et de mouvement d’orientation de la main. Dans la tâche de perception (orientation-matching task), les sujets devaient passivement ou activement aligner une poignée de forme rectangulaire avec une cible fixée dans différentes orientations. Les rotations de l’avant-bras et du poignet étaient soit imposées par l’expérimentateur, soit effectuées par les sujets. Dans la tâche de mouvement d’orientation et d’atteinte simultanées (letter posting task 1), les sujets ont réalisé des mouvements d’atteinte et de rotation simultanées de la main afin d’insérer la poignée rectangulaire dans une fente fixée dans les mêmes orientations. Les tâches ont été réalisées dans différentes conditions sensorielles où l’information visuelle de la cible et de la main était manipulée. Dans la tâche perceptive, une augmentation des erreurs d’orientation de la main a été observée avec le retrait des informations visuelles concernant la cible et/ou ou la main. Lorsque la vision de la main n’était pas permise, il a généralement été observé que les erreurs d’orientation de la main augmentaient avec le degré de rotation nécessaire pour aligner la main et la cible. Dans la tâche de mouvement d’orientation et d’atteinte simultanées, les erreurs ont également augmenté avec le retrait des informations visuelles. Toutefois, les patrons d’erreurs étaient différents de ceux observés dans la tâche de perception, et les erreurs d’orientation n’ont pas augmenté avec le degré de rotation nécessaire pour insérer la poignée dans la fente. En absence de vision de la main, il a été observé que les erreurs d’orientation étaient plus petites dans la tâche de mouvement que de perception, suggérant l’implication de la proprioception pour le contrôle de l’orientation spatiale de la main lors des mouvements d’orientation et d’atteinte simultanées. Le deuxième objectif de cette recherche était d’étudier l’influence de la vision et de la proprioception dans le contrôle en temps réel de l’orientation spatiale de la main. Dans une tâche d’orientation de la main suivie d’une atteinte manuelle (letter posting task 2), les sujets devaient d’abord aligner l’orientation de la même poignée avec la fente fixée dans les mêmes orientations, puis réaliser un mouvement d’atteinte sans modifier l’orientation initiale de la main. Une augmentation des erreurs initiales et finales a été observée avec le retrait des informations visuelles. Malgré la consigne de ne pas changer l’orientation initiale de la main, une diminution des erreurs d’orientation a généralement été observée suite au mouvement d’atteinte, dans toutes les conditions sensorielles testées. Cette tendance n’a pas été observée lorsqu’aucune cible explicite n’était présentée et que les sujets devaient conserver l’orientation de départ de la main pendant le mouvement d’atteinte (mouvement intransitif; letter-posting task 3). La diminution des erreurs pendant l’atteinte manuelle transitive vers une cible explicite (letter-posting task 2), malgré la consigne de ne pas changer l’orientation de la main pendant le mouvement, suggère un mécanisme de corrections automatiques pour le contrôle en temps réel de l’orientation spatiale de la main pendant le mouvement d’atteinte naturel vers une cible stationnaire. Le troisième objectif de cette recherche était d’évaluer la contribution de l’expérience visuelle pour la perception et le contrôle de l’orientation spatiale de la main. Des sujets aveugles ont été testés dans les mêmes tâches de perception et de mouvement. De manière générale, les sujets aveugles ont présenté les mêmes tendances que les sujets voyants testés dans la condition proprioceptive (sans vision), suggérant que l’expérience visuelle n’est pas nécessaire pour le développement d’un mécanisme de correction en temps réel de l’orientation spatiale de la main basé sur la proprioception.

Head-controlled assistive telerobot with extended physiological proprioception capability

People with disabilities such as quadriplegia can use mouth-sticks and head-sticks as extension devices to perform desired manipulations. These extensions provide extended proprioception which allows users to directly feel forces and other perceptual cues such as texture present at the tip of the mouth-stick. Such devices are effective for two principle reasons: because of their close contact with the user's tactile and proprioceptive sensing abilities; and because they tend to be lightweight and very stiff, and can thus convey tactile and kinesthetic information with high-bandwidth. Unfortunately, traditional mouth-sticks and head-sticks are limited in workspace and in the mechanical power that can be transferred because of user mobility and strength limitations. We describe an alternative implementation of the head-stick device using the idea of a virtual head-stick: a head-controlled bilateral force-reflecting telerobot. In this system the end-effector of the slave robot moves as if it were at the tip of an imaginary extension of the user's head. The design goal is for the system is to have the same intuitive operation and extended proprioception as a regular mouth-stick effector but with augmentation of workspace volume and mechanical power. The input is through a specially modified six DOF master robot (a PerForceTM hand-controller) whose joints can be back-driven to apply forces at the user's head. The manipulation tasks in the environment are performed by a six degree-of-freedom slave robot (the Zebra-ZEROTM) with a built-in force sensor. We describe the prototype hardware/software implementation of the system, control system design, safety/disability issues, and initial evaluation tasks.

Contribution of Vision and Proprioception to the Precision of Reaching Movements

Ren and colleagues (2006) found that saccades to visual targets became less accurate when somatosensory information about hand location was added, suggesting that saccades rely mainly on vision. We conducted two kinematic experiments to examine whether or not reaching movements would also show such strong reliance on vision. In Experiment 1, subjects used their dominant right hand to perform reaches, with or without a delay, to an external visual target or to their own left fingertip positioned either by the experimenter or by the participant. Unlike saccades, reaches became more accurate and precise when proprioceptive information was available. In Experiment 2, subjects reached toward external or bodily targets with differing amounts of visual information. Proprioception improved performance only when vision was limited. Our results indicate that reaching movements, unlike saccades, are improved rather than impaired by the addition of somatosensory information.

The effect of weightbearing exercise with low frequency, whole body vibration on lumbosacral proprioception: A pilot study on normal subjects

Patients with low back pain (LBP) often present with impaired proprioception of the lumbopelvic region. For this reason, proprioception training usually forms part of the rehabilitation protocols. New exercise equipment that produces whole body, low frequency vibration (WBV) has been developed to improve muscle function, and reportedly improves proprioception. The aim of this pilot study was to investigate whether weightbearing exercise given in conjunction with WBV would affect lumbosacral position sense in healthy individuals. For this purpose, twenty-five young individuals with no LBP were assigned randomly to an experimental or control group. The experimental group received WBV for five minutes while holding a static, semi-squat position. The control group adopted the same weightbearing position for equal time but received no vibration. A two-dimensional motion analysis system measured the repositioning accuracy of pelvic tilting in standing. The experimental (WBV) group demonstrated a significant improvement in repositioning accuracy over time (mean 0.78 degrees) representing 39% improvement. It was concluded that WBV may induce improvements in lumbosacral repositioning accuracy when combined with a weightbearing exercise. Future studies with WBV should focus on evaluating its effects with different types of exercise, the exercise time needed for optimal outcomes, and the effects on proprioception deficits in LBP patients.

EPiCS:engineering proprioception in computing systems

Modern compute systems continue to evolve towards increasingly complex, heterogeneous and distributed architectures. At the same time, functionality and performance are no longer the only aspects when developing applications for such systems, and additional concerns such as flexibility, power efficiency, resource usage, reliability and cost are becoming increasingly important. This does not only raise the question of how to efficiently develop applications for such systems, but also how to cope with dynamic changes in the application behaviour or the system environment. The EPiCS Project aims to address these aspects through exploring self-awareness and self-expression. Self-awareness allows systems and applications to gather and maintain information about their current state and environment, and reason about their behaviour. Self-expression enables systems to adapt their behaviour autonomously to changing conditions. Innovations in EPiCS are based on systematic integration of research in concepts and foundations, customisable hardware/software platforms and operating systems, and self-aware networking and middleware infrastructure. The developed technologies are validated in three application domains: computational finance, distributed smart cameras and interactive mobile media systems. © 2012 IEEE.

Proprioception in ankle sprain from exercises and bandages

Ankle sprains are the most common injuries in sports, usually causing damage to the lateral ligaments. Recurrence has as usual result permanent instability, and thus loss of proprioception. This fact, together with residual symptoms, is what is known as chronic ankle instability, CAI, or FAI, if it is functional. This problem tries to be solved by improving musculoskeletal stability and proprioception by the application of bandages and performing exercises. The aim of this study has been to review articles (meta-analisis, systematic reviews and revisions) published in 2009-2015 in PubMed, Medline, ENFISPO and BUCea, using keywords such as “sprain instability”, “sprain proprioception”, “chronic ankle instability”. Evidence affirms that there does exist decreased proprioception in patients who suffer from CAI. Rehabilitation exercise regimen is indicated as a treatment because it generates a subjective improvement reported by the patient, and the application of bandages works like a sprain prevention method limiting the range of motion, reducing joint instability and increasing confidence during exercise. As podiatrists we should recommend proprioception exercises to all athletes in a preventive way, and those with CAI or FAI, as a rehabilitation programme, together with the application of bandages. However, further studies should be generated focusing on ways of improving proprioception, and on the exercise patterns that provide the maximum benefit.

O papel da propriocepção no desenvolvimento da força muscular e da flexibilidade

Universidade Estadual de Campinas . Faculdade de Educação Física

Insights into the organization of dorsal spinal cord pathways from an evolutionarily conserved raldh2 intronic enhancer

Comparative studies of the tetrapod raldh2 (aldh1a2) gene, which encodes a retinoic acid (RA) synthesis enzyme, have led to the identification of a dorsal spinal cord enhancer. Enhancer activity is directed dorsally to the roof plate and dorsal-most (dl1) interneurons through predicted Tcf- and Cdx-homeodomain binding sites and is repressed ventrally via predicted Tgif homeobox and ventral Lim-homeodomain binding sites. Raldh2 and Math1/Cath1 expression in mouse and chicken highlights a novel, transient, endogenous Raldh2 expression domain in dl1 interneurons, which give rise to ascending circuits and intraspinal commissural interneurons, suggesting roles for RA in the ontogeny of spinocerebellar and intraspinal proprioceptive circuits. Consistent with expression of raldh2 in the dorsal interneurons of tetrapods, we also found that raldh2 is expressed in dorsal interneurons throughout the agnathan spinal cord, suggesting ancestral roles for RA signaling in the ontogenesis of intraspinal proprioception.

Additional sensory information reduces body sway of individuals with anterior cruciate ligament injury

The purpose of this study was to investigate whether the additional sensory information could improve postural control in individuals with unilateral anterior cruciate ligament (ACL) injury. Twenty-eight individuals with unilateral ACL injury (mean age 23.6, 26 males, 2 females) and 28 healthy young control subjects (mean age 22.1 years, 26 males, 2 females) participated in this study. Postural control was evaluated with subjects single-leg standing on a force platform with eyes closed under two sensory conditions: normal sensory information and light touch to a stationary bar (applied force below 1 N). Three trials of 30 5 were performed in each single-leg stance and in each sensory condition. Mean sway amplitude and predominant frequency of center of pressure were calculated for both anterior-posterior and medial-lateral directions. Individuals with ACL injury showed greater mean sway amplitude than healthy control individuals even though the predominant frequency was similar for both groups. Additional sensory information improved postural control performance in individuals with ACL injury and healthy control, with a greater effect observed for the ACL group. Based on these results, we suggest that reduction in postural control performance in individuals with ACL injury would be due to the reduction of sensory information provided by the ACL, but when sensory information is enhanced, postural control performance improves. These results have implications for novel approaches to improve stability in individuals with ACL injury. (c) 2008 Elsevier Ireland Ltd. All rights reserved.

The effect of musculoskeletal pain on motor activity and control

Aberrant movement patterns and postures are obvious to clinicians managing patients with musculoskeletal pain. However, some changes in motor function that occur in the presence of pain are less apparent. Clinical and basic science investigations have provided evidence of the effects of nociception on aspects of motor function. Both increases and decreases in muscle activity have been shown, along with alterations in neuronal control mechanisms, proprioception, and local muscle morphology. Various models have been proposed in an attempt to provide an explanation for some of these changes. These include the vicious cycle and pain adaptation models. Recent research has seen the emergence of a new model in which patterns of muscle activation and recruitment are altered in the presence of pain (neuromuscular activation model). These changes seem to particularly affect the ability of muscles to perform synergistic functions related to maintaining joint stability and control. These changes are believed to persist into the period of chronicity. This review shows current knowledge of the effect of musculoskeletal pain on the motor system and presents the various proposed models, in addition to other shown effects not covered by these models. The relevance of these models to both acute and chronic pain is considered. It is apparent that people experiencing musculoskeletal pain exhibit complex motor responses that may show some variation with the time course of the disorder. (C) 2001 by the American Pain Society.

The influence of age on weight-bearing joint reposition sense of the knee

Knee joint-position sensitivity has been shown to decline with increasing age, with much of the research reported in the literature investigating this age effect in non-weight-bearing (NWB) conditions. However, little data is available in the more functional position of weight-bearing conditions. The objective of this study was to identify the influence of age on the accuracy and nature of knee joint-position sense (JPS) in both full weight-bearing (FWB) and partial weight-bearing (PWB) conditions and to determine the effect of lower-extremity dominance on knee JPS. Sixty healthy subjects from three age groups (young: 20-35 years old, middle-aged: 40-55 years, and older: 60-75 years) were assessed. Tests were conducted on both the right and left legs to examine the ability of subjects to correctly reproduce knee angles in an active criterion-active repositioning paradigm. Knee angles were measured in degrees using an electromagnetic tracking device, Polhemus 3Space Fastrak, that detected positions of sensors placed on the test limb. Errors in FWB knee joint repositioning did not increase with age, but significant age-related increases in knee joint-repositioning error were found in PWB. It was found that elderly subjects tended to overshoot the criterion angle more often than subjects from the young and middle-aged groups. Subjects in all three age groups performed better in FWB than in PWB. Differences between the stance-dominant (STD) and skill-dominant (SKD) legs did not reach significance. Results demonstrated that for, normal pain-free individuals, there is no age-related decline in knee JPS in FWB, although an age effect does exist in PWB. This outcome challenges the current view that a generalised decline in knee joint proprioception occurs with age. In addition, lower-limb dominance is not a factor in acuity of knee JPS.

Hand positioning sense in children with spina bifida myelomeningocele

This study was undertaken to establish whether children with myelomeningocele have abnormal kinaesthesia of the hands. Twenty-one children with myelomeningocele and 21 control children, aged between six and 12 years, were involved in the study. The level of kinaesthetic awareness in the hands was measured by examining the child's ability to copy hand positions, using visual cueing and kinaesthetic cueing. Both accuracy and speed of copying hand gestures were assessed. Children with spina bifida were significantly less accurate in achieving hand positions than the control group (chi((1))(2) 22.60, p < 0.001), with 73% of the children with spina bifida achieving accurate replications compared with 87% in the control group. Furthermore, children with myelomeningocele were shown to be slower than the controls (F-(1,F-2810) = 15.49, p < 0.001). The impaired kinaesthetic awareness found in this study is considered to be one of the factors behind the poor hand function observed in children with myelomeningocele.