Learning Grasp Affordances from Vision

Monimutkaisissa ja muuttuvissa ympäristöissä työskentelevät robotit tarvitsevat kykyä manipuloida ja tarttua esineisiin. Tämä työ tutkii robottitarttumisen ja robottitartuntapis-teiden koneoppimisen aiempaa tutkimusta ja nykytilaa. Nykyaikaiset menetelmät käydään läpi, ja Le:n koneoppimiseen pohjautuva luokitin toteutetaan, koska se tarjoaa parhaan onnistumisprosentin tutkituista menetelmistä ja on muokattavissa sopivaksi käytettävissä olevalle robotille. Toteutettu menetelmä käyttää intensititeettikuvaan ja syvyyskuvaan po-hjautuvia ominaisuuksi luokitellakseen potentiaaliset tartuntapisteet. Tämän toteutuksen tulokset esitellään.

The preparation of reach to grasp movements in adults with Down syndrome

The aim of this study was to determine the extent to which adults with Down syndrome (DS) are able to utilise advance information to prepare reach to grasp movements. The study comprised ten adults with DS; ten children matched to an individual in the group with DS on the basis of their intellectual ability, and twelve adult controls. The participants used their right hand to reach out and grasp illuminated perspex blocks. Four target blocks were positioned on a table surface, two to each side of the midsagittal plane. In the complete precue condition, participants were provided with information specifying the location of the target. In the partial precue condition, participants were given advance information indicating the location of the object relative to the midsagittal plane (left or right). In the null condition, advance information concerning the position of the target object was entirely ambiguous. It was found that both reaction times and movement times were greater for the participants with DS than for the adults without DS. The reaction times exhibited by individuals with DS in the complete precue condition were lower than those observed in the null condition, indicating that they had utilised advance information to prepare their movements. In the group with DS, when advance information specified only the location of the target object relative to the midline, reaction times were equivalent to those obtained when ambiguous information was given. In contrast, the adults without DS exhibited reaction times that were lower in both the complete and partial precue conditions when compared to the null condition. The pattern of results exhibited by the children was similar to that of the adults without DS. The movement times exhibited by all groups were not influenced by the precue condition. In summary, our findings indicate that individuals with DS are able to use advance information if it specifies precisely the location of the target object in order to prepare a reach to grasp movement. The group with DS were unable, however, to obtain the normal advantage of advance information specifying only one dimension of the movement goal (i.e., the position of an object relative to the body midline). (C) 2001 Elsevier Science B.V. All rights reserved.

The effect of obstacle position on reach-to-grasp movements

Numerous everyday tasks require the nervous system to program a prehensile movement towards a target object positioned in a cluttered environment. Adult humans are extremely proficient in avoiding contact with any non-target objects (obstacles) whilst carrying out such movements. A number of recent studies have highlighted the importance of considering the control of reach-to-grasp (prehension) movements in the presence of such obstacles. The current study was constructed with the aim of beginning the task of studying the relative impact on prehension as the position of obstacles is varied within the workspace. The experimental design ensured that the obstacles were positioned within the workspace in locations where they did not interfere physically with the path taken by the hand when no obstacle was present. In all positions, the presence of an obstacle caused the hand to slow down and the maximum grip aperture to decrease. Nonetheless, the effect of the obstacle varied according to its position within the workspace. In the situation where an obstacle was located a small distance to the right of a target object, the obstacle showed a large effect on maximum grip aperture but a relatively small effect on movement time. In contrast, an object positioned in front and to the right of a target object had a large effect on movement speed but a relatively small effect on maximum grip aperture. It was found that the presence of two obstacles caused the system to decrease further the movement speed and maximum grip aperture. The position of the two obstacles dictated the extent to which their presence affected the movement parameters. These results show that the antic ipated likelihood of a collision with potential obstacles affects the planning of movement duration and maximum grip aperture in prehension.

Representation of motor habit in a sequence of repetitive reach and grasp movements performed by macaque monkeys: evidence for a contribution of the dorsolateral prefrontal cortex.

In the context of an autologous cell transplantation study, a unilateral biopsy of cortical tissue was surgically performed from the right dorsolateral prefrontal cortex (dlPFC) in two intact adult macaque monkeys (dlPFC lesioned group), together with the implantation of a chronic chamber providing access to the left motor cortex. Three other monkeys were subjected to the same chronic chamber implantation, but without dlPFC biopsy (control group). All monkeys were initially trained to perform sequential manual dexterity tasks, requiring precision grip. The motor performance and the prehension's sequence (temporal order to grasp pellets from different spatial locations) were analysed for each hand. Following the surgery, transient and moderate deficits of manual dexterity per se occurred in both groups, indicating that they were not due to the dlPFC lesion (most likely related to the recording chamber implantation and/or general anaesthesia/medication). In contrast, changes of motor habit were observed for the sequential order of grasping in the two monkeys with dlPFC lesion only. The changes were more prominent in the monkey subjected to the largest lesion, supporting the notion of a specific effect of the dlPFC lesion on the motor habit of the monkeys. These observations are reminiscent of previous studies using conditional tasks with delay that have proposed a specialization of the dlPFC for visuo-spatial working memory, except that this is in a different context of "free-will", non-conditional manual dexterity task, without a component of working memory.

Effects of unilateral motor cortex lesion on ipsilesional hand's reach and grasp performance in monkeys: relationship with recovery in the contralesional hand.

Manual dexterity, a prerogative of primates, is under the control of the corticospinal (CS) tract. Because 90-95% of CS axons decussate, it is assumed that this control is exerted essentially on the contralateral hand. Consistently, unilateral lesion of the hand representation in the motor cortex is followed by a complete loss of dexterity of the contralesional hand. During the months following lesion, spontaneous recovery of manual dexterity takes place to a highly variable extent across subjects, although largely incomplete. In the present study, we tested the hypothesis that after a significant postlesion period, manual performance in the ipsilesional hand is correlated with the extent of functional recovery in the contralesional hand. To this aim, ten adult macaque monkeys were subjected to permanent unilateral motor cortex lesion. Monkeys' manual performance was assessed for each hand during several months postlesion, using our standard behavioral test (modified Brinkman board task) that provides a quantitative measure of reach and grasp ability. The ipsilesional hand's performance was found to be significantly enhanced over the long term (100-300 days postlesion) in six of ten monkeys, with the six exhibiting the best, though incomplete, recovery of the contralesional hand. There was a statistically significant correlation (r = 0.932; P < 0.001) between performance in the ipsilesional hand after significant postlesion period and the extent of recovery of the contralesional hand. This observation is interpreted in terms of different possible mechanisms of recovery, dependent on the recruitment of motor areas in the lesioned and/or intact hemispheres.

Intelligent task-level grasp mapping for robot control

In the future, robots will enter our everyday lives to help us with various tasks.For a complete integration and cooperation with humans, these robots needto be able to acquire new skills. Sensor capabilities for navigation in real humanenvironments and intelligent interaction with humans are some of the keychallenges.Learning by demonstration systems focus on the problem of human robotinteraction, and let the human teach the robot by demonstrating the task usinghis own hands. In this thesis, we present a solution to a subproblem within thelearning by demonstration field, namely human-robot grasp mapping. Robotgrasping of objects in a home or office environment is challenging problem.Programming by demonstration systems, can give important skills for aidingthe robot in the grasping task.The thesis presents two techniques for human-robot grasp mapping, directrobot imitation from human demonstrator and intelligent grasp imitation. Inintelligent grasp mapping, the robot takes the size and shape of the object intoconsideration, while for direct mapping, only the pose of the human hand isavailable.These are evaluated in a simulated environment on several robot platforms.The results show that knowing the object shape and size for a grasping taskimproves the robot precision and performance

Technology educational affordance: Bridging the gap between patterns of interaction and technology usage

Peer-reviewed

Interaction between affordance and handedness recognition: a chronometric study

The visualization of tools and manipulable objects activates motor-related areas in the cortex, facilitating possible actions toward them. This pattern of activity may underlie the phenomenon of object affordance. Some cortical motor neurons are also covertly activated during the recognition of body parts such as hands. One hypothesis is that different subpopulations of motor neurons in the frontal cortex are activated in each motor program; for example, canonical neurons in the premotor cortex are responsible for the affordance of visual objects, while mirror neurons support motor imagery triggered during handedness recognition. However, the question remains whether these subpopulations work independently. This hypothesis can be tested with a manual reaction time (MRT) task with a priming paradigm to evaluate whether the view of a manipulable object interferes with the motor imagery of the subject's hand. The MRT provides a measure of the course of information processing in the brain and allows indirect evaluation of cognitive processes. Our results suggest that canonical and mirror neurons work together to create a motor plan involving hand movements to facilitate successful object manipulation.

From inference to affordance : the problem of visual depth-perception in the optical writings of Descartes, Berkeley, and Gibson

This thesis explores the debate and issues regarding the status of visual ;,iferellces in the optical writings of Rene Descartes, George Berkeley and James 1. Gibson. It gathers arguments from across their works and synthesizes an account of visual depthperception that accurately reflects the larger, metaphysical implications of their philosophical theories. Chapters 1 and 2 address the Cartesian and Berkelean theories of depth-perception, respectively. For Descartes and Berkeley the debate can be put in the following way: How is it possible that we experience objects as appearing outside of us, at various distances, if objects appear inside of us, in the representations of the individual's mind? Thus, the Descartes-Berkeley component of the debate takes place exclusively within a representationalist setting. Representational theories of depthperception are rooted in the scientific discovery that objects project a merely twodimensional patchwork of forms on the retina. I call this the "flat image" problem. This poses the problem of depth in terms of a difference between two- and three-dimensional orders (i.e., a gap to be bridged by one inferential procedure or another). Chapter 3 addresses Gibson's ecological response to the debate. Gibson argues that the perceiver cannot be flattened out into a passive, two-dimensional sensory surface. Perception is possible precisely because the body and the environment already have depth. Accordingly, the problem cannot be reduced to a gap between two- and threedimensional givens, a gap crossed with a projective geometry. The crucial difference is not one of a dimensional degree. Chapter 3 explores this theme and attempts to excavate the empirical and philosophical suppositions that lead Descartes and Berkeley to their respective theories of indirect perception. Gibson argues that the notion of visual inference, which is necessary to substantiate representational theories of indirect perception, is highly problematic. To elucidate this point, the thesis steps into the representationalist tradition, in order to show that problems that arise within it demand a tum toward Gibson's information-based doctrine of ecological specificity (which is to say, the theory of direct perception). Chapter 3 concludes with a careful examination of Gibsonian affordallces as the sole objects of direct perceptual experience. The final section provides an account of affordances that locates the moving, perceiving body at the heart of the experience of depth; an experience which emerges in the dynamical structures that cross the body and the world.

Neural correlates of affordance competition in dorsal premotor cortex

Le travail présenté dans cette thèse porte sur le rôle du cortex prémoteur dorsal (PMd) au sujet de la prise de décision (sélection d’une action parmis nombreux choix) et l'orientation visuelle des mouvements du bras. L’ouvrage décrit des expériences électrophysiologiques chez le singe éveillé (Macaca mulatta) permettant d’adresser une fraction importante des prédictions proposées par l'hypothèse des affordances concurrentes (Cisek, 2006; Cisek, 2007a). Cette hypothèse suggère que le choix de toute action est l’issue d'une concurrence entre les représentations internes des exigences et des atouts de chacune des options présentées (affordances; Gibson, 1979). Un intérêt particulier est donné au traitement de l'information spatiale et la valeur des options (expected value, EV) dans la prise de décisions. La première étude (article 1) explore la façon dont PMd reflète ces deux paramètres dans la période délai ainsi que de leur intéraction. La deuxième étude (article 2) explore le mécanisme de décision de façon plus détaillée et étend les résultats au cortex prémoteur ventral (PMv). Cette étude porte également sur la représentation spatiale et l’EV dans une perspective d'apprentissage. Dans un environnement nouveau les paramètres spatiaux des actions semblent être présents en tout temps dans PMd, malgré que la représentation de l’EV apparaît uniquement lorsque les animaux commencent à prendre des décisions éclairées au sujet de la valeur des options disponibles. La troisième étude (article 3) explore la façon dont PMd est impliqué aux “changements d'esprit“ dans un procès de décision. Cette étude décrit comment la sélection d’une action est mise à jour à la suite d'une instruction de mouvement (GO signal). I II Les résultats principaux des études sont reproduits par un modèle computationnel (Cisek, 2006) suggérant que la prise de décision entre plusieurs actions alternatives peux se faire par voie d’un mécanisme de concurrence (biased competition) qui aurait lieu dans la même région qui spécifie les actions.

Plataformas virtuales e innovaci??n docente universitaria : affordance de una gu??a de trabajo aut??nomo en Antropolog??a para trabajar competencias de intervenci??n en contextos de diversidad

Resumen basado en el de la publicaci??n

Reach & grasp therapy: Design and control of a 9-DOF robotic neuro-rehabilitation system

This paper presents the Gentle/G integrated system for reach & grasp therapy retraining following brain injury. The design, control and integration of an experimental grasp assistance unit is described for use in robot assisted stroke rehabilitation. The grasp assist unit is intended to work with the hardware and software of the Gentle/S robot although the hardware could be adapted to other rehabilitation applications. When used with the Gentle/S robot a total of 6 active and 3 passive degrees of freedom are available to provide active, active assist or passive grasp retraining in combination with reaching movements in a reach-grasp-transfer-release sequence.