Reactive reaching and grasping on a humanoid: Towards closing the action-perception loop on the iCub

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

Extending visual perception with haptic exploration for improved scene understanding

Scene understanding has been investigated from a mainly visual information point of view. Recently depth has been provided an extra wealth of information, allowing more geometric knowledge to fuse into scene understanding. Yet to form a holistic view, especially in robotic applications, one can create even more data by interacting with the world. In fact humans, when growing up, seem to heavily investigate the world around them by haptic exploration. We show an application of haptic exploration on a humanoid robot in cooperation with a learning method for object segmentation. The actions performed consecutively improve the segmentation of objects in the scene.

Rendering stiffer walls: a hybrid haptic system using continuous and discrete time feedback

Instability in conventional haptic rendering destroys the perception of rigid objects in virtual environments. Inherent limitations in the conventional haptic loop restrict the maximum stiffness that can be rendered. In this paper we present a method to render virtual walls that are much stiffer than those achieved by conventional techniques. By removing the conventional digital haptic loop and replacing it with a part-continuous and part-discrete time hybrid haptic loop, we were able to render stiffer walls. The control loop is implemented as a combinational logic circuit on an field-programmable gate array. We compared the performance of the conventional haptic loop and our hybrid haptic loop on the same haptic device, and present mathematical analysis to show the limit of stability of our device. Our hybrid method removes the computer-intensive haptic loop from the CPU-this can free a significant amount of resources that can be used for other purposes such as graphical rendering and physics modeling. It is our hope that, in the future, similar designs will lead to a haptics processing unit (HPU).

Haptic Feedback for Injecting Biological Cells using Miniature Compliant Mechanisms

We present a real-time haptics-aided injection technique for biological cells using miniature compliant mechanisms. Our system consists of a haptic robot operated by a human hand, an XYZ stage for micro-positioning, a camera for image capture, and a polydimethylsiloxane (PDMS) miniature compliant device that serves the dual purpose of an injecting tool and a force-sensor. In contrast to existing haptics-based micromanipulation techniques where an external force sensor is used, we use visually captured displacements of the compliant mechanism to compute the applied and reaction forces. The human hand can feel the magnified manipulation force through the haptic device in real-time while the motion of the human hand is replicated on the mechanism side. The images are captured using a camera at the rate of 30 frames per second for extracting the displacement data. This is used to compute the forces at the rate of 30 Hz. The force computed in this manner is sent at the rate of 1000 Hz to ensure stable haptic interaction. The haptic cell-manipulation system was tested by injecting into a zebrafish egg cell after validating the technique at a size larger than that of the cell.

Issues in Achievement of Haptic Realism

This paper deals with haptic realism related to Kinematic capabilities of the devices used in manipulation of virtual objects in virtual assembly environments and its effect on achieving haptic realism. Haptic realism implies realistic touch sensation. In virtual world all the operations are to be performed in the same way and with same level of accuracy as in the real world .In order to achieve realism there should be a complete mapping of real and virtual world dimensions. Experiments are conducted to know the kinematic capabilities of the device by comparing the dimensions of the object in the real and virtual world. Registered dimensions in the virtual world are found to be approximately 1.5 times that of the real world. Dimensional variations observed were discrepancy due to exoskeleton and discrepancy due to real and virtual hands. Experiments are conducted to know the discrepancy due to exoskeleton and this discrepancy can be taken care of by either at the hardware or software level. A Mathematical model is proposed to know the discrepancy between real and virtual hands. This could not give a fixed value and can not be taken care of by calibration. Experiments are conducted to figure out how much compensation can be given to achieve haptic realism.

Simulating grasping behavior of people using the relational description scheme

This paper presents an enhanced relational description for the prescription of the grasp requirement and evolution of the posture of a digital human hand towards satisfaction of this requirement. Precise relational description needs anatomical segmentation of the hand geometry into palmar, dorsal and lateral patches using the palm-plane and joint locations information, and operational segmentation of the object geometry into pull,push and lateral patches with due consideration to the effect of friction. Relational description identifies appropriate patches for a desired grasp condition. Satisfaction of this requirement occurs in two discrete stages,namely,contact establishment and post-contact force exertion for object capturing. Contact establishment occurs in four potentially overlapping phases,namely,re-orientation,transfer,pre- shaping,and closing-in. The novel h and re-orientation phase,enables the palm to face the object in a task sequence scenario, transfer takes the wrist to the ball park ; pre-shaping and close-in finally achieves the contact. In this paper, an anatomically pertinent closed-form formulation is presented for the closing-in phase for identification of the point of contact on the patches ,prescribed by the relational description. Since mere contact does not ensure grasp and slip phenomenon at the point of contact on application of force is a common occurrence, the effect of slip in presence of friction has been studied for 2D and 3D object grasping endeavours and a computational generation of the slip locus is presented.A general slip locus is found to be a non-linear curve even on planar faces.Two varieties of slip phenomena,namely,stabilizing and non-stabilizing slips, and their local characteristics have been identified.Study of the evolution of this slip characteristic over the slip locus exhibited diverse grasping behaviour possibilities. Thus, the relational description paradigm not only makes the requirement specification easy and meaningful but also enables high fidelity hand object interaction studies possible.