3D visual sensing of human hand for remote operation of a robotic hand

New low cost sensors and open free libraries for 3D image processing are making important advances in robot vision applications possible, such as three-dimensional object recognition, semantic mapping, navigation and localization of robots, human detection and/or gesture recognition for human-machine interaction. In this paper, a novel method for recognizing and tracking the fingers of a human hand is presented. This method is based on point clouds from range images captured by a RGBD sensor. It works in real time and it does not require visual marks, camera calibration or previous knowledge of the environment. Moreover, it works successfully even when multiple objects appear in the scene or when the ambient light is changed. Furthermore, this method was designed to develop a human interface to control domestic or industrial devices, remotely. In this paper, the method was tested by operating a robotic hand. Firstly, the human hand was recognized and the fingers were detected. Secondly, the movement of the fingers was analysed and mapped to be imitated by a robotic hand.

Inclusion of immersive virtual learning environments and visual control systems to support the learning of students with Asperger syndrome

This paper presents the use of immersive virtual reality systems in the educational intervention with Asperger students. The starting points of this study are features of these students' cognitive style that requires an explicit teaching style supported by visual aids and highly structured environments. The proposed immersive virtual reality system, not only to assess the student's behavior and progress, but also is able to adapt itself to the student's specific needs. Additionally, the immersive reality system is equipped with sensors that can determine certain behaviors of the students. This paper determines the possible inclusion of immersive virtual reality as a support tool and learning strategy in these particular students' intervention. With this objective two task protocols have been defined with which the behavior and interaction situations performed by participant students are recorded. The conclusions from this study talks in favor of the inclusion of these virtual immersive environments as a support tool in the educational intervention of Asperger syndrome students as their social competences and executive functions have improved.

RGBD Human-Hand recognition for the Interaction with Robot-Hand

New low cost sensors and the new open free libraries for 3D image processing are permitting to achieve important advances for robot vision applications such as tridimensional object recognition, semantic mapping, navigation and localization of robots, human detection and/or gesture recognition for human-machine interaction. In this paper, a method to recognize the human hand and to track the fingers is proposed. This new method is based on point clouds from range images, RGBD. It does not require visual marks, camera calibration, environment knowledge and complex expensive acquisition systems. Furthermore, this method has been implemented to create a human interface in order to move a robot hand. The human hand is recognized and the movement of the fingers is analyzed. Afterwards, it is imitated from a Barret hand, using communication events programmed from ROS.

Interaction between diurnal variations of intraocular pressure, pachymetry, and corneal response to an air puff: Preliminary evidence

Diurnal changes in corneal geometry, pachymetry, and intraocular pressure (IOP) in a healthy eye were recorded. The deformation response to an air puff was simulated using 3 levels of corneal stiffness. The response was dependent on IOP and pachymetry and not only on the biomechanical properties of the cornea. Similarly, the maximum variability due to the diurnal changes in pachymetry and IOP in the corneal displacement generated by the air puff was found to reach 5%. Therefore, diurnal changes in IOP and corneal thickness were able to induce some variability in the air puff–based corneal deformation response. This potential variability should be considered when the biomechanical properties of the cornea are analyzed with air-puff devices.

3D Visual Data-Driven Spatiotemporal Deformations for Non-Rigid Object Grasping Using Robot Hands

Sensing techniques are important for solving problems of uncertainty inherent to intelligent grasping tasks. The main goal here is to present a visual sensing system based on range imaging technology for robot manipulation of non-rigid objects. Our proposal provides a suitable visual perception system of complex grasping tasks to support a robot controller when other sensor systems, such as tactile and force, are not able to obtain useful data relevant to the grasping manipulation task. In particular, a new visual approach based on RGBD data was implemented to help a robot controller carry out intelligent manipulation tasks with flexible objects. The proposed method supervises the interaction between the grasped object and the robot hand in order to avoid poor contact between the fingertips and an object when there is neither force nor pressure data. This new approach is also used to measure changes to the shape of an object’s surfaces and so allows us to find deformations caused by inappropriate pressure being applied by the hand’s fingers. Test was carried out for grasping tasks involving several flexible household objects with a multi-fingered robot hand working in real time. Our approach generates pulses from the deformation detection method and sends an event message to the robot controller when surface deformation is detected. In comparison with other methods, the obtained results reveal that our visual pipeline does not use deformations models of objects and materials, as well as the approach works well both planar and 3D household objects in real time. In addition, our method does not depend on the pose of the robot hand because the location of the reference system is computed from a recognition process of a pattern located place at the robot forearm. The presented experiments demonstrate that the proposed method accomplishes a good monitoring of grasping task with several objects and different grasping configurations in indoor environments.