Selective visuo-haptic rendering of heterogeneous objects in 'parallel universes'

 Haptic rendering of complex models is usually prohibitive due to its much higher update rate requirement compared to visual rendering. Previous works have tried to solve this issue by introducing local simulation or multi-rate simulation for the two pipelines. Although these works have improved the capacity of haptic rendering pipeline, they did not take into consideration the situation of heterogeneous objects in one scenario, where rigid objects and deformable objects coexist in one scenario and close to each other. In this paper, we propose a novel idea to support interactive visuo-haptic rendering of complex heterogeneous models. The idea incorporates different collision detection and response algorithms and have them seamlessly switched on and off on the fly, as the HIP travels in the scenario. The selection of rendered models is based on the hypothesis of “parallel universes”, where the transition of rendering one group of models to another is totally transparent to users. To facilitate this idea, we proposed a procedure to convert the traditional single universe scenario into a “multiverse” scenario, where the original models are grouped and split into each parallel universe, depending on the scenario rendering requirement rather than just locality. We also proposed to add simplified visual objects as background avatars in each parallel universe to visually maintain the original scenario while not overly increase the scenario complexity. We tested the proposed idea in a haptically-enabled needle thoracostomy training environment and the result demonstrates that our idea is able to substantially accelerate visuo-haptic rendering with complex heterogeneous scenario objects.

Robotic grasping and manipulation controller framework: architecture redevelopment

This paper details the further improvements obtained by redesigning a previously offered Manipulation Controller Framework to provide support to an innovative, friction-based object slippage detection strategy employed by the robotic object manipulator. This upgraded Manipulation Controller Framework includes improved slippage detection functionality and a streamlined architecture designed to improve controller robustness, reliability and speed. Improvements include enhancements to object slippage detection strategy, the removal of the decision making module and integration of its functionality into the Motion Planner, and the stream-lining of the Motion Planner to improve its effectiveness. It is anticipated that this work will be useful to researchers developing integrated robot controller architectures and slippage control.

Haptic virtual reality training environment for micro-robotic cell injection

Micro-robotic cell injection is typically performed manually by a trainedbio-operator, and success rates are often low. To enhance bio-operator performance during real-time cell injection, our earlier work introduced a haptically-enabled micro-robotic cell injection system. The system employed haptic virtual fixtures to provide haptic guidance according to articular performance metrics. This paper extends the work by replicating the system within a virtual reality (VR) environment for bio-operator training. Using the virtual environment, the bio-operator is able to control the virtual injection process in the same way they would with the physical haptic micro-robotic cell injection system, while benefiting from the enhanced visualisation capabilities offered by the 3D VR environment. The system is achieved using cost-effective components offering training at much lower cost than using the physical system.

Haptic technology for micro-robotic cell injection training systems — a review

Currently, the micro-robotic cell injection procedure is performed manually by expert human bio-operators. In order to be proficient at the task, lengthy and expensive dedicated training is required. As such, effective specialized training systems for this procedure can prove highly beneficial. This paper presents a comprehensive review of haptic technology relevant to cell injection training and discusses the feasibility of developing such training systems, providing researchers with an inclusive resource enabling the application of the presented approaches, or extension and advancement of the work. A brief explanation of cell injection and the challenges associated with the procedure are first presented. Important skills, such as accuracy, trajectory, speed and applied force, which need to be mastered by the bio-operator in order to achieve successful injection, are then discussed. Then an overview of various types of haptic feedback, devices and approaches is presented. This is followed by discussion on the approaches to cell modeling. Discussion of the application of haptics to skills training across various fields and haptically-enabled virtual training systems evaluation are then presented. Finally, given the findings of the review, this paper concludes that a haptically-enabled virtual cell injection training system is feasible and recommendations are made to developers of such systems.

Applying inverse just-noticeable-differences of velocity to position data for haptic data reduction

ust-Noticeable-Differences (JND) as a dead-band in perceptual analysis has been widely used for more than a decade. This technique has been employed for data reduction in hap tic data transmission systems by several researchers. In fact, researchers use two different JND coefficients that are JNDV and JNDF for velocity and force data respectively. For position data, they usually rely on the resolution of hap tic display device to omit data that are unperceivable to human. In this paper, pruning undesirable position data that are produced by the vibration of the device or subject and/or noise in transmission line is addressed. It is shown that using inverse JNDV for position data can prune undesirable position data. Comparison of the results of the proposed method in this paper with several well known filters and some available methods proposed by other researchers is performed. It is shown that combination of JNDV could provide lower error with desirable curve smoothness, and as little as possible computation effort and complexity. It also has been shown that this method reduces much more data rather than using forward-JNDV.

Countering Improvised explosive devices through a multi-point haptic teleoperation system

Improvised Explosive Devices (IEDs) are reported as the number one cause of injury and death for allied troops in the current theater of operation. Current stand-off technologies for Counter IED (CIED) tasks rely on robotic platforms that have not improved in capability over the past decade to combat the ever increasing threat of IEDs. While they provide operational capability, the effectiveness of these platforms is limited. This is because they primarily utilise video and audio feedback, and require extensive training and specialist operators. Recent operational experience has demonstrated the need for robotic systems that are highly capable, yet easily operable for high fidelity manipulation. Force feedback provides an operator with more intuitive control of a robotic system. This sense of touch allows an operator to obtain a sense of feel from a stand-off location of what the robot touches or grasps through a human-robot interface. This paper reports the design and development of a Haptically-Enabled Counter IED robotic system that was funded by the Australian Defence Force. The presented work focuses on the design methodology for the system, and provides the results of the manipulator analysis and trial outcomes.

Haptic Stabilization of Posture in Adults With Intellectual Disabilities Using a Nonrigid Tool

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Structure learning of graphical models for task-oriented robot grasping

In the collective imaginaries a robot is a human like machine as any androids in science fiction. However the type of robots that you will encounter most frequently are machinery that do work that is too dangerous, boring or onerous. Most of the robots in the world are of this type. They can be found in auto, medical, manufacturing and space industries. Therefore a robot is a system that contains sensors, control systems, manipulators, power supplies and software all working together to perform a task. The development and use of such a system is an active area of research and one of the main problems is the development of interaction skills with the surrounding environment, which include the ability to grasp objects. To perform this task the robot needs to sense the environment and acquire the object informations, physical attributes that may influence a grasp. Humans can solve this grasping problem easily due to their past experiences, that is why many researchers are approaching it from a machine learning perspective finding grasp of an object using information of already known objects. But humans can select the best grasp amongst a vast repertoire not only considering the physical attributes of the object to grasp but even to obtain a certain effect. This is why in our case the study in the area of robot manipulation is focused on grasping and integrating symbolic tasks with data gained through sensors. The learning model is based on Bayesian Network to encode the statistical dependencies between the data collected by the sensors and the symbolic task. This data representation has several advantages. It allows to take into account the uncertainty of the real world, allowing to deal with sensor noise, encodes notion of causality and provides an unified network for learning. Since the network is actually implemented and based on the human expert knowledge, it is very interesting to implement an automated method to learn the structure as in the future more tasks and object features can be introduced and a complex network design based only on human expert knowledge can become unreliable. Since structure learning algorithms presents some weaknesses, the goal of this thesis is to analyze real data used in the network modeled by the human expert, implement a feasible structure learning approach and compare the results with the network designed by the expert in order to possibly enhance it.

Haptics in forensics: the possibilities and advantages in using the haptic device for reconstruction approaches in forensic science

Non-invasive documentation methods such as surface scanning and radiological imaging are gaining in importance in the forensic field. These three-dimensional technologies provide digital 3D data, which are processed and handled in the computer. However, the sense of touch gets lost using the virtual approach. The haptic device enables the use of the sense of touch to handle and feel digital 3D data. The multifunctional application of a haptic device for forensic approaches is evaluated and illustrated in three different cases: the representation of bone fractures of the lower extremities, by traffic accidents, in a non-invasive manner; the comparison of bone injuries with the presumed injury-inflicting instrument; and in a gunshot case, the identification of the gun by the muzzle imprint, and the reconstruction of the holding position of the gun. The 3D models of the bones are generated from the Computed Tomography (CT) images. The 3D models of the exterior injuries, the injury-inflicting tools and the bone injuries, where a higher resolution is necessary, are created by the optical surface scan. The haptic device is used in combination with the software FreeForm Modelling Plus for touching the surface of the 3D models to feel the minute injuries and the surface of tools, to reposition displaced bone parts and to compare an injury-causing instrument with an injury. The repositioning of 3D models in a reconstruction is easier, faster and more precisely executed by means of using the sense of touch and with the user-friendly movement in the 3D space. For representation purposes, the fracture lines of bones are coloured. This work demonstrates that the haptic device is a suitable and efficient application in forensic science. The haptic device offers a new way in the handling of digital data in the virtual 3D space.