Haptic mobility control for teleoperated rovers

Future missions involving human and robotic systems co-resident on the lunar surface may call for rovers to be teleoperated from the safety of pressurized habitats or vehicles. An approach is presented that emphasizes human-level judgment and intuition in the total control of a rover’s mobility actions. This is facilitated through human-robotic haptics interaction. The concept of a haptics cone control surface is presented, which provides a teleoperator with a means to intuitively determine the velocities he/she is commanding to control rover motion. The teleoperator is also provided with real-time, tasks-relevant haptic augmentation indicating suggestive control actions concerning the desired mobility objective. Utility of the approach for teleoperated control of steep terrain traversal is demonstrated in simulation.

Data-based dynamic haptic interaction model with deformable 3D objects

The data-based modeling of the haptic interaction simulation is a growing trend in research. These techniques offer a quick alternative to parametric modeling of the simulation. So far, most of the use of the data-based techniques was applied to static simulations. This paper introduces how to use data-based model in dynamic simulations. This ensures realistic behavior and produce results that are very close to parametric modeling. The results show that a quick and accurate response can be achieved using the proposed methods.

Haptic guidance for microrobotic intracellular injection

The ability for a bio-operator to utilise a haptic device to manipulate a microrobot for intracellular injection offers immense benefits. One significant benefit is for the bio-operator to receive haptic guidance while performing the injection process. In order to address this, this paper investigates the use of haptic virtual fixtures for cell injection and proposes a novel force field virtual fixture. The guidance force felt by the bio-operator is determined by force field analysis within the virtual fixture. The proposed force field virtual fixture assists the bio-operator when performing intracellular injection by limiting the micropipette tip's motion to a conical volume as well as recommending the desired path for optimal injection. A virtual fixture plane is also introduced to prevent the bio-operator from moving the micropipette tip beyond the deposition target inside the cell. Simulation results demonstrate the operation of the guidance system.

A haptic training environment for the heart myoblast cell injection procedure

The heart muscle of a cardiac arrest victim continues to accumulate damage throughout its lifetime. This reduces the heart's ability to pump sufficient oxygen and nutrient blood to meet the body's needs. Medical researchers have shown that direct injection of pre-harvested skeletal myoblast cells into the heart can restore some muscle function [1]. This operative procedure usually necessitates the surgeon to open a patient's chest. The open chest procedure is usually a lengthy process and often extends the recovery time of the patient. Alternatively, a high accuracy surgical aid robotic system can be used to assist the thoracoscopic surgery [2][3]. While the robotic surgical method aids faster patient recovery, a less experienced surgeon can potentially cause damage to surrounding tissue.

This paper presents a study into the development of a virtual haptically-enabled heart myoblast injection simulation environment, which can be used to train new surgeons to get hands on experience with the process. The paper also discusses the development of a generic constraint motion technique for needle insertion. Experiments on human performance measures and efficacy, while interacting with haptic feedback training models, are also presented. The experiment involved 10 operators, with each person repeating the needle insertion and injection 10 times. A notable improvement in the task execution time with the number of repetitions was observed. Operators improved their time by up to 300% compared to their first training attempt for a static heart scenario. Under a dynamic heart motion, operator's performance was slightly lower, with the successful rate of completing the experiment reduced from 84% to 75%.

Haptic microrobotic intracellular injection assistance using virtual fixtures

In manual cell injection the operator relies completely on visual information for task feedback and is subject to extended training times as well as poor success rates and repeatability. From this perspective, enhancing human-in-the-loop intracellular injection through haptic interaction offers significant benefits. This paper outlines two haptic virtual fixtures aiming to assist the human operator while performing cell injection. The first haptic virtual fixture is a parabolic force field designed to assist the operator in guiding the micropipette's tip to a desired penetration point on the cell's surface. The second is a planar virtual fixture which attempts to assist the operator from moving the micropipette's tip beyond the deposition target location inside the cell. Preliminary results demonstrate the operation of the haptically assisted microrobotic cell injection system.

3D particle-based cell modelling for haptic microrobotic cell injection

Introducing haptic interface to conduct microrobotic intracellular injection has many beneficial implications. In particular, the haptic device provides force feedback to the bio-operator's hand. This paper introduces a 3D particle-based model to simulate the deformation of the cell membrane and corresponding cellular forces during microrobotic cell injection. The model is based on the kinematic and dynamic of spring – damper multi particle joints considering visco-elastic fluidic properties. It simulates the indentation force feedback as well as cell visual deformation during the microinjection. The model is verified using experimental data of zebrafish embryo microinjection. The results demonstrate that the developed cell model is capable of estimating zebrafish embryo deformation and force feedback accurately.

A comparative study of supervised learning techniques for data-driven haptic simulation

This paper focuses on the choice of a supervised learning algorithm and possible data preprocessing in the domain of data-driven haptic simulation. This is done through a comparison of the performance of different supervised learning techniques with and without data preprocessing. The simulation of haptic interactions with deformable objects using data-driven methods has emerged as an alternative to parametric methods. The accuracy of the simulation depends on the empirical data and the learning method. Several methods were suggested in the literature and here we provide a comparison between their performance and applicability to this domain. We selected four examples to be compared: singular learning mechanism which is artificial neural networks (ANN), attribute selection followed by ANN learning process, ensemble of multiple learning techniques, and attribute selection followed by the learning ensemble. These methods performance was compared in the domain of simulating multiple interactions with a deformable object with nonlinear material behavior.

User-centered design and evaluation of an interactive visual-haptic-auditory interface : a user study on assembly

Utilizing user-centred system design and evaluation method has become an increasingly important tool to foster better usability in the field of virtual environments (VEs). In recent years, although it is still the norm that designers and developers are concerning the technological advancement and striving for designing impressive multimodal multisensory interfaces, more and more awareness are aroused among the development team that in order to produce usable and useful interfaces, it is essential to have users in mind during design and validate a new design from users' perspective. In this paper, we describe a user study carried out to validate a newly developed haptically enabled virtual training system. By taking consideration of the complexity of individual differences on human performance, adoption and acceptance of haptic and audio-visual I/O devices, we address how well users learn, perform, adapt to and perceive object assembly training. We also explore user experience and interaction with the system, and discuss how multisensory feedback affects user performance, perception and acceptance. At last, we discuss how to better design VEs that enhance users perception, their interaction and motor activity.

Virtual haptic cell model for operator training

Microrobotic cell injection is an area of growing research interest. Typically, operators rely on visual feedback to perceive the microscale environment and are subject to lengthy training times and low success rates. Haptic interaction offers the ability to utilise the operator’s haptic modality and to enhance operator performance. Our earlier work presented a haptically enabled system for assisting the operator with certain aspects of the cell injection task. The system aimed to enhance the operator’s controllability of the micropipette through a logical mapping between the haptic device and microrobot, as well as introducing virtual fixtures for haptic guidance. The system was also designed in such a way that given the availability of appropriate force sensors, haptic display of the cell penetration force is straightforward. This work presents our progress towards a virtual replication of the system, aimed at facilitating offline operator training. It is suggested that operators can use the virtual system to train offline and later transfer their skills to the physical system. In order to achieve the necessary representation of the cell within the virtual system, methods based on a particle-based cell model are utilised. In addition to providing the necessary visual representation, the cell model provides the ability to estimate cell penetration forces and haptically display them to the operator. Two different approaches to achieving the virtual system are discussed.

Driver recognition based on dynamic handgrip pattern on steeling wheel

This paper proposes a novel biometric authentication method based on the recognition of drivers' dynamic handgrip on steering wheel. A pressure sensitive mat mounted on a steering wheel is employed to collect handgrip data exerted by the hands of drivers who intend to start the vehicle. Then, the likelihood-ratio-based classifier is designed to distinguish rightful driver of a car after analyzing their inherent dynamic features of grasping. The experimental results obtained in this study show that mean acceptance rates of 85.4% for the trained subjects and mean rejection rates of 82.65% for the un-trained ones are achieved by the classifier in the two batches of testing. It can be concluded that the driver verification approach based on dynamic handgrip recognition on steering wheel is a promising biometric technology and will be further explored in the near future in smart car design.

Function-based approach to mixed haptic effects rendering

Commonly, surface and solid haptic effects are defined in such a way that they hardly can be rendered together. We propose a method for defining mixed haptic effects including surface, solid, and force fields. These haptic effects can be applied to virtual scenes containing various objects, including polygon meshes, point clouds, impostors, and layered textures, voxel models as well as function-based shapes. Accordingly, we propose a way how to identify location of the haptic tool in such virtual scenes as well as consistently and seamlessly determine haptic effects when the haptic tool moves in the scenes with objects having different sizes, locations, and mutual penetrations. To provide for an efficient and flexible rendering of haptic effects, we propose to concurrently use explicit, implicit and parametric functions, and algorithmic procedures.

3D locomotion biomimetic robot fish with haptic feedback

This thesis developed a biomimetic robot fish and built a novel haptic robot fish system based on the kinematic modelling and three-dimentional computational fluid dynamic (CFD) hydrodynamic analysis. The most important contribution is the successful CFD simulation of the robot fish, supporting users in understanding the hydrodynamic properties around it.