3D particle-based cell modelling for haptic microrobotic cell injection

Introducing haptic interface to conduct microrobotic intracellular injection has many beneficial implications. In particular, the haptic device provides force feedback to the bio-operator's hand. This paper introduces a 3D particle-based model to simulate the deformation of the cell membrane and corresponding cellular forces during microrobotic cell injection. The model is based on the kinematic and dynamic of spring – damper multi particle joints considering visco-elastic fluidic properties. It simulates the indentation force feedback as well as cell visual deformation during the microinjection. The model is verified using experimental data of zebrafish embryo microinjection. The results demonstrate that the developed cell model is capable of estimating zebrafish embryo deformation and force feedback accurately.

A comparative study of supervised learning techniques for data-driven haptic simulation

This paper focuses on the choice of a supervised learning algorithm and possible data preprocessing in the domain of data-driven haptic simulation. This is done through a comparison of the performance of different supervised learning techniques with and without data preprocessing. The simulation of haptic interactions with deformable objects using data-driven methods has emerged as an alternative to parametric methods. The accuracy of the simulation depends on the empirical data and the learning method. Several methods were suggested in the literature and here we provide a comparison between their performance and applicability to this domain. We selected four examples to be compared: singular learning mechanism which is artificial neural networks (ANN), attribute selection followed by ANN learning process, ensemble of multiple learning techniques, and attribute selection followed by the learning ensemble. These methods performance was compared in the domain of simulating multiple interactions with a deformable object with nonlinear material behavior.

Enabling multi-point haptic grasping in virtual environments

Haptic interaction has received increasing research interest in recent years. Currently, most commercially available haptic devices provide the user with a single point of interaction. Multi-point haptic devices present a logical progression in device design and enable the operator to experience a far wider range of haptic interactions, particularly the ability to grasp via multiple fingers. This is highly desirable for various haptically enabled applications including virtual training, telesurgery and telemanipulation. This paper presents a gripper attachment which utilises two low-cost commercially available haptic devices to facilitate multi-point haptic grasping. It provides the ability to render forces to the user's fingers independently and using Phantom Omni haptic devices offers several benefits over more complex approaches such as low-cost, reliability, and ease of programming. The workspace of the gripper attachment is considered and in order to haptically render the desired forces to the user's fingers, kinematic analysis is discussed and necessary formulations presented. The integrated multi-point haptic platform is presented and exploration of a virtual environment using CHAI 3D is demonstrated.

User-centered design and evaluation of an interactive visual-haptic-auditory interface : a user study on assembly

Utilizing user-centred system design and evaluation method has become an increasingly important tool to foster better usability in the field of virtual environments (VEs). In recent years, although it is still the norm that designers and developers are concerning the technological advancement and striving for designing impressive multimodal multisensory interfaces, more and more awareness are aroused among the development team that in order to produce usable and useful interfaces, it is essential to have users in mind during design and validate a new design from users' perspective. In this paper, we describe a user study carried out to validate a newly developed haptically enabled virtual training system. By taking consideration of the complexity of individual differences on human performance, adoption and acceptance of haptic and audio-visual I/O devices, we address how well users learn, perform, adapt to and perceive object assembly training. We also explore user experience and interaction with the system, and discuss how multisensory feedback affects user performance, perception and acceptance. At last, we discuss how to better design VEs that enhance users perception, their interaction and motor activity.

Virtual haptic cell model for operator training

Microrobotic cell injection is an area of growing research interest. Typically, operators rely on visual feedback to perceive the microscale environment and are subject to lengthy training times and low success rates. Haptic interaction offers the ability to utilise the operator’s haptic modality and to enhance operator performance. Our earlier work presented a haptically enabled system for assisting the operator with certain aspects of the cell injection task. The system aimed to enhance the operator’s controllability of the micropipette through a logical mapping between the haptic device and microrobot, as well as introducing virtual fixtures for haptic guidance. The system was also designed in such a way that given the availability of appropriate force sensors, haptic display of the cell penetration force is straightforward. This work presents our progress towards a virtual replication of the system, aimed at facilitating offline operator training. It is suggested that operators can use the virtual system to train offline and later transfer their skills to the physical system. In order to achieve the necessary representation of the cell within the virtual system, methods based on a particle-based cell model are utilised. In addition to providing the necessary visual representation, the cell model provides the ability to estimate cell penetration forces and haptically display them to the operator. Two different approaches to achieving the virtual system are discussed.

A haptic platform to enable fingertip grasping and manipulation

Haptic technologies allow human users to haptically interact with virtual environments. Haptics has been employed in many application domains including operator training, virtual exploration and teleoperation. Currently, most commercially available haptic devices focus on a single point of haptic interaction. While single-point haptics have been successfully employed in many applications, they remain limited to particular types of haptic interaction. Multi-point haptic devices are a logical progression and facilitate a far wider range of interactions including object grasping, multi-finger object manipulation and size discrimination. The ability to effectively achieve such interactions offers significant benefits for many applications including virtual training, telesurgery and telemanipulation. In such applications, the ability to use multi-point haptic interactions can provide far more effective user interaction as well improved perception of the virtual environment.

A multi-point haptic platform for grasping and manipulating virtual objects

This work presents a multi-point haptic platform that employs two Phantom Omni haptic devices. A gripper attachment connects to both devices and enables multi-point haptic grasping in virtual environments. In contrast to more complex approaches, this setup benefits from low-cost, reliability, and ease of programming while being capable of independently rendering forces to each of the user’s fingertips. The ability to grasp with multiple points potentially lends itself to applications such as virtual training, telesurgery and telemanipulation.

Function-based approach to mixed haptic effects rendering

Commonly, surface and solid haptic effects are defined in such a way that they hardly can be rendered together. We propose a method for defining mixed haptic effects including surface, solid, and force fields. These haptic effects can be applied to virtual scenes containing various objects, including polygon meshes, point clouds, impostors, and layered textures, voxel models as well as function-based shapes. Accordingly, we propose a way how to identify location of the haptic tool in such virtual scenes as well as consistently and seamlessly determine haptic effects when the haptic tool moves in the scenes with objects having different sizes, locations, and mutual penetrations. To provide for an efficient and flexible rendering of haptic effects, we propose to concurrently use explicit, implicit and parametric functions, and algorithmic procedures.

3D locomotion biomimetic robot fish with haptic feedback

This thesis developed a biomimetic robot fish and built a novel haptic robot fish system based on the kinematic modelling and three-dimentional computational fluid dynamic (CFD) hydrodynamic analysis. The most important contribution is the successful CFD simulation of the robot fish, supporting users in understanding the hydrodynamic properties around it.

Haptic microrobotic cell injection system

Microrobotic cell injection is the subject of increasing research interest. At present, an operator relies completely on visual information and can be subject to low success rates, poor repeatability, and extended training times. This paper focuses on increasing operator performance during cell injection in two ways. First, our completed haptic cell injection system aims to increase the operator's performance during real-time cell injection. Haptic bilateralism is investigated and a mapping framework provides an intuitive method for manoeuvring the micropipette in a manner similar to handheld needle insertion. Volumetric virtual fixtures are then introduced to haptically assist the operator to penetrate the cell at the desired location. The performance of the volumetric virtual fixtures is also discussed. Second, the haptically enabled cell injection system is replicated as a virtual environment facilitating virtual offline operator training. Virtual operator training utilizes the same mapping framework and haptic virtual fixtures as the physical system allowing the operator to train offline and then directly transfer their skills to real-time cell injection.

Towards an efficient haptic rendering using data-driven modeling

This thesis focuses on the optimisation of haptic rendering of interactions with deformable models. The research demonstrated that data-driven techniques can produce a real-time, accurate and complex simulation experience. Applications include, but not limited to, virtual training, rapid prototyping, virtual presence, and entertainment.