Haptic microrobotic cell injection system

Microrobotic cell injection is the subject of increasing research interest. At present, an operator relies completely on visual information and can be subject to low success rates, poor repeatability, and extended training times. This paper focuses on increasing operator performance during cell injection in two ways. First, our completed haptic cell injection system aims to increase the operator's performance during real-time cell injection. Haptic bilateralism is investigated and a mapping framework provides an intuitive method for manoeuvring the micropipette in a manner similar to handheld needle insertion. Volumetric virtual fixtures are then introduced to haptically assist the operator to penetrate the cell at the desired location. The performance of the volumetric virtual fixtures is also discussed. Second, the haptically enabled cell injection system is replicated as a virtual environment facilitating virtual offline operator training. Virtual operator training utilizes the same mapping framework and haptic virtual fixtures as the physical system allowing the operator to train offline and then directly transfer their skills to real-time cell injection.

Towards an efficient haptic rendering using data-driven modeling

This thesis focuses on the optimisation of haptic rendering of interactions with deformable models. The research demonstrated that data-driven techniques can produce a real-time, accurate and complex simulation experience. Applications include, but not limited to, virtual training, rapid prototyping, virtual presence, and entertainment.

Function-based single and dual point haptic interaction in Cyberworlds

Polygon and point based models dominate virtual reality. These models also affect haptic rendering algorithms, which are often based on collision with polygons. With application to dual point haptic devices for operations like grasping, complex polygon and point based models will make the collision detection procedure slow. This results in the system not able to achieve interactivity for force rendering. To solve this issue, we use mathematical functions to define and implement geometry (curves, surfaces and solid objects), visual appearance (3D colours and geometric textures) and various tangible physical properties (elasticity, friction, viscosity, and force fields). The function definitions are given as analytical formulas (explicit, implicit and parametric), function scripts and procedures. We proposed an algorithm for haptic rendering of virtual scenes including mutually penetrating objects with different sizes and arbitrary location of the observer without a prior knowledge of the scene to be rendered. The algorithm is based on casting multiple haptic rendering rays from the Haptic Interaction Point (HIP), and it builds a stack to keep track on all colliding objects with the HIP. The algorithm uses collision detection based on implicit function representation of the object surfaces. The proposed approach allows us to be flexible when choosing the actual rendering platform, while it can also be easily adopted for dual point haptic collision detection as well as force and torque rendering. The function-defined objects and parts constituting them can be used together with other common definitions of virtual objects such as polygon meshes, point sets, voxel volumes, etc. We implemented an extension of X3D and VRML as well as several standalone application examples to validate the proposed methodology. Experiments show that our concern about fast, accurate rendering as well as compact representation could be fulfilled in various application scenarios and on both single and dual point haptic devices.

Enhancing audio-haptic enabled novel dental training platform performance

Existing haptic and non-haptic dental simulators do not eliminate the problem of hand instability while using the haptic devices for training purpose. This paper reports an audio-haptic dental training platform, which uses a Hand Stability System to reduce the effect of nervousness and hand instability for trainee dental students. Maintaining the ease of implementation, application customizability and the cost factor, the proposed platform increases the training efficiency by enhancing the immersive haptic experience with hand stability. This haptic platform includes multiple angle viewing techniques, audio feedback and session recording for after action review. Trials using this preliminary platform reduced the effect of human nervousness and hand instability due to the customized design.

Wireless haptic rendering for mobile platforms

Computer haptics has so far been performed on a personal computer (PC). Off the shelf haptic devices provide only PC interfaces and software drivers for control and communication. The new wave of high capable tablet PCs and high end smart phones introduced new platforms for haptic applications. The major problem was to communicate wirelessly to provide user convenience and support mobility which is an essential feature for these platforms. In this paper we provide a wireless layered communication protocol and a hardware setup that enables off the shelf haptic devices to communicate wirelessly with a mobile device. The layers in the protocol enable the change of any hardware components without affecting the data flow. However, the adoption of the wireless interface instead of the wired one comes with the price of speed. Haptic refresh loops require a relatively high refresh rate of 1000 Hz compared to graphics loop which require between 30 and 60 only. An interpolation algorithm was demonstrated to compensate the latency and secure a stable user experience. The introduced setup was tested against portable environments and the users could perform similar functionalities to what are available on a wired setup to a PC.

Embedded vision-based autonomous move-to-grasp approach for a mobile manipulator

This paper proposes a vision‐based autonomous move‐to‐grasp approach for a compact mobile manipulator under some low and small environments. The visual information of specified object with a radial symbol and an overhead colour block is extracted from two CMOS cameras in an embedded way. Furthermore, the mobile platform and the postures of the manipulator are adjusted continuously by vision‐based control, which drives the mobile manipulator approaching the object. When the mobile manipulator is sufficiently close to the object, only the manipulator moves to grasp the object based on the incremental movement with its head end centre of the end‐effector conforming to a Bezier curve. The effectiveness of the proposed approach is verified by experiments.

Human performance measures for interactive haptic-audio-visual interfaces

Virtual reality and simulation are becoming increasingly important in modern society and it is essential to improve our understanding of system usability and efficacy from the users’ perspective. This paper introduces a novel evaluation method designed to assess human user capability when undertaking technical and procedural training using virtual training systems. The evaluation method falls under the user-centred design and evaluation paradigm and draws on theories of cognitive, skillbased and affective learning outcomes. The method focuses on user interaction with haptic-audio-visual interfaces and the complexities related to variability in users’ performance, and the adoption and acceptance of the technologies. A large scale user study focusing on object assembly training tasks involving selecting, rotating, releasing, inserting and manipulating 3D objects was performed. The study demonstrated the advantages of the method in obtaining valuable multimodal information for accurate and comprehensive evaluation of virtual training system efficacy. The study investigated how well users learn, perform, adapt to and perceive the virtual training. The results of the study revealed valuable aspects of the design and evaluation of virtual training systems contributing to an improved understanding of more usable virtual training systems.

A framework for visual and haptic collaboration in shared virtual spaces

We propose a framework for visual and haptic collaboration in X3D/VRML shared virtual spaces. In this collaborative framework, two pipelines— visual and haptic—complement each other to provide a simple and efficient solution to problem requiring collaboration in shared virtual spaces on the web. We consider shared objects defined as virtual object with their visual and physical properties rendered synchronously on each client computer. We introduce virtual tools which are shared objects associated with interactive and haptic devices. We implemented the proposed ideas as a server-client framework with a dedicated viewer. We discuss two implementation frameworks based on the strong and thin server concepts.

Automatic deformation transfer for data-driven haptic rendering

This paper addresses a major challenge in datadriven haptic modeling of deformable objects. Data-driven modelling is done for specific objects and is difficult to generalize for nearly isometric objects that have similarities in semantics or topology. This limitation prevents the wide use of the data-driven modeling techniques when compared with parametric methods such as finite element methods. The proposed solution is to incorporate deformation transfer methods when processing similar instances. The contributions of this work are focused on the novel automatic shape correspondence method that overcomes the problems of symmetry and semantics presence requirement. The results shows that the proposed algorithm can efficiently calculate the correspondence and transfer deformations for a range of similar 3D objects.

Multipoint haptic mediator interface for robotic teleoperation

Despite recent advances in artificial intelligence and autonomous robotics, teleoperation can provide distinct benefits in applications requiring real-time human judgement and intuition. However, as robotic systems are increasingly becoming sophisticated and are performing more complex tasks, realizing these benefits requires new approaches to teleoperation. This paper introduces a novel haptic mediator interface for teleoperating mobile robotic platforms that have a variety of manipulators and functions. Identical master-slave bilateral teleoperation of the robotic manipulators is achieved by representing them in virtual reality and by allowing the operator to interact with them using a multipoint haptic device. The operator is also able to command motions to the mobile platform by using a novel haptic interaction metaphor rather than a separate dedicated input device. The presented interaction techniques enable the operator to perform a wide range of control functions and achieve functionality similar to that of conventional teleoperation schemes that use a single haptic interface. The mediator interface is presented, and important considerations such as workspace mapping and scaling are discussed. © 2015 IEEE.

Haptic-based color representation framework

In this paper we propose a novel technique to associate color RGB dimensions into other dimensions that can be explored Haptically. These dimensions will be the force, vibration and geometric representation. The color association between the color information and the other modalities will be called the Haptic-Color Palette. This is of great importance for visually impaired and color blind persons to get access to media in which colors play a great role, such as artwork. The proposed color conversion technique is based on the human perception and interpretations of colors.