Towards haptic microrobotic intracellular injection

This paper proposes a system providing the operator with an intuitive method for controlling a micromanipulator during intracellular injection. A low-cost haptic device is utilised and 3D position-to-position kinematic mapping allows the operator to control the micropipette using a similar method to handheld needle insertion. The workspaces of the haptic device and micromanipulator are analysed and the importance of appropriate scaling to positioning resolution and tracking performance is investigated. The control issues integral to achieving adequate control of the micromanipulator using the Phantom Omni haptic device are addressed. Aside from offering an intuitive method for controlling the micropipette, this work lays the foundation for real-time haptic assistance in the cell injection task.

Haptic guidance for microrobotic intracellular injection

The ability for a bio-operator to utilise a haptic device to manipulate a microrobot for intracellular injection offers immense benefits. One significant benefit is for the bio-operator to receive haptic guidance while performing the injection process. In order to address this, this paper investigates the use of haptic virtual fixtures for cell injection and proposes a novel force field virtual fixture. The guidance force felt by the bio-operator is determined by force field analysis within the virtual fixture. The proposed force field virtual fixture assists the bio-operator when performing intracellular injection by limiting the micropipette tip's motion to a conical volume as well as recommending the desired path for optimal injection. A virtual fixture plane is also introduced to prevent the bio-operator from moving the micropipette tip beyond the deposition target inside the cell. Simulation results demonstrate the operation of the guidance system.

A haptic training environment for the heart myoblast cell injection procedure

The heart muscle of a cardiac arrest victim continues to accumulate damage throughout its lifetime. This reduces the heart's ability to pump sufficient oxygen and nutrient blood to meet the body's needs. Medical researchers have shown that direct injection of pre-harvested skeletal myoblast cells into the heart can restore some muscle function [1]. This operative procedure usually necessitates the surgeon to open a patient's chest. The open chest procedure is usually a lengthy process and often extends the recovery time of the patient. Alternatively, a high accuracy surgical aid robotic system can be used to assist the thoracoscopic surgery [2][3]. While the robotic surgical method aids faster patient recovery, a less experienced surgeon can potentially cause damage to surrounding tissue.

This paper presents a study into the development of a virtual haptically-enabled heart myoblast injection simulation environment, which can be used to train new surgeons to get hands on experience with the process. The paper also discusses the development of a generic constraint motion technique for needle insertion. Experiments on human performance measures and efficacy, while interacting with haptic feedback training models, are also presented. The experiment involved 10 operators, with each person repeating the needle insertion and injection 10 times. A notable improvement in the task execution time with the number of repetitions was observed. Operators improved their time by up to 300% compared to their first training attempt for a static heart scenario. Under a dynamic heart motion, operator's performance was slightly lower, with the successful rate of completing the experiment reduced from 84% to 75%.

A simulation-based control interface layer for a high-fidelity anthropomorphic training simulator

Deakin University’s futuristic Universal Motion simulator will overcome the limitations of current motion simulator platforms by employing an anthropomorphic robot arm to provide the motion fidelity necessary to exploit the potential of modern simulation environments. Full motion simulators frequently utilize Stewart platforms to mimic the movement of vehicles during simulation. However, due to the limited motion range and dexterity of such systems, and their inability to convey realistic accelerations, they are unable to represent accurate motion characteristics. The Universal Motion Simulation aims to close the gap between the limitations of the current motion technology and real world, by introducing a flexible, modular, high-fidelity motion system that can be used for a variety of immersive training applications. The modular nature of the design allows interchangeable and configurable simulation pods to be attached to the end effectors.

A simple motion heuristic for controlling mobile sinks in delay tolerant sensor networks

Optimising energy consumption in wireless sensor networks is of dominant importance. Sink mobility is introduced to deal with this problem by approaching the sensor nodes and collecting their data buffers using the less energy demanding single-hop communication. The sink route is very crucial for the data collection operation performed in the network especially when the collection requests generated by the sensors are revealed dynamically to the sink and not known ahead. This paper presents a practical motion heuristic for constructing the sink route based on the dynamic arrival of the collection requests. Three control schemes are proposed for coordinating the interaction of multiple mobile sinks collectively performing the data collection in the network. The main objective is maximising the data collected by each mobile sink while minimising the sleeping time of each sensor awaiting the collection service. Simulation results show the performance of the mobile sinks under the proposed control schemes and the impact of the motion heuristic on the sensors' sleeping time in the network.

Insect based navigation and its applications to the autonomous control of mobile robots

Giving robots the ability to autonomously move around in various real-world environments has been a major goal of AI (artificial intelligence) for quite some time. To this end it is vital for robots to be able to perceive their surroundings in 3D; they must be able to estimate the range of obstacles in their path.

Animals navigate through various uncontrolled environments with seemingly little effort. Flying insects, especially, are quite adept at manoeuvring in complex, unpredictable and possibly hostile and hazardous environments.

In this paper it is shown that very simple motion cues, inspired by the visual navigation of flying insects, can be used to provide a mobile robot with the ability to successfully traverse a corridor environment. Equipping an autonomous mobile robot with the ability to successfully navigate real-word environments (in real-time) constitutes a major challenge for AI and robotics. It is in this area that insect based navigation has something to offer.

Insect inspired behaviours for the autonomous control of mobile robots

Animals navigate through various uncontrolled environments with seemingly little effort. Flying insects, especially, are quite adept at manoeuvring in complex, unpredictable and possibly hostile environments. Through both simulation and real-world experiments, we demonstrate the feasibility of equipping a mobile robot with the ability to navigate a corridor environment, in real time, using principles based on insect-based visual guidance. In particular we have used the bees’ navigational strategy of measuring object range in terms of image velocity. We have also shown the viability and usefulness of various other insect behaviours: (i) keeping walls equidistant, (ii) slowing down when approaching an object, (iii) regulating speed according to tunnel width, and (iv) using visual motion as a measure of distance travelled.

Wireless haptic rendering for mobile platforms

Computer haptics has so far been performed on a personal computer (PC). Off the shelf haptic devices provide only PC interfaces and software drivers for control and communication. The new wave of high capable tablet PCs and high end smart phones introduced new platforms for haptic applications. The major problem was to communicate wirelessly to provide user convenience and support mobility which is an essential feature for these platforms. In this paper we provide a wireless layered communication protocol and a hardware setup that enables off the shelf haptic devices to communicate wirelessly with a mobile device. The layers in the protocol enable the change of any hardware components without affecting the data flow. However, the adoption of the wireless interface instead of the wired one comes with the price of speed. Haptic refresh loops require a relatively high refresh rate of 1000 Hz compared to graphics loop which require between 30 and 60 only. An interpolation algorithm was demonstrated to compensate the latency and secure a stable user experience. The introduced setup was tested against portable environments and the users could perform similar functionalities to what are available on a wired setup to a PC.

Integrating kinect and haptics for interactive STEM education in local and distributed environments

Skill shortage is a realistic social problem that Australia is currently facing, especially in the fields of Science, Technology, Engineering and Mathematics (STEM). Various approaches have been proposed to soften this issue. By now the most successful approach is to attract pre-university youth and university freshmen into those fields before they make a decision on future subjects by introducing them with interactive, modifiable and inspiring virtual environments, which incorporates most essential knowledge of STEM. We propose to design a comprehensive virtual reality platform with immersive interactions, pluggable components and flexible configurations. It also involves haptics, motion capture and gesture recognition, and could be deployed in both local and distributed environments. The platform utilizes off the shelf low cost haptics and motion capture products, however the fidelity can be maintained at a good level. The proposed platform has been implemented with different configurations and has been tested on a group of users. Preliminary test results show that the interactivity, flexibility and fidelity of the platform are highly appreciated by users. User surveys also indicate that the proposed platform could help pre-university students and university freshmen build an overview of various aspects of STEM education. Besides, users are also positive on the fact that the platform enabled them to identify the challenges for higher education in STEM by providing them opportunities to interactively modify system configurations and instantly experience the corresponding results both visually and haptically.

An illumination invariant face recognition system for access control using video

Illumination and pose invariance are the most challenging aspects of face recognition. In this paper we describe a fully automatic face recognition system that uses video information to achieve illumination and pose robustness. In the proposed method, highly nonlinear manifolds of face motion are approximated using three Gaussian pose clusters. Pose robustness is achieved by comparing the corresponding pose clusters and probabilistically combining the results to derive a measure of similarity between two manifolds. Illumination is normalized on a per-pose basis. Region-based gamma intensity correction is used to correct for coarse illumination changes, while further refinement is achieved by combining a learnt linear manifold of illumination variation with constraints on face pattern distribution, derived from video. Comparative experimental evaluation is presented and the proposed method is shown to greatly outperform state-of-the-art algorithms. Consistent recognition rates of 94-100% are achieved across dramatic changes in illumination.

Multipoint haptic mediator interface for robotic teleoperation

Despite recent advances in artificial intelligence and autonomous robotics, teleoperation can provide distinct benefits in applications requiring real-time human judgement and intuition. However, as robotic systems are increasingly becoming sophisticated and are performing more complex tasks, realizing these benefits requires new approaches to teleoperation. This paper introduces a novel haptic mediator interface for teleoperating mobile robotic platforms that have a variety of manipulators and functions. Identical master-slave bilateral teleoperation of the robotic manipulators is achieved by representing them in virtual reality and by allowing the operator to interact with them using a multipoint haptic device. The operator is also able to command motions to the mobile platform by using a novel haptic interaction metaphor rather than a separate dedicated input device. The presented interaction techniques enable the operator to perform a wide range of control functions and achieve functionality similar to that of conventional teleoperation schemes that use a single haptic interface. The mediator interface is presented, and important considerations such as workspace mapping and scaling are discussed. © 2015 IEEE.

Multipoint haptic guidance for micrograsping systems

The ability to perform accurate micromanipulation offers wide-reaching benefits and is of increasing interest to researchers. Recent research into microgripper, microtweezer, and microforcep systems contributes toward accurate micrograsping and manipulation. Despite these efforts, achieving adequate operator control remains a distinct research challenge. Haptic interfaces interact with the human's haptic modality and offer the ability to enhance the operator's controllability of micromanipulation systems. Our previous work introduced single-point haptic guidance to assist the operator during intracellular microinjection. This paper extends the approach to propose multipoint haptic guidance for micrograsping tasks. Accurate micrograsping is valuable in many applications, including microassembly and biomanipulation. A multipoint haptic gripper facilitates haptic interaction, and haptic guidance assists the operator in controlling systems suitable for micrograsping. Force fields are used to guide the operator to suitable grasp points on micrometer-sized objects and consist of attractive and repulsive forces. The ability of the force field to effectively assist the operator in grasping the cell is evaluated using a virtual environment. Evaluation results demonstrate the ability of the approach to significantly reduce participants' average grasping error.

Haptic virtual reality training environment for micro-robotic cell injection

Micro-robotic cell injection is typically performed manually by a trainedbio-operator, and success rates are often low. To enhance bio-operator performance during real-time cell injection, our earlier work introduced a haptically-enabled micro-robotic cell injection system. The system employed haptic virtual fixtures to provide haptic guidance according to articular performance metrics. This paper extends the work by replicating the system within a virtual reality (VR) environment for bio-operator training. Using the virtual environment, the bio-operator is able to control the virtual injection process in the same way they would with the physical haptic micro-robotic cell injection system, while benefiting from the enhanced visualisation capabilities offered by the 3D VR environment. The system is achieved using cost-effective components offering training at much lower cost than using the physical system.

Robust optimal motion cueing algorithm based on the linear quadratic regulator method and a genetic algorithm

The aim of this paper is to design and develop an optimal motion cueing algorithm (MCA) based on the genetic algorithm (GA) that can generate high-fidelity motions within the motion simulator's physical limitations. Both, angular velocity and linear acceleration are adopted as the inputs to the MCA for producing the higher order optimal washout filter. The linear quadratic regulator (LQR) method is used to constrain the human perception error between the real and simulated driving tasks. To develop the optimal MCA, the latest mathematical models of the vestibular system and simulator motion are taken into account. A reference frame with the center of rotation at the driver's head to eliminate false motion cues caused by rotation of the simulator to the translational motion of the driver's head as well as to reduce the workspace displacement is employed. To improve the developed LQR-based optimal MCA, a new strategy based on optimal control theory and the GA is devised. The objective is to reproduce a signal that can follow closely the reference signal and avoid false motion cues by adjusting the parameters from the obtained LQR-based optimal washout filter. This is achieved by taking a series of factors into account, which include the vestibular sensation error between the real and simulated cases, the main dynamic limitations, the human threshold limiter in tilt coordination, the cross correlation coefficient, and the human sensation error fluctuation. It is worth pointing out that other related investigations in the literature normally do not consider the effects of these factors. The proposed optimized MCA based on the GA is implemented using the MATLAB/Simulink software. The results show the effectiveness of the proposed GA-based method in enhancing human sensation, maximizing the reference shape tracking, and reducing the workspace usage.

Countering Improvised explosive devices through a multi-point haptic teleoperation system

Improvised Explosive Devices (IEDs) are reported as the number one cause of injury and death for allied troops in the current theater of operation. Current stand-off technologies for Counter IED (CIED) tasks rely on robotic platforms that have not improved in capability over the past decade to combat the ever increasing threat of IEDs. While they provide operational capability, the effectiveness of these platforms is limited. This is because they primarily utilise video and audio feedback, and require extensive training and specialist operators. Recent operational experience has demonstrated the need for robotic systems that are highly capable, yet easily operable for high fidelity manipulation. Force feedback provides an operator with more intuitive control of a robotic system. This sense of touch allows an operator to obtain a sense of feel from a stand-off location of what the robot touches or grasps through a human-robot interface. This paper reports the design and development of a Haptically-Enabled Counter IED robotic system that was funded by the Australian Defence Force. The presented work focuses on the design methodology for the system, and provides the results of the manipulator analysis and trial outcomes.