Survey on haptic rendering of data sets: Exploration of scalar and vector fields

Complementary to automatic extraction processes, Virtual Reality technologies provide an adequate framework to integrate human perception in the exploration of large data sets. In such multisensory system, thanks to intuitive interactions, a user can take advantage of all his perceptual abilities in the exploration task. In this context the haptic perception, coupled to visual rendering, has been investigated for the last two decades, with significant achievements. In this paper, we present a survey related to exploitation of the haptic feedback in exploration of large data sets. For each haptic technique introduced, we describe its principles and its effectiveness.

Dental training system using haptic technology

In the U.K., dental students require to perform training and practice on real human tissues at the very early stage of their courses. Currently, the human tissues, such as decayed teeth, are mounted in a human head like physical model. The problems with these models in teaching are; (1) every student operates on tooth, which are always unique; (2) the process cannot be recorded for examination purposes and (3) same training are not repeatable. The aim of the PHATOM Project is to develop a dental training system using Haptic technology. This paper documents the project background, specification, research and development of the first prototype system. It also discusses the research in the visual display, haptic devices and haptic rendering. This includes stereo vision, motion parallax, volumetric modelling, surface remapping algorithms as well as analysis design of the system. A new volumetric to surface model transformation algorithm is also introduced. This paper includes the future work on the system development and research.

Multi-finger grasps in a dynamic environment

Most current state-of-the-art haptic devices render only a single force, however almost all human grasps are characterised by multiple forces and torques applied by the fingers and palms of the hand to the object. In this chapter we will begin by considering the different types of grasp and then consider the physics of rigid objects that will be needed for correct haptic rendering. We then describe an algorithm to represent the forces associated with grasp in a natural manner. The power of the algorithm is that it considers only the capabilities of the haptic device and requires no model of the hand, thus applies to most practical grasp types. The technique is sufficiently general that it would also apply to multi-hand interactions, and hence to collaborative interactions where several people interact with the same rigid object. Key concepts in friction and rigid body dynamics are discussed and applied to the problem of rendering multiple forces to allow the person to choose their grasp on a virtual object and perceive the resulting movement via the forces in a natural way. The algorithm also generalises well to support computation of multi-body physics

Volume Haptics and Dendritic Spine Editing

Esta tesis se ha desarrollado en el contexto del proyecto Cajal Blue Brain, una iniciativa europea dedicada al estudio del cerebro. Uno de los objetivos de esta iniciativa es desarrollar nuevos métodos y nuevas tecnologías que simplifiquen el análisis de datos en el campo neurocientífico. El presente trabajo se ha centrado en diseñar herramientas que combinen información proveniente de distintos canales sensoriales con el fin de acelerar la interacción y análisis de imágenes neurocientíficas. En concreto se estudiará la posibilidad de combinar información visual con información háptica. Las espinas dendríticas son pequeñas protuberancias que recubren la superficie dendrítica de muchas neuronas del cerebro. A día de hoy, se cree que tienen un papel clave en la transmisión de señales neuronales. Motivo por el cual, el interés por parte de la comunidad científica por estas estructuras ha ido en aumento a medida que las técnicas de adquisición de imágenes mejoraban hasta alcanzar una calidad suficiente para analizar dichas estructuras. A menudo, los neurocientíficos utilizan técnicas de microscopía con luz para obtener los datos que les permitan analizar estructuras neuronales tales como neuronas, dendritas y espinas dendríticas. A pesar de que estas técnicas ofrezcan ciertas ventajas frente a su equivalente electrónico, las técnicas basadas en luz permiten una menor resolución. En particular, estructuras pequeñas como las espinas dendríticas pueden capturarse de forma incorrecta en las imágenes obtenidas, impidiendo su análisis. En este trabajo, se presenta una nueva técnica, que permite editar imágenes volumétricas, mediante un dispositivo háptico, con el fin de reconstruir de los cuellos de las espinas dendríticas. Con este objetivo, en un primer momento se desarrolló un algoritmo que proporciona retroalimentación háptica en datos volumétricos, completando la información que provine del canal visual. Dicho algoritmo de renderizado háptico permite a los usuarios tocar y percibir una isosuperficie en el volumen de datos. El algoritmo asegura un renderizado robusto y eficiente. Se utiliza un método basado en las técnicas de “marching tetrahedra” para la extracción local de una isosuperficie continua, lineal y definida por intervalos. La robustez deriva tanto de una etapa de detección de colisiones continua de la isosuperficie extraída, como del uso de técnicas eficientes de renderizado basadas en un proxy puntual. El método de “marching tetrahedra” propuesto garantiza que la topología de la isosuperficie extraída coincida con la topología de una isosuperficie equivalente determinada utilizando una interpolación trilineal. Además, con el objetivo de mejorar la coherencia entre la información háptica y la información visual, el algoritmo de renderizado háptico calcula un segundo proxy en la isosuperficie pintada en la pantalla. En este trabajo se demuestra experimentalmente las mejoras en, primero, la etapa de extracción de isosuperficie, segundo, la robustez a la hora de mantener el proxy en la isosuperficie deseada y finalmente la eficiencia del algoritmo. En segundo lugar, a partir del algoritmo de renderizado háptico propuesto, se desarrolló un procedimiento, en cuatro etapas, para la reconstrucción de espinas dendríticas. Este procedimiento, se puede integrar en los cauces de segmentación automática y semiautomática existentes como una etapa de pre-proceso previa. El procedimiento está diseñando para que tanto la navegación como el proceso de edición en sí mismo estén controlados utilizando un dispositivo háptico. Se han diseñado dos experimentos para evaluar esta técnica. El primero evalúa la aportación de la retroalimentación háptica y el segundo se centra en evaluar la idoneidad del uso de un háptico como dispositivo de entrada. En ambos casos, los resultados demuestran que nuestro procedimiento mejora la precisión de la reconstrucción. En este trabajo se describen también dos casos de uso de nuestro procedimiento en el ámbito de la neurociencia: el primero aplicado a neuronas situadas en la corteza cerebral humana y el segundo aplicado a espinas dendríticas situadas a lo largo de neuronas piramidales de la corteza del cerebro de una rata. Por último, presentamos el programa, Neuro Haptic Editor, desarrollado a lo largo de esta tesis junto con los diferentes algoritmos ya mencionados. ABSTRACT This thesis took place within the Cajal Blue Brain project, a European initiative dedicated to the study of the brain. One of the main goals of this project is the development of new methods and technologies simplifying data analysis in neuroscience. This thesis focused on the development of tools combining information originating from distinct sensory channels with the aim of accelerating both the interaction with neuroscience images and their analysis. In concrete terms, the objective is to study the possibility of combining visual information with haptic information. Dendritic spines are thin protrusions that cover the dendritic surface of numerous neurons in the brain and whose function seems to play a key role in neural circuits. The interest of the neuroscience community toward those structures kept increasing as and when acquisition methods improved, eventually to the point that the produced datasets enabled their analysis. Quite often, neuroscientists use light microscopy techniques to produce the dataset that will allow them to analyse neuronal structures such as neurons, dendrites and dendritic spines. While offering some advantages compared to their electronic counterpart, light microscopy techniques achieve lower resolutions. Particularly, small structures such as dendritic spines might suffer from a very low level of fluorescence in the final dataset, preventing further analysis. This thesis introduces a new technique enabling the edition of volumetric datasets in order to recreate dendritic spine necks using a haptic device. In order to fulfil this objective, we first presented an algorithm to provide haptic feedback directly from volumetric datasets, as an aid to regular visualization. The haptic rendering algorithm lets users perceive isosurfaces in volumetric datasets, and it relies on several design features that ensure a robust and efficient rendering. A marching tetrahedra approach enables the dynamic extraction of a piecewise linear continuous isosurface. Robustness is derived using a Continuous Collision Detection step coupled with acknowledged proxy-based rendering methods over the extracted isosurface. The introduced marching tetrahedra approach guarantees that the extracted isosurface will match the topology of an equivalent isosurface computed using trilinear interpolation. The proposed haptic rendering algorithm improves the coherence between haptic and visual cues computing a second proxy on the isosurface displayed on screen. Three experiments demonstrate the improvements on the isosurface extraction stage as well as the robustness and the efficiency of the complete algorithm. We then introduce our four-steps procedure for the complete reconstruction of dendritic spines. Based on our haptic rendering algorithm, this procedure is intended to work as an image processing stage before the automatic segmentation step giving the final representation of the dendritic spines. The procedure is designed to allow both the navigation and the volume image editing to be carried out using a haptic device. We evaluated our procedure through two experiments. The first experiment concerns the benefits of the force feedback and the second checks the suitability of the use of a haptic device as input. In both cases, the results shows that the procedure improves the editing accuracy. We also report two concrete cases where our procedure was employed in the neuroscience field, the first one concerning dendritic spines in the human cortex, the second one referring to an ongoing experiment studying dendritic spines along dendrites of mouse cortical pyramidal neurons. Finally, we present the software program, Neuro Haptic Editor, that was built along the development of the different algorithms implemented during this thesis, and used by neuroscientists to use our procedure.

Haptic Stabilization of Posture in Adults With Intellectual Disabilities Using a Nonrigid Tool

This study assessed the effects of haptic information on the postural control systems of individuals with intellectual disabilities (ID), through the use of a nonrigid tool that we call the ""anchor system"" (e.g., ropes attached to graduated weights that rest on the floor). Eleven participants with ID were asked to stand, blindfolded, on a balance beam placed at two heights (10 and 20 cm), for 30 s, while using the anchor system at two weights. The lighter anchor weight appeared to improve the individuals' balance in contrast to a control task condition; therefore, we concluded that haptic sensitivity was more significant in helping to orient the body than was the anchor's mechanical support alone.

Haptic information stabilizes and destabilizes coordination dynamics

Goal-directed, coordinated movements in humans emerge from a variety of constraints that range from 'high-level' cognitive strategies based oil perception of the task to 'low-level' neuromuscular-skeletal factors such as differential contributions to coordination from flexor and extensor muscles. There has been a tendency in the literature to dichotomize these sources of constraint, favouring one or the other rather than recognizing and understanding their mutual interplay. In this experiment, subjects were required to coordinate rhythmic flexion and extension movements with an auditory metronome, the rate of which was systematically increased. When subjects started in extension on the beat of the metronome, there was a small tendency to switch to flexion at higher rates, but not vice versa. When subjects: were asked to contact a physical stop, the location of which was either coincident with or counterphase to the auditor) stimulus, two effects occurred. When haptic contact was coincident with sound, coordination was stabilized for both flexion and extension. When haptic contact was counterphase to the metronome, coordination was actually destabilized, with transitions occurring from both extension to flexion on the beat and from flexion to extension on the beat. These results reveal the complementary nature of strategic and neuromuscular factors in sensorimotor coordination. They also suggest the presence of a multimodal neural integration process-which is parametrizable by rate and context - in which intentional movement, touch and sound are bound into a single, coherent unit.

Rendering peacekeeping instrumental? The Brazilian approach to United Nations peacekeeping during the Lula da Silva years (2003-2010)

The article analyses how Brazilian state actions and policies regarding peace operations during the Presidency of Lula da Silva relate to the country's positions and attitudes towards United Nations peacekeeping. It argues that the inconsistencies identified on the Brazilian positions reflect the lack of a clear strategic horizon guiding the country's participation in UN peacekeeping, which consequentially hinders the country emergence as a great power.

A survey of techniques and technologies for web-based real-time interactive rendering

When exploring a virtual environment, realism depends mainly on two factors: realistic images and real-time feedback (motions, behaviour etc.). In this context, photo realism and physical validity of computer generated images required by emerging applications, such as advanced e-commerce, still impose major challenges in the area of rendering research whereas the complexity of lighting phenomena further requires powerful and predictable computing if time constraints must be attained. In this technical report we address the state-of-the-art on rendering, trying to put the focus on approaches, techniques and technologies that might enable real-time interactive web-based clientserver rendering systems. The focus is on the end-systems and not the networking technologies used to interconnect client(s) and server(s).

The influence of consumption vocabulary on the encoding and retrieval of haptic information

A Work Project, presented as part of the requirements for the Award of a Masters Degree in Management from the NOVA – School of Business and Economics

Haptic robot-environment interaction for self-supervised learning in ground mobility

Dissertação para obtenção do Grau de Mestre em Engenharia Eletrotécnica e de Computadores

3D virtual rendering before laparoscopic partial splenectomy in children.

INTRODUCTION: Partial splenectomy in children is a good surgical option for hematological diseases and focal splenic tumors because it allows the preservation of the spleen's immunological function. Furthermore, it can be performed by laparoscopy in children as it is a safe procedure, offering the benefits of a minimally invasive approach. MATERIALS AND METHODS: The software VR-render LE version 0.81 is a system that enables the visualization of bidimentional 3D images with magnification of anatomical details. We have applied this system to five cases of non-parasitic splenic cysts before laparoscopic partial splenectomy. RESULTS: The images obtained with VR rendering software permitted the preoperative reconstruction of the vascularization of the splenic hilum, allowing the surgeon safe vessel control during laparoscopic procedures. All five partial splenectomies were carried out with no complications or major blood loss. CONCLUSIONS: Laparoscopic partial splenectomy should be a first choice procedure because it is feasible, reproducible, and safe for children; furthermore, it preserves enough splenic tissue thereby preventing post-splenectomy infections. Volume rendering provides high anatomical resolution and can be useful in guiding the surgical procedure.

Integration of a 3D rendering engine with a physics simulator

The goal of this project is the integration of a set of technologies (graphics, physical simulation, input), with the azm of assembling an application framework in phyton. In this research, a set of key introductory concepts are presented in adoption of a deep study of the state of the art of 3D applications. Phyton is selected an justified as the programing language due to the features and advantages that it offers in front of other languages. Finally the design and implementation of the framework is presented in the last chapter with some client application examples.

Serious games for rehabilitation using head-mounted display and haptic devices

In the health domain, the field of rehabilitation suffers from a lack specialized staff while hospital costs only increase. Worse, almost no tools are dedicated to motivate patients or help the personnel to carry out monitoring of therapeutic exercises. This paper demonstrates the high potential that can bring the virtual reality with a platform of serious games for the rehabilitation of the legs involving a head-mounted display and haptic robot devices. We first introduce SG principles and the current context regarding rehabilitation interventions followed by the description of an original haptic device called Lambda Health System. The architecture of the model is then detailed, including communication specifications showing that lag is imperceptible for user (60Hz). Finally, four serious games for rehabilitation using haptic robots and/or HMD were tested by 33 health specialists.

Multidetector-row computed tomography in the preoperative diagnosis of intestinal complications caused by clinically unsuspected ingested dietary foreign bodies: a case series emphasizing the use of volume rendering techniques

Objective The present study was aimed at describing a case series where a preoperative diagnosis of intestinal complications secondary to accidentally ingested dietary foreign bodies was made by multidetector-row computed tomography (MDCT), with emphasis on complementary findings yielded by volume rendering techniques (VRT) and curved multiplanar reconstructions (MPR). Materials and Methods The authors retrospectively assessed five patients with surgically confirmed intestinal complications (perforation and /or obstruction) secondary to unsuspected ingested dietary foreign bodies, consecutively assisted in their institution between 2010 and 2012. Demographic, clinical, laboratory and radiological data were analyzed. VRT and curved MPR were subsequently performed. Results Preoperative diagnosis of intestinal complications was originally performed in all cases. In one case the presence of a foreign body was not initially identified as the causal factor, and the use of complementary techniques facilitated its retrospective identification. In all cases these tools allowed a better depiction of the entire foreign bodies on a single image section, contributing to the assessment of their morphology. Conclusion Although the use of complementary techniques has not had a direct impact on diagnostic performance in most cases of this series, they may provide a better depiction of foreign bodies' morphology on a single image section.