Low-cost optical tracking for immersive collaboration in the CAVE using the Wii remote

We present a novel way of interacting with an immersive virtual environment which involves inexpensive motion-capture using the Wii Remote®. A software framework is also presented to visualize and share this information across two remote CAVETM-like environments. The resulting applications can be used to assist rehabilitation by sending motion information across remote sites. The application’s software and hardware components are scalable enough to be used on desktop computer when home-based rehabilitation is preferred.

Low-cost optical tracking for collaborative applications in immersive virtual environments

We present a novel way of interacting with an immersive virtual environment which involves inexpensive motion-capture using the Wii Remote®. A software framework is also presented to visualize and share this information across two remote CAVETM-like environments. The resulting application can be used to assist rehabilitation by sending motion information across remote sites. The application’s software and hardware components are scalable enough to be used on a desktop computer when home-based rehabilitation is preferred.

A low cost one-camera optical tracking system for indoor wide-area augmented and virtual reality environments

In the last years the number of industrial applications for Augmented Reality (AR) and Virtual Reality (VR) environments has significantly increased. Optical tracking systems are an important component of AR/VR environments. In this work, a low cost optical tracking system with adequate attributes for professional use is proposed. The system works in infrared spectral region to reduce optical noise. A highspeed camera, equipped with daylight blocking filter and infrared flash strobes, transfers uncompressed grayscale images to a regular PC, where image pre-processing software and the PTrack tracking algorithm recognize a set of retro-reflective markers and extract its 3D position and orientation. Included in this work is a comprehensive research on image pre-processing and tracking algorithms. A testbed was built to perform accuracy and precision tests. Results show that the system reaches accuracy and precision levels slightly worse than but still comparable to professional systems. Due to its modularity, the system can be expanded by using several one-camera tracking modules linked by a sensor fusion algorithm, in order to obtain a larger working range. A setup with two modules was built and tested, resulting in performance similar to the stand-alone configuration.

Comparison of three optical tracking systems in a complex navigation scenario

Three-dimensional rotational X-ray imaging with the SIREMOBIL Iso-C3D (Siemens AG, Medical Solutions, Erlangen, Germany) has become a well-established intra-operative imaging modality. In combination with a tracking system, the Iso-C3D provides inherently registered image volumes ready for direct navigation. This is achieved by means of a pre-calibration procedure. The aim of this study was to investigate the influence of the tracking system used on the overall navigation accuracy of direct Iso-C3D navigation. Three models of tracking system were used in the study: Two Optotrak 3020s, a Polaris P4 and a Polaris Spectra system, with both Polaris systems being in the passive operation mode. The evaluation was carried out at two different sites using two Iso-C3D devices. To measure the navigation accuracy, a number of phantom experiments were conducted using an acrylic phantom equipped with titanium spheres. After scanning, a special pointer was used to pinpoint these markers. The difference between the digitized and navigated positions served as the accuracy measure. Up to 20 phantom scans were performed for each tracking system. The average accuracy measured was 0.86 mm and 0.96 mm for the two Optotrak 3020 systems, 1.15 mm for the Polaris P4, and 1.04 mm for the Polaris Spectra system. For the Polaris systems a higher maximal error was found, but all three systems yielded similar minimal errors. On average, all tracking systems used in this study could deliver similar navigation accuracy. The passive Polaris system showed ? as expected ? higher maximal errors; however, depending on the application constraints, this might be negligible.

Augmenting the effective field of view of optical tracking cameras--a way to overcome difficulties during intraoperative camera alignment

An Internet survey demonstrated the existence of problems related to intraoperative tracking camera set-up and alignment. It is hypothesized that these problems are a result of the limited field of view of today's optoelectronic camera systems, which is usually insufficiently large to keep the entire site of surgical action in view during an intervention. A method is proposed to augment a camera's field of view by actively controlling camera orientation, enabling it to track instruments as they are used intraoperatively. In an experimental study, an increase of almost 300% was found in the effective volume in which instruments could be tracked.

Quantitative analysis of accuracy of an inertial/acoustic 6DOF tracking system in motion

An increasing number of neuroscience experiments are using virtual reality to provide a more immersive and less artificial experimental environment. This is particularly useful to navigation and three-dimensional scene perception experiments. Such experiments require accurate real-time tracking of the observer's head in order to render the virtual scene. Here, we present data on the accuracy of a commonly used six degrees of freedom tracker (Intersense IS900) when it is moved in ways typical of virtual reality applications. We compared the reported location of the tracker with its location computed by an optical tracking method. When the tracker was stationary, the root mean square error in spatial accuracy was 0.64 mm. However, we found that errors increased over ten-fold (up to 17 mm) when the tracker moved at speeds common in virtual reality applications. We demonstrate that the errors we report here are predominantly due to inaccuracies of the IS900 system rather than the optical tracking against which it was compared. (c) 2006 Elsevier B.V. All rights reserved.

Phase calibration of the EISCAT Svalbard Radar interferometer using optical satellite signatures

The link between natural ion-line enhancements in radar spectra and auroral activity has been the subject of recent studies but conclusions have been limited by the spatial and temporal resolution previously available. The next challenge is to use shorter sub-second integration times in combination with interferometric programmes to resolve spatial structure within the main radar beam, and so relate enhanced filaments to individual auroral rays. This paper presents initial studies of a technique, using optical and spectral satellite signatures, to calibrate the received phase of a signal with the position of the scattering source along the interferometric baseline of the EISCAT Svalbard Radar. It is shown that a consistent relationship can be found only if the satellite passage through the phase fringes is adjusted from the passage predicted by optical tracking. This required adjustment is interpreted as being due to the vector between the theoretical focusing points of the two antennae, i.e. the true radar baseline, differing from the baseline obtained by survey between the antenna foot points. A method to obtain a measurement of the true interferometric baseline using multiple satellite passes is outlined.

Endoscopic optical coherence tomography : design and application

This thesis presents an investigation on endoscopic optical coherence tomography (OCT). As a noninvasive imaging modality, OCT emerges as an increasingly important diagnostic tool for many clinical applications. Despite of many of its merits, such as high resolution and depth resolvability, a major limitation is the relatively shallow penetration depth in tissue (about 2∼3 mm). This is mainly due to tissue scattering and absorption. To overcome this limitation, people have been developing many different endoscopic OCT systems. By utilizing a minimally invasive endoscope, the OCT probing beam can be brought to the close vicinity of the tissue of interest and bypass the scattering of intervening tissues so that it can collect the reflected light signal from desired depth and provide a clear image representing the physiological structure of the region, which can not be disclosed by traditional OCT. In this thesis, three endoscope designs have been studied. While they rely on vastly different principles, they all converge to solve this long-standing problem.

A hand-held endoscope with manual scanning is first explored. When a user is holding a hand- held endoscope to examine samples, the movement of the device provides a natural scanning. We proposed and implemented an optical tracking system to estimate and record the trajectory of the device. By registering the OCT axial scan with the spatial information obtained from the tracking system, one can use this system to simply ‘paint’ a desired volume and get any arbitrary scanning pattern by manually waving the endoscope over the region of interest. The accuracy of the tracking system was measured to be about 10 microns, which is comparable to the lateral resolution of most OCT system. Targeted phantom sample and biological samples were manually scanned and the reconstructed images verified the method.

Next, we investigated a mechanical way to steer the beam in an OCT endoscope, which is termed as Paired-angle-rotation scanning (PARS). This concept was proposed by my colleague and we further developed this technology by enhancing the longevity of the device, reducing the diameter of the probe, and shrinking down the form factor of the hand-piece. Several families of probes have been designed and fabricated with various optical performances. They have been applied to different applications, including the collector channel examination for glaucoma stent implantation, and vitreous remnant detection during live animal vitrectomy.

Lastly a novel non-moving scanning method has been devised. This approach is based on the EO effect of a KTN crystal. With Ohmic contact of the electrodes, the KTN crystal can exhibit a special mode of EO effect, termed as space-charge-controlled electro-optic effect, where the carrier electron will be injected into the material via the Ohmic contact. By applying a high voltage across the material, a linear phase profile can be built under this mode, which in turn deflects the light beam passing through. We constructed a relay telescope to adapt the KTN deflector into a bench top OCT scanning system. One of major technical challenges for this system is the strong chromatic dispersion of KTN crystal within the wavelength band of OCT system. We investigated its impact on the acquired OCT images and proposed a new approach to estimate and compensate the actual dispersion. Comparing with traditional methods, the new method is more computational efficient and accurate. Some biological samples were scanned by this KTN based system. The acquired images justified the feasibility of the usage of this system into a endoscopy setting. My research above all aims to provide solutions to implement an OCT endoscope. As technology evolves from manual, to mechanical, and to electrical approaches, different solutions are presented. Since all have their own advantages and disadvantages, one has to determine the actual requirements and select the best fit for a specific application.

High-throughput indentational elasticity measurements of hydrogel extracellular matrix substrates

Much interest surrounds the effect of extracellular matrix (ECM) elasticity on cell behavior. Here we present a rapid method for measuring the elasticity of synthetic ECM substrates based on indentation of the substrate with a ferromagnetic sphere and optical tracking of the resulting deformation. We find that this method yields order-of-magnitude agreement with atomic force microscopy elasticity measurements, but that the degree of this agreement depends strongly on sphere density and gel elasticity. In its regime of greatest accuracy, we envision that this method may be used for high-throughput characterization of ECM substrates in cell biological studies.

针尖扫描原子力显微镜的光点跟踪设计

提出了一种用于针尖扫描原子力显微镜（AFM）的光点跟踪设计方案，结构简单，容易实现。设计方案对扫描器的负载能力要求不高，而且能使原子力显微镜实现较大范围的针尖扫描。实验结果表明，采用此光点跟踪设计方案的针尖扫描原子力显微镜能实现最大100μm×100μm范围的扫描，z方向上的误差最大1nm，能很好地满足大样品扫描的需要。

Exploring the use of sensors to measure behavioral interactions: an experimental evaluation of using hand trajectories.

Humans appear to be sensitive to relative small changes in their surroundings. These changes are often initially perceived as irrelevant, but they can cause significant changes in behavior. However, how exactly people's behavior changes is often hard to quantify. A reliable and valid tool is needed in order to address such a question, ideally measuring an important point of interaction, such as the hand. Wearable-body-sensor systems can be used to obtain valuable, behavioral information. These systems are particularly useful for assessing functional interactions that occur between the endpoints of the upper limbs and our surroundings. A new method is explored that consists of computing hand position using a wearable sensor system and validating it against a gold standard reference measurement (optical tracking device). Initial outcomes related well to the gold standard measurements (r = 0.81) showing an acceptable average root mean square error of 0.09 meters. Subsequently, the use of this approach was further investigated by measuring differences in motor behavior, in response to a changing environment. Three subjects were asked to perform a water pouring task with three slightly different containers. Wavelet analysis was introduced to assess how motor consistency was affected by these small environmental changes. Results showed that the behavioral motor adjustments to a variable environment could be assessed by applying wavelet coherence techniques. Applying these procedures in everyday life, combined with correct research methodologies, can assist in quantifying how environmental changes can cause alterations in our motor behavior.

Análise do padrão motor em pessoas com diagnóstico de esquizofrenia: uma abordagem em tempo real

A esquizofrenia é uma perturbação mental grave caracterizada pela coexistência de sintomas positivos, negativos e de desorganização do pensamento e do comportamento. As alterações motoras são consistentemente observadas mas, ainda pouco estudadas na esquizofrenia, sendo relevantes para o seu diagnóstico. Neste quadro, o presente estudo tem como objetivo verificar se os indivíduos com esquizofrenia apresentam alterações na coordenação motora, comparativamente com o grupo sem esquizofrenia, bem como analisar se as disfunções dos sinais neurológicos subtis (SNS) motores se encontram correlacionadas com o funcionamento executivo e com os domínios psicopatológicos da perturbação. No total participaram 29 indivíduos (13 com diagnóstico de esquizofrenia e 16 sem diagnóstico) equivalentes em termos de idade, género, escolaridade e índice de massa corporal. Para avaliar o desempenho motor recorreu-se ao sistema Biostage de parametrização do movimento em tempo real, com a tarefa de lançameto ao alvo; a presença de SNS foi examinada através da Brief Motor Scale; o funcionamento executivo pela aplicação do subteste do Vocabulário e da fluência verbal e a sintomatologia clínica através da Positive and Negative Sindrome Scale. Pela análise cinemática do movimento constatou-se que os indivíduos com esquizofrenia recrutam um padrão motor menos desenvolvido e imaturo de movimento, com menor individualização das componentes (principalmente do tronco e pélvis), necessitando de mais tempo para executar a tarefa, comparativamente com os sujeitos sem a perturbação que evidenciaram um movimento mais avançado de movimento. Os indivíduos com esquizofrenia mostraram índices elevados de disfunção dos SNS (média =6,01) estabelecendo este domínio uma relação boa e negativa com o desempenho verbal (rho Spearman=-0,62) e uma relação forte e positiva com todos os domínios psicopatológicos (rho Spearman=0,74). O estudo da existência de alterações motoras como parte intrínseca da esquizofrenia revela-se pertinente uma vez que possibilita uma compreensão mais aprofundada da sua fisiopatologia e permite que se desenvolvam práticas mais efetivas na área da saúde e reabilitação.

Mutual occlusion between real and virtual elements in Augmented Reality based on fiducial markers

Augmented Reality (AR) systems which use optical tracking with fiducial marker for registration have had an important role in popularizing this technology, since only a personal computer with a conventional webcam is required. However, in most these applications, the virtual elements are shown only in the foreground a real element does not occlude a virtual one. The method presented enables AR environments based on fiducial markers to support mutual occlusion between a real element and many virtual ones, according to the elements position (depth) in the environment. © 2012 IEEE.

Multimodal augmented reality system for surgical microscopy

Image-guided, computer-assisted neurosurgery has emerged to improve localization and targeting, to provide a better anatomic definition of the surgical field, and to decrease invasiveness. Usually, in image-guided surgery, a computer displays the surgical field in a CT/MR environment, using axial, coronal or sagittal views, or even a 3D representation of the patient. Such a system forces the surgeon to look away from the surgical scene to the computer screen. Moreover, this kind of information, being pre-operative imaging, can not be modified during the operation, so it remains valid for guidance in the first stage of the surgical procedure, and mainly for rigid structures like bones. In order to solve the two constraints mentioned before, we are developing an ultrasoundguided surgical microscope. Such a system takes the advantage that surgical microscopy and ultrasound systems are already used in neurosurgery, so it does not add more complexity to the surgical procedure. We have integrated an optical tracking device in the microscope and an augmented reality overlay system with which we avoid the need to look away from the scene, providing correctly aligned surgical images with sub-millimeter accuracy. In addition to the standard CT and 3D views, we are able to track an ultrasound probe, and using a previous calibration and registration of the imaging, the image obtained is correctly projected to the overlay system, so the surgeon can always localize the target and verify the effects of the intervention. Several tests of the system have been already performed to evaluate the accuracy, and clinical experiments are currently in progress in order to validate the clinical usefulness of the system.

Impact of a self-developed planning and self-constructed navigation system on skull base surgery: 10 years experience

CONCLUSION: Our self-developed planning and navigation system has proven its capacity for accurate surgery on the anterior and lateral skull base. With the incorporation of augmented reality, image-guided surgery will evolve into 'information-guided surgery'. OBJECTIVE: Microscopic or endoscopic skull base surgery is technically demanding and its outcome has a great impact on a patient's quality of life. The goal of the project was aimed at developing and evaluating enabling navigation surgery tools for simulation, planning, training, education, and performance. This clinically applied technological research was complemented by a series of patients (n=406) who were treated by anterior and lateral skull base procedures between 1997 and 2006. MATERIALS AND METHODS: Optical tracking technology was used for positional sensing of instruments. A newly designed dynamic reference base with specific registration techniques using fine needle pointer or ultrasound enables the surgeon to work with a target error of < 1 mm. An automatic registration assessment method, which provides the user with a color-coded fused representation of CT and MR images, indicates to the surgeon the location and extent of registration (in)accuracy. Integration of a small tracker camera mounted directly on the microscope permits an advantageous ergonomic way of working in the operating room. Additionally, guidance information (augmented reality) from multimodal datasets (CT, MRI, angiography) can be overlaid directly onto the surgical microscope view. The virtual simulator as a training tool in endonasal and otological skull base surgery provides an understanding of the anatomy as well as preoperative practice using real patient data. RESULTS: Using our navigation system, no major complications occurred in spite of the fact that the series included difficult skull base procedures. An improved quality in the surgical outcome was identified compared with our control group without navigation and compared with the literature. The surgical time consumption was reduced and more minimally invasive approaches were possible. According to the participants' questionnaires, the educational effect of the virtual simulator in our residency program received a high ranking.