Multimodal augmented reality system for surgical microscopy

Image-guided, computer-assisted neurosurgery has emerged to improve localization and targeting, to provide a better anatomic definition of the surgical field, and to decrease invasiveness. Usually, in image-guided surgery, a computer displays the surgical field in a CT/MR environment, using axial, coronal or sagittal views, or even a 3D representation of the patient. Such a system forces the surgeon to look away from the surgical scene to the computer screen. Moreover, this kind of information, being pre-operative imaging, can not be modified during the operation, so it remains valid for guidance in the first stage of the surgical procedure, and mainly for rigid structures like bones. In order to solve the two constraints mentioned before, we are developing an ultrasoundguided surgical microscope. Such a system takes the advantage that surgical microscopy and ultrasound systems are already used in neurosurgery, so it does not add more complexity to the surgical procedure. We have integrated an optical tracking device in the microscope and an augmented reality overlay system with which we avoid the need to look away from the scene, providing correctly aligned surgical images with sub-millimeter accuracy. In addition to the standard CT and 3D views, we are able to track an ultrasound probe, and using a previous calibration and registration of the imaging, the image obtained is correctly projected to the overlay system, so the surgeon can always localize the target and verify the effects of the intervention. Several tests of the system have been already performed to evaluate the accuracy, and clinical experiments are currently in progress in order to validate the clinical usefulness of the system.

Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy

PURPOSE: The aim of this study is to implement augmented reality in real-time image-guided interstitial brachytherapy to allow an intuitive real-time intraoperative orientation. METHODS AND MATERIALS: The developed system consists of a common video projector, two high-resolution charge coupled device cameras, and an off-the-shelf notebook. The projector was used as a scanning device by projecting coded-light patterns to register the patient and superimpose the operating field with planning data and additional information in arbitrary colors. Subsequent movements of the nonfixed patient were detected by means of stereoscopically tracking passive markers attached to the patient. RESULTS: In a first clinical study, we evaluated the whole process chain from image acquisition to data projection and determined overall accuracy with 10 patients undergoing implantation. The described method enabled the surgeon to visualize planning data on top of any preoperatively segmented and triangulated surface (skin) with direct line of sight during the operation. Furthermore, the tracking system allowed dynamic adjustment of the data to the patient's current position and therefore eliminated the need for rigid fixation. Because of soft-part displacement, we obtained an average deviation of 1.1 mm by moving the patient, whereas changing the projector's position resulted in an average deviation of 0.9 mm. Mean deviation of all needles of an implant was 1.4 mm (range, 0.3-2.7 mm). CONCLUSIONS: The developed low-cost augmented-reality system proved to be accurate and feasible in interstitial brachytherapy. The system meets clinical demands and enables intuitive real-time intraoperative orientation and monitoring of needle implantation.

Performance of vertebral cancellous bone augmented with compliant PMMA under dynamic loads

Increased fracture risk has been reported for the adjacent vertebral bodies after vertebroplasty. This increase has been partly attributed to the high Young's modulus of commonly used polymethylmethacrylate (PMMA). Therefore, a compliant bone cement of PMMA with a bulk modulus closer to the apparent modulus of cancellous bone has been produced. This compliant bone cement was achieved by introducing pores in the cement. Due to the reduced failure strength of that porous PMMA cement, cancellous bone augmented with such cement could deteriorate under dynamic loading. The aim of the present study was to assess the potential of acute failure, particle generation and mechanical properties of cancellous bone augmented with this compliant cement in comparison to regular cement. For this purpose, vertebral biopsies were augmented with porous- and regular PMMA bone cement, submitted to dynamic tests and compression to failure. Changes in Young's modulus and height due to dynamic loading were determined. Afterwards, yield strength and Young's modulus were determined by compressive tests to failure and compared to the individual composite materials. No failure occurred and no particle generation could be observed during dynamical testing for both groups. Height loss was significantly higher for the porous cement composite (0.53+/-0.21%) in comparison to the biopsies augmented with regular cement (0.16+/-0.1%). Young's modulus of biopsies augmented with porous PMMA was comparable to cancellous bone or porous cement alone (200-700 MPa). The yield strength of those biopsies (21.1+/-4.1 MPa) was around two times higher than for porous cement alone (11.6+/-3.3 MPa).

Augmented reality endoscopic system (ARES): preliminary results

During endoscopic surgery, it is difficult to ascertain the anatomical landmarks once the anatomy is fiddled with or if the operating area is filled with blood. An augmented reality system will enhance the endoscopic view and further enable surgeons to view hidden critical structures or the results of preoperative planning.

Differences in endplate deformation of the adjacent and augmented vertebra following cement augmentation

Vertebral cement augmentation can restore the stiffness and strength of a fractured vertebra and relieve chronic pain. Previous finite element analysis, biomechanical tests and clinical studies have indirectly associated new adjacent vertebral fractures following augmentation to altered loading. The aim of this repeated measures in situ biomechanical study was to determine the changes in the adjacent and augmented endplate deformation following cement augmentation of human cadaveric functional spine units (FSU) using micro-computed tomography (micro-CT). The surrounding soft tissue and posterior elements of 22 cadaveric human FSU were removed. FSU were assigned to two groups, control (n = 8) (loaded on day 1 and day 2) and augmented (n = 14) (loaded on day 1, augmented 20% cement fill, and loaded on day 2). The augmented group was further subdivided into a prophylactic augmentation group (n = 9), and vertebrae which spontaneously fractured during loading on day 1 (n = 5). The FSU were axially loaded (200, 1,000, 1,500-2,000 N) within a custom made radiolucent, saline filled loading device. At each loading step, FSUs were scanned using the micro-CT. Endplate heights were determined using custom software. No significant increase in endplate deformation following cement augmentation was noted for the adjacent endplate (P > 0.05). The deformation of the augmented endplate was significantly reduced following cement augmentation for both the prophylactic and fracture group (P < 0.05, P < 0.01, respectively). Endplate deformation of the controls showed no statistically significant differences between loading on day 1 and day 2. A linear relationship was noted between the applied compressive load and endplate deflection (R (2) = 0.58). Evidence of significant endplate deformation differences between unaugmented and augmented FSU, while evident for the augmented endplate, was not present for the adjacent endplate. This non-invasive micro-CT method may also be useful to investigate endplate failure, and parameters that predict vertebral failure.

Landmark-based augmented reality system for paranasal and transnasal endoscopic surgeries

BACKGROUND: In this paper we present a landmark-based augmented reality (AR) endoscope system for endoscopic paranasal and transnasal surgeries along with fast and automatic calibration and registration procedures for the endoscope. METHODS: Preoperatively the surgeon selects natural landmarks or can define new landmarks in CT volume. These landmarks are overlaid, after proper registration of preoperative CT to the patient, on the endoscopic video stream. The specified name of the landmark, along with selected colour and its distance from the endoscope tip, is also augmented. The endoscope optics are calibrated and registered by fast and automatic methods. Accuracy of the system is evaluated in a metallic grid and cadaver set-up. RESULTS: Root mean square (RMS) error of the system is 0.8 mm in a controlled laboratory set-up (metallic grid) and was 2.25 mm during cadaver studies. CONCLUSIONS: A novel landmark-based AR endoscope system is implemented and its accuracy is evaluated. Augmented landmarks will help the surgeon to orientate and navigate the surgical field. Studies prove the capability of the system for the proposed application. Further clinical studies are planned in near future.