Multimodal augmented reality system for surgical microscopy

Image-guided, computer-assisted neurosurgery has emerged to improve localization and targeting, to provide a better anatomic definition of the surgical field, and to decrease invasiveness. Usually, in image-guided surgery, a computer displays the surgical field in a CT/MR environment, using axial, coronal or sagittal views, or even a 3D representation of the patient. Such a system forces the surgeon to look away from the surgical scene to the computer screen. Moreover, this kind of information, being pre-operative imaging, can not be modified during the operation, so it remains valid for guidance in the first stage of the surgical procedure, and mainly for rigid structures like bones. In order to solve the two constraints mentioned before, we are developing an ultrasoundguided surgical microscope. Such a system takes the advantage that surgical microscopy and ultrasound systems are already used in neurosurgery, so it does not add more complexity to the surgical procedure. We have integrated an optical tracking device in the microscope and an augmented reality overlay system with which we avoid the need to look away from the scene, providing correctly aligned surgical images with sub-millimeter accuracy. In addition to the standard CT and 3D views, we are able to track an ultrasound probe, and using a previous calibration and registration of the imaging, the image obtained is correctly projected to the overlay system, so the surgeon can always localize the target and verify the effects of the intervention. Several tests of the system have been already performed to evaluate the accuracy, and clinical experiments are currently in progress in order to validate the clinical usefulness of the system.

Impact of a self-developed planning and self-constructed navigation system on skull base surgery: 10 years experience

CONCLUSION: Our self-developed planning and navigation system has proven its capacity for accurate surgery on the anterior and lateral skull base. With the incorporation of augmented reality, image-guided surgery will evolve into 'information-guided surgery'. OBJECTIVE: Microscopic or endoscopic skull base surgery is technically demanding and its outcome has a great impact on a patient's quality of life. The goal of the project was aimed at developing and evaluating enabling navigation surgery tools for simulation, planning, training, education, and performance. This clinically applied technological research was complemented by a series of patients (n=406) who were treated by anterior and lateral skull base procedures between 1997 and 2006. MATERIALS AND METHODS: Optical tracking technology was used for positional sensing of instruments. A newly designed dynamic reference base with specific registration techniques using fine needle pointer or ultrasound enables the surgeon to work with a target error of < 1 mm. An automatic registration assessment method, which provides the user with a color-coded fused representation of CT and MR images, indicates to the surgeon the location and extent of registration (in)accuracy. Integration of a small tracker camera mounted directly on the microscope permits an advantageous ergonomic way of working in the operating room. Additionally, guidance information (augmented reality) from multimodal datasets (CT, MRI, angiography) can be overlaid directly onto the surgical microscope view. The virtual simulator as a training tool in endonasal and otological skull base surgery provides an understanding of the anatomy as well as preoperative practice using real patient data. RESULTS: Using our navigation system, no major complications occurred in spite of the fact that the series included difficult skull base procedures. An improved quality in the surgical outcome was identified compared with our control group without navigation and compared with the literature. The surgical time consumption was reduced and more minimally invasive approaches were possible. According to the participants' questionnaires, the educational effect of the virtual simulator in our residency program received a high ranking.

Hardware accelerated ray cast of volume data and volume gradient for an optimized splines-based multi-resolution 2D-3D registration

This paper describes a method for DRR generation as well as for volume gradients projection using hardware accelerated 2D texture mapping and accumulation buffering and demonstrates its application in 2D-3D registration of X-ray fluoroscopy to CT images. The robustness of the present registration scheme are guaranteed by taking advantage of a coarse-to-fine processing of the volume/image pyramids based on cubic B-splines. A human cadaveric spine specimen together with its ground truth was used to compare the present scheme with a purely software-based scheme in three aspects: accuracy, speed, and capture ranges. Our experiments revealed an equivalent accuracy and capture ranges but with much shorter registration time with the present scheme. More specifically, the results showed 0.8 mm average target registration error, 55 second average execution time per registration, and 10 mm and 10° capture ranges for the present scheme when tested on a 3.0 GHz Pentium 4 computer.

Non-photorealistic rendering of virtual implant models for computer-assisted fluoroscopy-based surgical procedures

Surgical navigation systems visualize the positions and orientations of surgical instruments and implants as graphical overlays onto a medical image of the operated anatomy on a computer monitor. The orthopaedic surgical navigation systems could be categorized according to the image modalities that are used for the visualization of surgical action. In the so-called CT-based systems or 'surgeon-defined anatomy' based systems, where a 3D volume or surface representation of the operated anatomy could be constructed from the preoperatively acquired tomographic data or through intraoperatively digitized anatomy landmarks, a photorealistic rendering of the surgical action has been identified to greatly improve usability of these navigation systems. However, this may not hold true when the virtual representation of surgical instruments and implants is superimposed onto 2D projection images in a fluoroscopy-based navigation system due to the so-called image occlusion problem. Image occlusion occurs when the field of view of the fluoroscopic image is occupied by the virtual representation of surgical implants or instruments. In these situations, the surgeon may miss part of the image details, even if transparency and/or wire-frame rendering is used. In this paper, we propose to use non-photorealistic rendering to overcome this difficulty. Laboratory testing results on foamed plastic bones during various computer-assisted fluoroscopybased surgical procedures including total hip arthroplasty and long bone fracture reduction and osteosynthesis are shown.

Range of motion in anterior femoroacetabular impingement

The range of motion of normal hips and hips with femoroacetabular impingement relative to some specific anatomic reference landmarks is unknown. We therefore described: (1) the range of motion pattern relative to landmarks; (2) the location of the impingement zones in normal and impinging hips; and (3) the influence of surgical débridement on the range of motion. We used a previously developed and validated noninvasive 3-D CT-based method for kinematic hip analysis to compare the range of motion pattern, the location of impingement, and the effect of virtual surgical reconstruction in 28 hips with anterior femoroacetabular impingement and a control group of 33 normal hips. Hips with femoroacetabular impingement had decreased flexion, internal rotation, and abduction. Internal rotation decreased with increasing flexion and adduction. The calculated impingement zones were localized in the anterosuperior quadrant of the acetabulum and were similar in the two groups and in impingement subgroups. The average improvement of internal rotation was 5.4 degrees for pincer hips, 8.5 degrees for cam hips, and 15.7 degrees for mixed impingement. This method helps the surgeon quantify the severity of impingement and choose the appropriate treatment option; it provides a basis for future image-guided surgical reconstruction in femoroacetabular impingement with less invasive techniques.

Calibration of a surgical microscope with automated zoom lenses using an active optical tracker

BACKGROUND: In this paper, we present a new method for the calibration of a microscope and its registration using an active optical tracker. METHODS: Practically, both operations are done simultaneously by moving an active optical marker within the field of view of the two devices. The IR LEDs composing the marker are first segmented from the microscope images. By knowing their corresponding three-dimensional (3D) position in the optical tracker reference system, it is possible to find the transformation matrix between the referential of the two devices. Registration and calibration parameters can be extracted directly from that transformation. In addition, since the zoom and focus can be modified by the surgeon during the operation, we propose a spline based method to update the camera model to the new setup. RESULTS: The proposed technique is currently being used in an augmented reality system for image-guided surgery in the fields of ear, nose and throat (ENT) and craniomaxillofacial surgeries. CONCLUSIONS: The results have proved to be accurate and the technique is a fast, dynamic and reliable way to calibrate and register the two devices in an OR environment.

Surgical approach for synovial cyst of the atlantoaxial joint: a case report and review of the literature

STUDY DESIGN: Case report and review of the literature. OBJECTIVE: We describe the first case of a synovial cyst arising from pseudarthrosis of a previous dens fracture. The literature is reviewed and etiological, diagnostic, and therapeutic options of atlantoaxial cysts are discussed. SUMMARY OF BACKGROUND DATA: Symptomatic synovial cysts of the atlantoaxial joint are rare. To the authors' knowledge only 24 cases have been reported.A 60-year-old patient presented with bilateral hand numbness, quadrihyperreflexia, and gait deterioration. Magnetic resonance imaging of the cervical spine disclosed a cystic mass located at the transverse ligament of dens axis causing bulbomedullary compression. METHODS: Surgery was performed via transoral image guided approach. The ventral atlas arch, dens, transverse ligament, tectorial membrane, and the compressing cyst were removed, followed by a C0-C3 fusion. RESULTS: Two months postsurgery the patient recovered completely from the cervical myelopathy with transient remnant dysparesthesia of the finger tips. CONCLUSION: Magnetic resonance imaging findings are not specific enough to establish a preoperative diagnosis. Radical resection via image-guided transoral route followed by posterior fusion allows complete resection of the cystic lesion and results in excellent long-term decompression.

Cone Beam and Micro-Computed Tomography Validation of Manual Array Insertion for Minimally Invasive Cochlear Implantation

Delivering cochlear implants through a minimally invasive tunnel (1.8 mm in diameter) from the mastoid surface to the inner ear is referred to as direct cochlear access (DCA). Based on cone beam as well as micro-computed tomography imaging, this in vitro study evaluates the feasibility and efficacy of manual cochlear electrode array insertions via DCA. Free-fitting electrode arrays were inserted in 8 temporal bone specimens with previously drilled DCA tunnels. The insertion depth angle, procedural time, tunnel alignment as well as the inserted scala and intracochlear trauma were assessed. Seven of the 8 insertions were full insertions, with insertion depth angles higher than 520°. Three cases of atraumatic scala tympani insertion, 3 cases of probable basilar membrane rupture and 1 case of dislocation into the scala vestibuli were observed (1 specimen was damaged during extraction). Manual electrode array insertion following a DCA procedure seems to be feasible and safe and is a further step toward clinical application of image-guided otological microsurgery.

Estimation of tool pose based on force-density correlation during robotic drilling

The application of image-guided systems with or without support by surgical robots relies on the accuracy of the navigation process, including patient-to-image registration. The surgeon must carry out the procedure based on the information provided by the navigation system, usually without being able to verify its correctness beyond visual inspection. Misleading surrogate parameters such as the fiducial registration error are often used to describe the success of the registration process, while a lack of methods describing the effects of navigation errors, such as those caused by tracking or calibration, may prevent the application of image guidance in certain accuracy-critical interventions. During minimally invasive mastoidectomy for cochlear implantation, a direct tunnel is drilled from the outside of the mastoid to a target on the cochlea based on registration using landmarks solely on the surface of the skull. Using this methodology, it is impossible to detect if the drill is advancing in the correct direction and that injury of the facial nerve will be avoided. To overcome this problem, a tool localization method based on drilling process information is proposed. The algorithm estimates the pose of a robot-guided surgical tool during a drilling task based on the correlation of the observed axial drilling force and the heterogeneous bone density in the mastoid extracted from 3-D image data. We present here one possible implementation of this method tested on ten tunnels drilled into three human cadaver specimens where an average tool localization accuracy of 0.29 mm was observed.

Fully Automatic Segmentation of AP Pelvis X-rays via Random Forest Regression and Hierarchical Sparse Shape Composition

Knowledge of landmarks and contours in anteroposterior (AP) pelvis X-rays is invaluable for computer aided diagnosis, hip surgery planning and image-guided interventions. This paper presents a fully automatic and robust approach for landmarking and segmentation of both pelvis and femur in a conventional AP X-ray. Our approach is based on random forest regression and hierarchical sparse shape composition. Experiments conducted on 436 clinical AP pelvis x-rays show that our approach achieves an average point-to-curve error around 1.3 mm for femur and 2.2 mm for pelvis, both with success rates around 98%. Compared to existing methods, our approach exhibits better performance in both the robustness and the accuracy.

Semiautomatic Cochleostomy Target and Insertion Trajectory Planning for Minimally Invasive Cochlear Implantation

A major component of minimally invasive cochlear implantation is atraumatic scala tympani (ST) placement of the electrode array. This work reports on a semiautomatic planning paradigm that uses anatomical landmarks and cochlear surface models for cochleostomy target and insertion trajectory computation. The method was validated in a human whole head cadaver model (n = 10 ears). Cochleostomy targets were generated from an automated script and used for consecutive planning of a direct cochlear access (DCA) drill trajectory from the mastoid surface to the inner ear. An image-guided robotic system was used to perform both, DCA and cochleostomy drilling. Nine of 10 implanted specimens showed complete ST placement. One case of scala vestibuli insertion occurred due to a registration/drilling error of 0.79 mm. The presented approach indicates that a safe cochleostomy target and insertion trajectory can be planned using conventional clinical imaging modalities, which lack sufficient resolution to identify the basilar membrane.

Dissecting abdominal aortic aneurysm in Ang II-infused mice: suprarenal branch ruptures and apparent luminal dilatation.

AIMS In this work, we provide novel insight into the morphology of dissecting abdominal aortic aneurysms in angiotensin II-infused mice. We demonstrate why they exhibit a large variation in shape and, unlike their human counterparts, are located suprarenally rather than infrarenally. METHODS AND RESULTS We combined synchrotron-based, ultra-high resolution ex vivo imaging (phase contrast X-Ray tomographic microscopy) with in vivo imaging (high-frequency ultrasound and contrast-enhanced micro-CT) and image-guided histology. In all mice, we observed a tear in the tunica media of the abdominal aorta near the ostium of the celiac artery. Independently we found that, unlike the gradual luminal expansion typical for human aneurysms, the outer diameter increase of angiotensin II-induced dissecting aneurysms in mice was related to one or several intramural haematomas. These were caused by ruptures of the tunica media near the ostium of small suprarenal side branches, which had never been detected by the established small animal imaging techniques. The tear near the celiac artery led to apparent luminal dilatation, while the intramural haematoma led to a dissection of the tunica adventitia on the left suprarenal side of the aorta. The number of ruptured branches was higher in those aneurysms that extended into the thoracic aorta, which explained the observed variability in aneurysm shape. CONCLUSION Our results are the first to describe apparent luminal dilatation, suprarenal branch ruptures, and intramural haematoma formation in dissecting abdominal aortic aneurysms in mice. Moreover, we validate and demonstrate the vast potential of phase contrast X-ray tomographic microscopy in cardiovascular small animal applications.

Fully automatic segmentation of AP pelvis X-rays via random forest regression with efficient feature selection and hierarchical sparse shape composition

In clinical practice, traditional X-ray radiography is widely used, and knowledge of landmarks and contours in anteroposterior (AP) pelvis X-rays is invaluable for computer aided diagnosis, hip surgery planning and image-guided interventions. This paper presents a fully automatic approach for landmark detection and shape segmentation of both pelvis and femur in conventional AP X-ray images. Our approach is based on the framework of landmark detection via Random Forest (RF) regression and shape regularization via hierarchical sparse shape composition. We propose a visual feature FL-HoG (Flexible- Level Histogram of Oriented Gradients) and a feature selection algorithm based on trace radio optimization to improve the robustness and the efficacy of RF-based landmark detection. The landmark detection result is then used in a hierarchical sparse shape composition framework for shape regularization. Finally, the extracted shape contour is fine-tuned by a post-processing step based on low level image features. The experimental results demonstrate that our feature selection algorithm reduces the feature dimension in a factor of 40 and improves both training and test efficiency. Further experiments conducted on 436 clinical AP pelvis X-rays show that our approach achieves an average point-to-curve error around 1.2 mm for femur and 1.9 mm for pelvis.

Endovascular Treatment of Acute Stroke: Evolution and Selection of Techniques and Instruments Based on Thrombus Imaging.

Mechanical thrombectomy provides higher recanalization rates than intravenous or intra-arterial thrombolysis. Finally this has been shown to translate into improved clinical outcome in six multicentric randomized controlled trials. However, within cohorts the clinical outcomes may vary, depending on the endovascular techniques applied. Systems aiming mainly for thrombus fragmentation and lacking a protection against distal embolization have shown disappointing results when compared to recent stent-retriever studies or even to historical data on local arterial fibrinolysis. Procedure-related embolic events are usually graded as adverse events in interventional neuroradiology. In stroke, however, the clinical consequences of secondary emboli have so far mostly been neglected and attributed to progression of the stroke itself. We summarize the evolution of instruments and techniques for endovascular, image-guided, microneurosurgical recanalization in acute stroke, and discuss how to avoid procedure-related embolic complications.