A Portable Image Overlay Projection Device for Computer-Aided Open Liver Surgery

Image overlay projection is a form of augmented reality that allows surgeons to view underlying anatomical structures directly on the patient surface. It improves intuitiveness of computer-aided surgery by removing the need for sight diversion between the patient and a display screen and has been reported to assist in 3-D understanding of anatomical structures and the identification of target and critical structures. Challenges in the development of image overlay technologies for surgery remain in the projection setup. Calibration, patient registration, view direction, and projection obstruction remain unsolved limitations to image overlay techniques. In this paper, we propose a novel, portable, and handheld-navigated image overlay device based on miniature laser projection technology that allows images of 3-D patient-specific models to be projected directly onto the organ surface intraoperatively without the need for intrusive hardware around the surgical site. The device can be integrated into a navigation system, thereby exploiting existing patient registration and model generation solutions. The position of the device is tracked by the navigation system’s position sensor and used to project geometrically correct images from any position within the workspace of the navigation system. The projector was calibrated using modified camera calibration techniques and images for projection are rendered using a virtual camera defined by the projectors extrinsic parameters. Verification of the device’s projection accuracy concluded a mean projection error of 1.3 mm. Visibility testing of the projection performed on pig liver tissue found the device suitable for the display of anatomical structures on the organ surface. The feasibility of use within the surgical workflow was assessed during open liver surgery. We show that the device could be quickly and unobtrusively deployed within the sterile environment.

The foundations of computer assisted surgery

Using navigation systems in general orthopaedic surgery and, in particular, knee replacement is becoming more and more accepted. This paper describes the basic technological concepts of modern computer assisted surgical systems. It explains the variation in currently available systems and outlines research activities that will potentially influence future products. In general, each navigation system is defined by three components: (1) the therapeutic object is the anatomical structure that is operated on using the navigation system, (2) the virtual object represents an image of the therapeutic object, with radiological images or computer generated models potentially being used, and (3) last but not least, the navigator acquires the spatial position and orientation of instruments and anatomy thus providing the necessary data to replay surgical action in real-time on the navigation system's screen.

Computer-assisted, fluoroscopy-based ventral spondylodesis of thoracolumbar fractures

OBJECTIVE: To design and evaluate a novel computer-assisted, fluoroscopy-based planning and navigation system for minimally invasive ventral spondylodesis of thoracolumbar fractures. MATERIALS AND METHODS: Instruments and an image intensifier are tracked with the SurgiGATE navigation system (Praxim-Medivision). Two fluoroscopic images, one acquired from anterior-posterior (AP) direction and the other from lateral-medial (LM) direction, are used for the complete procedure of planning and navigation. Both of them are calibrated with a custom-made software to recover their projection geometry and to co-register them to a common patient reference coordinate system, which is established by attaching an opto-electronically trackable dynamic reference base (DRB) on the operated vertebra. A bi-planar landmark reconstruction method is used to acquire deep-seated anatomical landmarks such that an intraoperative planning of graft bed can be interactively done. Finally, surgical actions such as the placement of the stabilization devices and the formation of the graft bed using a custom-made chisel are visualized to the surgeon by superimposing virtual instrument representations onto the acquired images. The distance between the instrument tip and each wall of the planned graft bed are calculated on the fly and presented to the surgeon so that the surgeon could formalize the graft bed exactly according to his/her plan. RESULTS: Laboratory studies on phantom and on 27 plastic vertebras demonstrate the high precision of the proposed navigation system. Compared with CT-based measurement, a mean error of 1.0 mm with a standard deviation of 0.1 mm was found. CONCLUSIONS: The proposed computer assisted, fluoroscopy-based planning and navigation system promises to increase the accuracy and reliability of minimally invasive ventral spondylodesis of thoracolumbar fractures.

Non-photorealistic rendering of virtual implant models for computer-assisted fluoroscopy-based surgical procedures

Surgical navigation systems visualize the positions and orientations of surgical instruments and implants as graphical overlays onto a medical image of the operated anatomy on a computer monitor. The orthopaedic surgical navigation systems could be categorized according to the image modalities that are used for the visualization of surgical action. In the so-called CT-based systems or 'surgeon-defined anatomy' based systems, where a 3D volume or surface representation of the operated anatomy could be constructed from the preoperatively acquired tomographic data or through intraoperatively digitized anatomy landmarks, a photorealistic rendering of the surgical action has been identified to greatly improve usability of these navigation systems. However, this may not hold true when the virtual representation of surgical instruments and implants is superimposed onto 2D projection images in a fluoroscopy-based navigation system due to the so-called image occlusion problem. Image occlusion occurs when the field of view of the fluoroscopic image is occupied by the virtual representation of surgical implants or instruments. In these situations, the surgeon may miss part of the image details, even if transparency and/or wire-frame rendering is used. In this paper, we propose to use non-photorealistic rendering to overcome this difficulty. Laboratory testing results on foamed plastic bones during various computer-assisted fluoroscopybased surgical procedures including total hip arthroplasty and long bone fracture reduction and osteosynthesis are shown.

Cholesterol granuloma of the petrous apex: benefit of computer-aided surgery

The following is an analysis of the role of computer aided surgery by infralabyrinthine-subcochlear approach to the petrous apex for cholesterol granulomas with hearing preservation. In a retrospective case review from 1996 to 2008 six patients were analysed in our tertiary referral centre, otorhinolaryngology outpatient clinic. Excellent intraoperative localisation of the carotid artery, facial nerve and the entrance into the cholesterol cyst of the bone by means of the navigation system was seen. Additionally, the operation time decreased from an initial 4 h down to 2 h. The application of computer-aided surgery allows intraoperative monitoring of the position of the tip of the microsurgical instruments in case of a rare disease and in the delicate area of the petrous apex giving a high security level.

Accuracy of navigated surgery of the pelvis after surface matching with an a-mode ultrasound probe

Computer-aided surgery (CAS) allows for real-time intraoperative feedback resulting in increased accuracy, while reducing intraoperative radiation. CAS is especially useful for the treatment of certain pelvic ring fractures, which necessitate the precise placement of screws. Flouroscopy-based CAS modules have been developed for many orthopedic applications. The integration of the isocentric flouroscope even enables navigation using intraoperatively acquired three-dimensional (3D) data, though the scan volume and imaging quality are limited. Complicated and comprehensive pathologies in regions like the pelvis can necessitate a CT-based navigation system because of its larger field of view. To be accurate, the patient's anatomy must be registered and matched with the virtual object (CT data). The actual precision within the region of interest depends on the area of the bone where surface matching is performed. Conventional surface matching with a solid pointer requires extensive soft tissue dissection. This contradicts the primary purpose of CAS as a minimally invasive alternative to conventional surgical techniques. We therefore integrated an a-mode ultrasound pointer into the process of surface matching for pelvic surgery and compared it to the conventional method. Accuracy measurements were made in two pelvic models: a foam model submerged in water and one with attached porcine muscle tissue. Three different tissue depths were selected based on CT scans of 30 human pelves. The ultrasound pointer allowed for registration of virtually any point on the pelvis. This method of surface matching could be successfully integrated into CAS of the pelvis.

Estimation of tool pose based on force-density correlation during robotic drilling

The application of image-guided systems with or without support by surgical robots relies on the accuracy of the navigation process, including patient-to-image registration. The surgeon must carry out the procedure based on the information provided by the navigation system, usually without being able to verify its correctness beyond visual inspection. Misleading surrogate parameters such as the fiducial registration error are often used to describe the success of the registration process, while a lack of methods describing the effects of navigation errors, such as those caused by tracking or calibration, may prevent the application of image guidance in certain accuracy-critical interventions. During minimally invasive mastoidectomy for cochlear implantation, a direct tunnel is drilled from the outside of the mastoid to a target on the cochlea based on registration using landmarks solely on the surface of the skull. Using this methodology, it is impossible to detect if the drill is advancing in the correct direction and that injury of the facial nerve will be avoided. To overcome this problem, a tool localization method based on drilling process information is proposed. The algorithm estimates the pose of a robot-guided surgical tool during a drilling task based on the correlation of the observed axial drilling force and the heterogeneous bone density in the mastoid extracted from 3-D image data. We present here one possible implementation of this method tested on ten tunnels drilled into three human cadaver specimens where an average tool localization accuracy of 0.29 mm was observed.

Comparison of freehand-navigated and aiming device-navigated targeting of liver lesions

BACKGROUND Accurate needle placement is crucial for the success of percutaneous radiological needle interventions. We compared three guiding methods using an optical-based navigation system: freehand, using a stereotactic aiming device and active depth control, and using a stereotactic aiming device and passive depth control. METHODS For each method, 25 punctures were performed on a non-rigid phantom. Five 1 mm metal screws were used as targets. Time requirements were recorded, and target positioning errors (TPE) were measured on control scans as the distance between needle tip and target. RESULTS Time requirements were reduced using the aiming device and passive depth control. The Euclidian TPE was similar for each method (4.6 ± 1.2-4.9 ± 1.7 mm). However, the lateral component was significantly lower when an aiming device was used (2.3 ± 1.3-2.8 ± 1.6 mm with an aiming device vs 4.2 ± 2.0 mm without). DISCUSSION Using an aiming device may increase the lateral accuracy of navigated needle insertion.

Cone-beam computed tomography-guided stereotactic liver punctures: a phantom study

PURPOSE Images from computed tomography (CT), combined with navigation systems, improve the outcomes of local thermal therapies that are dependent on accurate probe placement. Although the usage of CT is desired, its availability for time-consuming radiological interventions is limited. Alternatively, three-dimensional images from C-arm cone-beam CT (CBCT) can be used. The goal of this study was to evaluate the accuracy of navigated CBCT-guided needle punctures, controlled with CT scans. METHODS Five series of five navigated punctures were performed on a nonrigid phantom using a liver specific navigation system and CBCT volumetric dataset for planning and navigation. To mimic targets, five titanium screws were fixed to the phantom. Target positioning accuracy (TPECBCT) was computed from control CT scans and divided into lateral and longitudinal components. Additionally, CBCT-CT guidance accuracy was deducted by performing CBCT-to-CT image coregistration and measuring TPECBCT-CT from fused datasets. Image coregistration was evaluated using fiducial registration error (FRECBCT-CT) and target registration error (TRECBCT-CT). RESULTS Positioning accuracies in lateral directions pertaining to CBCT (TPECBCT = 2.1 ± 1.0 mm) were found to be better to those achieved from previous study using CT (TPECT = 2.3 ± 1.3 mm). Image coregistration error was 0.3 ± 0.1 mm, resulting in an average TRE of 2.1 ± 0.7 mm (N = 5 targets) and average Euclidean TPECBCT-CT of 3.1 ± 1.3 mm. CONCLUSIONS Stereotactic needle punctures might be planned and performed on volumetric CBCT images and controlled with multidetector CT with positioning accuracy higher or similar to those performed using CT scanners.

Angiographic C-arm CT- versus MDCT-guided stereotactic punctures of liver lesions: nonrigid phantom study

OBJECTIVE Angiographic C-arm CT may allow performing percutaneous stereotactic tumor ablations in the interventional radiology suite. Our purpose was to evaluate the accuracy of using C-arm CT for single and multimodality image fusions and to compare the targeting accuracy of liver lesions with the reference standard of MDCT. MATERIALS AND METHODS C-arm CT and MDCT scans were obtained of a nonrigid rapid prototyping liver phantom containing five 1-mm targets that were placed under skin-simulating deformable plastic foam. Target registration errors of image fusion were evaluated for single-modality and multimodality image fusions. A navigation system and stereotactic aiming device were used to evaluate target positioning errors on postinterventional scans with the needles in place fused with the C-arm CT or MDCT planning images. RESULTS Target registration error of the image fusion showed no significant difference (p > 0.05) between both modalities. In five series with a total of 25 punctures for each modality, the lateral target positioning error (i.e., the lateral distance between the needle tip and the planned trajectory) was similar for C-arm CT (mean [± SD], 1.6 ± 0.6 mm) and MDCT (1.82 ± .97 mm) (p = 0.33). CONCLUSION In a nonrigid liver phantom, angiographic C-arm CT may provide similar image fusion accuracy for comparison of intra- and postprocedure control images with the planning images and enables stereotactic targeting accuracy similar to that of MDCT.

Laparoscopic computer-navigated ablation of liver metastases

Objective In order to benefit from the obvious advantages of minimally invasive liver surgery there is a need to develop high precision tools for intraoperative anatomical orientation, navigation and safety control. In a pilot study we adapted a newly developed system for computer-assisted liver surgery (CALS) in terms of accuracy and technical feasibility to the specific requirements of laparoscopy. Here, we present practical aspects related to laparoscopic computer assisted liver surgery (LCALS). Methods Our video relates to a patient presenting with 3 colorectal liver metastases in Seg. II, III and IVa who was selected in an appropriate oncological setting for LCALS using the CAScination system combined with 3D MEVIS reconstruction. After minimal laparoscopic mobilization of the liver, a 4- landmark registration method was applied to enable navigation. Placement of microwave needles was performed using the targeting module of the navigation system and correct needle positioning was confirmed by intraoperative sonography. Ablation of each lesion was carried out by application of microwave energy at 100 Watts for 1 minute. Results To acquire an accurate (less 0.5 cm) registration, 4 registration cycles were necessary. In total, seven minutes were required to accomplish precise registration. Successful ablation with complete response in all treated areas was assessed by intraoperative sonography and confirmed by postoperative CT scan. Conclusions This teaching video demonstrates the theoretical and practical key points of LCALS with a special emphasis on preoperative planning, intraoperative registration and accuracy testing by laparoscopic methodology. In contrast to mere ultrasound-guided ablation of liver lesions, LCALS offers a more dimensional targeting and higher safety control. This is currently also in routine use to treat vanishing lesions and other difficult to target focal lesions within the liver.

Computer planned, image-guided combined resection and ablation for bilobar colorectal liver metastases

For patients with extensive bilobar colorectal liver metastases (CRLM), initial surgery may not be feasible and a multimodal approach including microwave ablation (MWA) provides the only chance for prolonged survival. Intraoperative navigation systems may improve the accuracy of ablation and surgical resection of so-called "vanishing lesions", ultimately improving patient outcome. Clinical application of intraoperative navigated liver surgery is illustrated in a patient undergoing combined resection/MWA for multiple, synchronous, bilobar CRLM. Regular follow-up with computed tomography (CT) allowed for temporal development of the ablation zones. Of the ten lesions detected in a preoperative CT scan, the largest lesion was resected and the others were ablated using an intraoperative navigation system. Twelve months post-surgery a new lesion (Seg IVa) was detected and treated by trans-arterial embolization. Nineteen months post-surgery new liver and lung metastases were detected and a palliative chemotherapy started. The patient passed away four years after initial diagnosis. For patients with extensive CRLM not treatable by standard surgery, navigated MWA/resection may provide excellent tumor control, improving longer-term survival. Intraoperative navigation systems provide precise, real-time information to the surgeon, aiding the decision-making process and substantially improving the accuracy of both ablation and resection. Regular follow-ups including 3D modeling allow for early discrimination between ablation zones and recurrent tumor lesions.

Preliminary Study on Finite Element Simulation for Optimizing Acetabulum Reorientation after Periacetabular Osteotomy

Periacetabular osteotomy (PAO) is an effective approach for surgical treatment of hip dysplasia. The aim of PAO is to increase acetabular coverage of the femoral head and to reduce contact pressures by reorienting the acetabulum fragment after PAO. The success of PAO significantly depends on the surgeon’s experience. Previously, we have developed a computer-assisted planning and navigation system for PAO, which allows for not only quantifying the 3D hip morphology for a computer-assisted diagnosis of hip dysplasia but also a virtual PAO surgical planning and simulation. In this paper, based on this previously developed PAO planning and navigation system, we developed a 3D finite element (FE) model to investigate the optimal acetabulum reorientation after PAO. Our experimental results showed that an optimal position of the acetabulum can be achieved that maximizes contact area and at the same time minimizes peak contact pressure in pelvic and femoral cartilages. In conclusion, our computer-assisted planning and navigation system with FE modeling can be a promising tool to determine the optimal PAO planning strategy.

A new system for computer-aided preoperative planning and intraoperative navigation during corrective jaw surgery

A new system for computer-aided corrective surgery of the jaws has been developed and introduced clinically. It combines three-dimensional (3-D) surgical planning with conventional dental occlusion planning. The developed software allows simulating the surgical correction on virtual 3-D models of the facial skeleton generated from computed tomography (CT) scans. Surgery planning and simulation include dynamic cephalometry, semi-automatic mirroring, interactive cutting of bone and segment repositioning. By coupling the software with a tracking system and with the help of a special registration procedure, we are able to acquire dental occlusion plans from plaster model mounts. Upon completion of the surgical plan, the setup is used to manufacture positioning splints for intraoperative guidance. The system provides further intraoperative assistance with the help of a display showing jaw positions and 3-D positioning guides updated in real time during the surgical procedure. The proposed approach offers the advantages of 3-D visualization and tracking technology without sacrificing long-proven cast-based techniques for dental occlusion evaluation. The system has been applied on one patient. Throughout this procedure, we have experienced improved assessment of pathology, increased precision, and augmented control.