Hierarchical Markov Random Fields Applied to Model Soft Tissue Deformations on Graphics Hardware

Many methodologies dealing with prediction or simulation of soft tissue deformations on medical image data require preprocessing of the data in order to produce a different shape representation that complies with standard methodologies, such as mass–spring networks, finite element method s (FEM). On the other hand, methodologies working directly on the image space normally do not take into account mechanical behavior of tissues and tend to lack physics foundations driving soft tissue deformations. This chapter presents a method to simulate soft tissue deformations based on coupled concepts from image analysis and mechanics theory. The proposed methodology is based on a robust stochastic approach that takes into account material properties retrieved directly from the image, concepts from continuum mechanics and FEM. The optimization framework is solved within a hierarchical Markov random field (HMRF) which is implemented on the graphics processor unit (GPU See Graphics processing unit ).

Estimation of tool pose based on force-density correlation during robotic drilling

The application of image-guided systems with or without support by surgical robots relies on the accuracy of the navigation process, including patient-to-image registration. The surgeon must carry out the procedure based on the information provided by the navigation system, usually without being able to verify its correctness beyond visual inspection. Misleading surrogate parameters such as the fiducial registration error are often used to describe the success of the registration process, while a lack of methods describing the effects of navigation errors, such as those caused by tracking or calibration, may prevent the application of image guidance in certain accuracy-critical interventions. During minimally invasive mastoidectomy for cochlear implantation, a direct tunnel is drilled from the outside of the mastoid to a target on the cochlea based on registration using landmarks solely on the surface of the skull. Using this methodology, it is impossible to detect if the drill is advancing in the correct direction and that injury of the facial nerve will be avoided. To overcome this problem, a tool localization method based on drilling process information is proposed. The algorithm estimates the pose of a robot-guided surgical tool during a drilling task based on the correlation of the observed axial drilling force and the heterogeneous bone density in the mastoid extracted from 3-D image data. We present here one possible implementation of this method tested on ten tunnels drilled into three human cadaver specimens where an average tool localization accuracy of 0.29 mm was observed.

Implementation of a virtual correlative light and transmission electron microscope

In the long run, the widespread use of slide scanners by pathologists requires an adaptation of teaching methods in histology and cytology in order to target these new possibilities of image processing and presentation via the internet. Accordingly, we were looking for a tool with the possibility to teach microscopic anatomy, histology, and cytology of tissue samples which would be able to combine image data from light and electron microscopes independently of microscope suppliers. With the example of a section through the villus of jejunum, we describe here how to process image data from light and electron microscopes in order to get one image-stack which allows a correlation of structures from the microscopic anatomic to the cytological level. With commercially available image-presentation software that we adapted to our needs, we present here a platform which allows for the presentation of this new but also of older material independently of microscope suppliers.

Quantitative 3D strain analysis in analogue experiments: Integration of X-ray computed tomography and digital volume correlation techniques

The combination of scaled analogue experiments, material mechanics, X-ray computed tomography (XRCT) and Digital Volume Correlation techniques (DVC) is a powerful new tool not only to examine the 3 dimensional structure and kinematic evolution of complex deformation structures in scaled analogue experiments, but also to fully quantify their spatial strain distribution and complete strain history. Digital image correlation (DIC) is an important advance in quantitative physical modelling and helps to understand non-linear deformation processes. Optical non-intrusive (DIC) techniques enable the quantification of localised and distributed deformation in analogue experiments based either on images taken through transparent sidewalls (2D DIC) or on surface views (3D DIC). X-ray computed tomography (XRCT) analysis permits the non-destructive visualisation of the internal structure and kinematic evolution of scaled analogue experiments simulating tectonic evolution of complex geological structures. The combination of XRCT sectional image data of analogue experiments with 2D DIC only allows quantification of 2D displacement and strain components in section direction. This completely omits the potential of CT experiments for full 3D strain analysis of complex, non-cylindrical deformation structures. In this study, we apply digital volume correlation (DVC) techniques on XRCT scan data of “solid” analogue experiments to fully quantify the internal displacement and strain in 3 dimensions over time. Our first results indicate that the application of DVC techniques on XRCT volume data can successfully be used to quantify the 3D spatial and temporal strain patterns inside analogue experiments. We demonstrate the potential of combining DVC techniques and XRCT volume imaging for 3D strain analysis of a contractional experiment simulating the development of a non-cylindrical pop-up structure. Furthermore, we discuss various options for optimisation of granular materials, pattern generation, and data acquisition for increased resolution and accuracy of the strain results. Three-dimensional strain analysis of analogue models is of particular interest for geological and seismic interpretations of complex, non-cylindrical geological structures. The volume strain data enable the analysis of the large-scale and small-scale strain history of geological structures.

Three-dimensional printing creates models for surgical planning of aortic valve replacement after previous coronary bypass grafting.

PURPOSE Resternotomy for aortic valve replacement in patients with previous coronary artery bypass grafting and an internal mammary artery graft may be a surgical problem. Thus, we are exploring the effect of using rapid prototyping techniques for surgical planning and intraoperative orientation during aortic valve replacement after previous coronary artery bypass grafting (CABG). DESCRIPTION As a proof of concept, we studied a patient who had undergone CABG 5 years earlier. At that time the patient received a left internal mammary artery graft to the left anterior descending artery and a venous graft to the right coronary artery. Now the patient required aortic valve replacement due to symptomatic aortic valve stenosis. The left internal mammary artery bypass and the right coronary artery bypass were patent and showed good flow in the angiography. The patient was examined by 128-slice computed tomography. The image data were visualized and reconstructed. Afterwards, a replica showing the anatomic structures was fabricated using a rapid prototyping machine. EVALUATION Using data derived from 128-slice computed tomography angiography linked to proprietary software, we were able to create three-dimensional reconstructions of the vascular anatomy after the previous CABG. The models were sterilized and taken to the operating theatre for orientation during the surgical procedure. CONCLUSIONS Stereolithographic replicas are helpful for choosing treatment strategies in surgical planning and for intraoperative orientation during reoperations of patients with previous CABG.

Patient-Specific Semi-Supervised Learning for Postoperative Brain Tumor Segmentation

In contrast to preoperative brain tumor segmentation, the problem of postoperative brain tumor segmentation has been rarely approached so far. We present a fully-automatic segmentation method using multimodal magnetic resonance image data and patient-specific semi-supervised learning. The idea behind our semi-supervised approach is to effectively fuse information from both pre- and postoperative image data of the same patient to improve segmentation of the postoperative image. We pose image segmentation as a classification problem and solve it by adopting a semi-supervised decision forest. The method is evaluated on a cohort of 10 high-grade glioma patients, with segmentation performance and computation time comparable or superior to a state-of-the-art brain tumor segmentation method. Moreover, our results confirm that the inclusion of preoperative MR images lead to a better performance regarding postoperative brain tumor segmentation.

The Multimodal Brain TumorImage Segmentation Benchmark (BRATS)

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients - manually annotated by up to four raters - and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all subregions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

Positron emission mammography in the diagnosis of breast cancer. Is maximum PEM uptake value a valuable threshold for malignant breast cancer detection?

AIM To evaluate the diagnostic value (sensitivity, specificity) of positron emission mammography (PEM) in a single site non-interventional study using the maximum PEM uptake value (PUVmax). PATIENTS, METHODS In a singlesite, non-interventional study, 108 patients (107 women, 1 man) with a total of 151 suspected lesions were scanned with a PEM Flex Solo II (Naviscan) at 90 min p.i. with 3.5 MBq 18F-FDG per kg of body weight. In this ROI(region of interest)-based analysis, maximum PEM uptake value (PUV) was determined in lesions, tumours (PUVmaxtumour), benign lesions (PUVmaxnormal breast) and also in healthy tissues on the contralateral side (PUVmaxcontralateral breast). These values were compared and contrasted. In addition, the ratios of PUVmaxtumour / PUVmaxcontralateral breast and PUVmaxnormal breast / PUVmaxcontralateral breast were compared. The image data were interpreted independently by two experienced nuclear medicine physicians and compared with histology in cases of suspected carcinoma. RESULTS Based on a criteria of PUV>1.9, 31 out of 151 lesions in the patient cohort were found to be malignant (21%). A mean PUVmaxtumour of 3.78 ± 2.47 was identified in malignant tumours, while a mean PUVmaxnormal breast of 1.17 ± 0.37 was reported in the glandular tissue of the healthy breast, with the difference being statistically significant (p < 0.001). Similarly, the mean ratio between tumour and healthy glandular tissue in breast cancer patients (3.15 ± 1.58) was found to be significantly higher than the ratio for benign lesions (1.17 ± 0.41, p < 0.001). CONCLUSION PEM is capable of differentiating breast tumours from benign lesions with 100% sensitivity along with a high specificity of 96%, when a threshold of PUVmax >1.9 is applied.

Natural History of Geographic Atrophy Progression Secondary to Age-Related Macular Degeneration (Geographic Atrophy Progression Study).

PURPOSE The Geographic Atrophy Progression (GAP) study was designed to assess the rate of geographic atrophy (GA) progression and to identify prognostic factors by measuring the enlargement of the atrophic lesions using fundus autofluorescence (FAF) and color fundus photography (CFP). DESIGN Prospective, multicenter, noninterventional natural history study. PARTICIPANTS A total of 603 participants were enrolled in the study; 413 of those had gradable lesion data from FAF or CFP, and 321 had gradable lesion data from both FAF and CFP. METHODS Atrophic lesion areas were measured by FAF and CFP to assess lesion progression over time. Lesion size assessments and best-corrected visual acuity (BCVA) were conducted at screening/baseline (day 0) and at 3 follow-up visits: month 6, month 12, and month 18 (or early exit). MAIN OUTCOME MEASURES The GA lesion progression rate in disease subgroups and mean change from baseline visual acuity. RESULTS Mean (standard error) lesion size changes from baseline, determined by FAF and CFP, respectively, were 0.88 (0.1) and 0.78 (0.1) mm(2) at 6 months, 1.85 (0.1) and 1.57 (0.1) mm(2) at 12 months, and 3.14 (0.4) and 3.17 (0.5) mm(2) at 18 months. The mean change in lesion size from baseline to month 12 was significantly greater in participants who had eyes with multifocal atrophic spots compared with those with unifocal spots (P < 0.001) and those with extrafoveal lesions compared with those with foveal lesions (P = 0.001). The mean (standard deviation) decrease in visual acuity was 6.2 ± 15.6 letters for patients with image data available. Atrophic lesions with a diffuse (mean 0.95 mm(2)) or banded (mean 1.01 mm(2)) FAF pattern grew more rapidly by month 6 compared with those with the "none" (mean, 0.13 mm(2)) and focal (mean, 0.36 mm(2)) FAF patterns. CONCLUSIONS Although differences were observed in mean lesion size measurements using FAF imaging compared with CFP, the measurements were highly correlated with one another. Significant differences were found in lesion progression rates in participants stratified by hyperfluorescence pattern subtype. This large GA natural history study provides a strong foundation for future clinical trials.

Automated detection of fiducial screws from CT/DVT volume data for image-guided ENT surgery

This paper presents an automated solution for precise detection of fiducial screws from three-dimensional (3D) Computerized Tomography (CT)/Digital Volume Tomography (DVT) data for image-guided ENT surgery. Unlike previously published solutions, we regard the detection of the fiducial screws from the CT/DVT volume data as a pose estimation problem. We thus developed a model-based solution. Starting from a user-supplied initialization, our solution detects the fiducial screws by iteratively matching a computer aided design (CAD) model of the fiducial screw to features extracted from the CT/DVT data. We validated our solution on one conventional CT dataset and on five DVT volume datasets, resulting in a total detection of 24 fiducial screws. Our experimental results indicate that the proposed solution achieves much higher reproducibility and precision than the manual detection. Further comparison shows that the proposed solution produces better results on the DVT dataset than on the conventional CT dataset.

Evaluation of a portable image overlay projector for the visualisation of surgical navigation data: phantom studies

Presenting visual feedback for image-guided surgery on a monitor requires the surgeon to perform time-consuming comparisons and diversion of sight and attention away from the patient. Deficiencies in previously developed augmented reality systems for image-guided surgery have, however, prevented the general acceptance of any one technique as a viable alternative to monitor displays. This work presents an evaluation of the feasibility and versatility of a novel augmented reality approach for the visualisation of surgical planning and navigation data. The approach, which utilises a portable image overlay device, was evaluated during integration into existing surgical navigation systems and during application within simulated navigated surgery scenarios.

Automatic X-ray landmark detection and shape segmentation via data-driven joint estimation of image displacements

In this paper, we propose a new method for fully-automatic landmark detection and shape segmentation in X-ray images. To detect landmarks, we estimate the displacements from some randomly sampled image patches to the (unknown) landmark positions, and then we integrate these predictions via a voting scheme. Our key contribution is a new algorithm for estimating these displacements. Different from other methods where each image patch independently predicts its displacement, we jointly estimate the displacements from all patches together in a data driven way, by considering not only the training data but also geometric constraints on the test image. The displacements estimation is formulated as a convex optimization problem that can be solved efficiently. Finally, we use the sparse shape composition model as the a priori information to regularize the landmark positions and thus generate the segmented shape contour. We validate our method on X-ray image datasets of three different anatomical structures: complete femur, proximal femur and pelvis. Experiments show that our method is accurate and robust in landmark detection, and, combined with the shape model, gives a better or comparable performance in shape segmentation compared to state-of-the art methods. Finally, a preliminary study using CT data shows the extensibility of our method to 3D data.