3D Conversion using Vanishing Points and Image Warping

We describe a user assisted technique for 3D stereo conversion from 2D images. Our approach exploits the geometric structure of perspective images including vanishing points. We allow a user to indicate lines, planes, and vanishing points in the input image, and directly employ these as constraints in an image warping framework to produce a stereo pair. By sidestepping explicit construction of a depth map, our approach is applicable to more general scenes and avoids potential artifacts of depth-image-based rendering. Our method is most suitable for scenes with large scale structures such as buildings.

Endoscopic Image Overlay for the Targeting of Hidden Anatomy in Laparoscopic Visceral Surgery

Limitations associated with the visual information provided to surgeons during laparoscopic surgery increases the difficulty of procedures and thus, reduces clinical indications and increases training time. This work presents a novel augmented reality visualization approach that aims to improve visual data supplied for the targeting of non visible anatomical structures in laparoscopic visceral surgery. The approach aims to facilitate the localisation of hidden structures with minimal damage to surrounding structures and with minimal training requirements. The proposed augmented reality visualization approach incorporates endoscopic images overlaid with virtual 3D models of underlying critical structures in addition to targeting and depth information pertaining to targeted structures. Image overlay was achieved through the implementation of camera calibration techniques and integration of the optically tracked endoscope into an existing image guidance system for liver surgery. The approach was validated in accuracy, clinical integration and targeting experiments. Accuracy of the overlay was found to have a mean value of 3.5 mm ± 1.9 mm and 92.7% of targets within a liver phantom were successfully located laparoscopically by non trained subjects using the approach.

In vitro accuracy evaluation of image-guided robot system for direct cochlear access

HYPOTHESIS A previously developed image-guided robot system can safely drill a tunnel from the lateral mastoid surface, through the facial recess, to the middle ear, as a viable alternative to conventional mastoidectomy for cochlear electrode insertion. BACKGROUND Direct cochlear access (DCA) provides a minimally invasive tunnel from the lateral surface of the mastoid through the facial recess to the middle ear for cochlear electrode insertion. A safe and effective tunnel drilled through the narrow facial recess requires a highly accurate image-guided surgical system. Previous attempts have relied on patient-specific templates and robotic systems to guide drilling tools. In this study, we report on improvements made to an image-guided surgical robot system developed specifically for this purpose and the resulting accuracy achieved in vitro. MATERIALS AND METHODS The proposed image-guided robotic DCA procedure was carried out bilaterally on 4 whole head cadaver specimens. Specimens were implanted with titanium fiducial markers and imaged with cone-beam CT. A preoperative plan was created using a custom software package wherein relevant anatomical structures of the facial recess were segmented, and a drill trajectory targeting the round window was defined. Patient-to-image registration was performed with the custom robot system to reference the preoperative plan, and the DCA tunnel was drilled in 3 stages with progressively longer drill bits. The position of the drilled tunnel was defined as a line fitted to a point cloud of the segmented tunnel using principle component analysis (PCA function in MatLab). The accuracy of the DCA was then assessed by coregistering preoperative and postoperative image data and measuring the deviation of the drilled tunnel from the plan. The final step of electrode insertion was also performed through the DCA tunnel after manual removal of the promontory through the external auditory canal. RESULTS Drilling error was defined as the lateral deviation of the tool in the plane perpendicular to the drill axis (excluding depth error). Errors of 0.08 ± 0.05 mm and 0.15 ± 0.08 mm were measured on the lateral mastoid surface and at the target on the round window, respectively (n =8). Full electrode insertion was possible for 7 cases. In 1 case, the electrode was partially inserted with 1 contact pair external to the cochlea. CONCLUSION The purpose-built robot system was able to perform a safe and reliable DCA for cochlear implantation. The workflow implemented in this study mimics the envisioned clinical procedure showing the feasibility of future clinical implementation.

Effect of irradiation distance on image contrast in epi-optoacoustic imaging of human volunteers

In combined clinical optoacoustic (OA) and ultrasound (US) imaging, epi-mode irradiation and detection integrated into one single probe offers flexible imaging of the human body. The imaging depth in epi-illumination is, however, strongly affected by clutter. As shown in previous phantom experiments, the location of irradiation plays an important role in clutter generation. We investigated the influence of the irradiation geometry on the local image contrast of clinical images, by varying the separation distance between the irradiated area and the acoustic imaging plane of a linear ultrasound transducer in an automated scanning setup. The results for different volunteers show that the image contrast can be enhanced on average by 25% and locally by more than a factor of two, when the irradiated area is slightly separated from the probe. Our findings have an important impact on the design of future optoacoustic probes for clinical application.

Image Space Adaptive Rendering

In this thesis, we develop an adaptive framework for Monte Carlo rendering, and more specifically for Monte Carlo Path Tracing (MCPT) and its derivatives. MCPT is attractive because it can handle a wide variety of light transport effects, such as depth of field, motion blur, indirect illumination, participating media, and others, in an elegant and unified framework. However, MCPT is a sampling-based approach, and is only guaranteed to converge in the limit, as the sampling rate grows to infinity. At finite sampling rates, MCPT renderings are often plagued by noise artifacts that can be visually distracting. The adaptive framework developed in this thesis leverages two core strategies to address noise artifacts in renderings: adaptive sampling and adaptive reconstruction. Adaptive sampling consists in increasing the sampling rate on a per pixel basis, to ensure that each pixel value is below a predefined error threshold. Adaptive reconstruction leverages the available samples on a per pixel basis, in an attempt to have an optimal trade-off between minimizing the residual noise artifacts and preserving the edges in the image. In our framework, we greedily minimize the relative Mean Squared Error (rMSE) of the rendering by iterating over sampling and reconstruction steps. Given an initial set of samples, the reconstruction step aims at producing the rendering with the lowest rMSE on a per pixel basis, and the next sampling step then further reduces the rMSE by distributing additional samples according to the magnitude of the residual rMSE of the reconstruction. This iterative approach tightly couples the adaptive sampling and adaptive reconstruction strategies, by ensuring that we only sample densely regions of the image where adaptive reconstruction cannot properly resolve the noise. In a first implementation of our framework, we demonstrate the usefulness of our greedy error minimization using a simple reconstruction scheme leveraging a filterbank of isotropic Gaussian filters. In a second implementation, we integrate a powerful edge aware filter that can adapt to the anisotropy of the image. Finally, in a third implementation, we leverage auxiliary feature buffers that encode scene information (such as surface normals, position, or texture), to improve the robustness of the reconstruction in the presence of strong noise.

Optimized aperture shapes for depth estimation

The finite depth of field of a real camera can be used to estimate the depth structure of a scene. The distance of an object from the plane in focus determines the defocus blur size. The shape of the blur depends on the shape of the aperture. The blur shape can be designed by masking the main lens aperture. In fact, aperture shapes different from the standard circular aperture give improved accuracy of depth estimation from defocus blur. We introduce an intuitive criterion to design aperture patterns for depth from defocus. The criterion is independent of a specific depth estimation algorithm. We formulate our design criterion by imposing constraints directly in the data domain and optimize the amount of depth information carried by blurred images. Our criterion is a quadratic function of the aperture transmission values. As such, it can be numerically evaluated to estimate optimized aperture patterns quickly. The proposed mask optimization procedure is applicable to different depth estimation scenarios. We use it for depth estimation from two images with different focus settings, for depth estimation from two images with different aperture shapes as well as for depth estimation from a single coded aperture image. In this work we show masks obtained with this new evaluation criterion and test their depth discrimination capability using a state-of-the-art depth estimation algorithm.

Enhanced-depth optical coherence tomography for imaging horizontal rectus muscles in Graves' orbitopathy.

PURPOSE Graves' orbitopathy (GO) is an extraocular eye disease with symptoms ranging from minor discomfort from dry eyes to strabismus and visual loss. One of the hallmarks of active GO is visible hyperemia at the insertion of the extraocular muscles. The aim of the present study was to evaluate the use of enhanced-depth imaging spectral domain anterior segment optical coherence tomography (EDI SD AS-OCT) for detecting pathological changes in horizontal recti muscles of patients with GO. METHODS Prospective cross sectional study of 27 eyes. Only women were included. EDI AS-OCT was used to measure the thickness of the tendons of the horizontal recti muscles in a predefined area in patients with GO and healthy controls. RESULTS EDI AS-OCT was able to image the tendons of the horizontal recti muscles in both healthy controls and patients suffering from GO. The mean thickness of the medial rectus muscle (MR) tendon was 256.4 μm [±17.13 μm standard deviation (SD)] in the GO group and, therefore, significantly thicker (p = 0.046) than in the healthy group which had a mean thickness of 214.7 μm (±5.516 μm SD). There was no significant difference in the mean thickness of the tendon of the lateral recti muscles (LRs) between these groups. CONCLUSION This is the first report showing that EDI AS-OCT is suitable to detect swelling at the insertion site of the MR muscle in GO. MR tendon thickness may be a useful parameter to monitor activity in these patients.

Enhanced depth imaging optical coherence tomography of congenital cavitary optic disc anomaly (CODA).

AIM To report the finding of extension of the 4th hyper-reflective band and retinal tissue into the optic disc in patients with cavitary optic disc anomalies (CODAs). METHODS In this observational study, 10 patients (18 eyes) with sporadic or autosomal dominant CODA were evaluated with enhanced depth imaging optical coherence tomography (EDI-OCT) and colour fundus images for the presence of 4th hyper-reflective band extension into the optic disc. RESULTS Of 10 CODA patients (18 eyes), five patients (8 eyes) showed a definite 4th hyper-reflective band (presumed retinal pigment epithelium (RPE)) extension into the optic disc. In these five patients (seven eyes), the inner retinal layers also extended with the 4th hyper-reflective band into the optic disc. Best corrected visual acuity ranged from 20/20 to 20/200. In three patients (four eyes), retinal splitting/schisis was present and in two patients (two eyes), the macula was involved. In all cases, the 4th hyper-reflective band extended far beyond the termination of the choroid into the optic disc. The RPE extension was found either temporally or nasally in areas of optic nerve head excavation, most often adjacent to peripapillary pigment. Compared with eyes without RPE extension, eyes with RPE extension were more myopic (mean dioptres -0.9±2.6 vs -8.8±5, p=0.043). CONCLUSIONS The RPE usually stops near the optic nerve border separated by a border tissue. With CODA, extension of this hyper-reflective band and retinal tissue into the disc is possible and best evaluable using EDI-OCT or analogous image modalities. Whether this is a finding specific for CODA, linked to specific gene loci or is also seen in patients with other optic disc abnormalities needs further evaluation.