Augmented environments for the targeting of hepatic lesions during image-guided robotic liver surgery

BACKGROUND Stereotactic navigation technology can enhance guidance during surgery and enable the precise reproduction of planned surgical strategies. Currently, specific systems (such as the CAS-One system) are available for instrument guidance in open liver surgery. This study aims to evaluate the implementation of such a system for the targeting of hepatic tumors during robotic liver surgery. MATERIAL AND METHODS Optical tracking references were attached to one of the robotic instruments and to the robotic endoscopic camera. After instrument and video calibration and patient-to-image registration, a virtual model of the tracked instrument and the available three-dimensional images of the liver were displayed directly within the robotic console, superimposed onto the endoscopic video image. An additional superimposed targeting viewer allowed for the visualization of the target tumor, relative to the tip of the instrument, for an assessment of the distance between the tumor and the tool for the realization of safe resection margins. RESULTS Two cirrhotic patients underwent robotic navigated atypical hepatic resections for hepatocellular carcinoma. The augmented endoscopic view allowed for the definition of an accurate resection margin around the tumor. The overlay of reconstructed three-dimensional models was also used during parenchymal transection for the identification of vascular and biliary structures. Operative times were 240 min in the first case and 300 min in the second. There were no intraoperative complications. CONCLUSIONS The da Vinci Surgical System provided an excellent platform for image-guided liver surgery with a stable optic and instrumentation. Robotic image guidance might improve the surgeon's orientation during the operation and increase accuracy in tumor resection. Further developments of this technological combination are needed to deal with organ deformation during surgery.

Symmetric inverse consistent nonlinear registration driven by mutual information.

A nonlinear viscoelastic image registration algorithm based on the demons paradigm and incorporating inverse consistent constraint (ICC) is implemented. An inverse consistent and symmetric cost function using mutual information (MI) as a similarity measure is employed. The cost function also includes regularization of transformation and inverse consistent error (ICE). The uncertainties in balancing various terms in the cost function are avoided by alternatively minimizing the similarity measure, the regularization of the transformation, and the ICE terms. The diffeomorphism of registration for preventing folding and/or tearing in the deformation is achieved by the composition scheme. The quality of image registration is first demonstrated by constructing brain atlas from 20 adult brains (age range 30-60). It is shown that with this registration technique: (1) the Jacobian determinant is positive for all voxels and (2) the average ICE is around 0.004 voxels with a maximum value below 0.1 voxels. Further, the deformation-based segmentation on Internet Brain Segmentation Repository, a publicly available dataset, has yielded high Dice similarity index (DSI) of 94.7% for the cerebellum and 74.7% for the hippocampus, attesting to the quality of our registration method.

Image-based vs. mesh-based statistical appearance models of the human femur: Implications for finite element simulations.

Statistical appearance models have recently been introduced in bone mechanics to investigate bone geometry and mechanical properties in population studies. The establishment of accurate anatomical correspondences is a critical aspect for the construction of reliable models. Depending on the representation of a bone as an image or a mesh, correspondences are detected using image registration or mesh morphing. The objective of this study was to compare image-based and mesh-based statistical appearance models of the femur for finite element (FE) simulations. To this aim, (i) we compared correspondence detection methods on bone surface and in bone volume; (ii) we created an image-based and a mesh-based statistical appearance models from 130 images, which we validated using compactness, representation and generalization, and we analyzed the FE results on 50 recreated bones vs. original bones; (iii) we created 1000 new instances, and we compared the quality of the FE meshes. Results showed that the image-based approach was more accurate in volume correspondence detection and quality of FE meshes, whereas the mesh-based approach was more accurate for surface correspondence detection and model compactness. Based on our results, we recommend the use of image-based statistical appearance models for FE simulations of the femur.

A clinically applicable laser-based image-guided system for laparoscopic liver procedures

PURPOSE Laser range scanners (LRS) allow performing a surface scan without physical contact with the organ, yielding higher registration accuracy for image-guided surgery (IGS) systems. However, the use of LRS-based registration in laparoscopic liver surgery is still limited because current solutions are composed of expensive and bulky equipment which can hardly be integrated in a surgical scenario. METHODS In this work, we present a novel LRS-based IGS system for laparoscopic liver procedures. A triangulation process is formulated to compute the 3D coordinates of laser points by using the existing IGS system tracking devices. This allows the use of a compact and cost-effective LRS and therefore facilitates the integration into the laparoscopic setup. The 3D laser points are then reconstructed into a surface to register to the preoperative liver model using a multi-level registration process. RESULTS Experimental results show that the proposed system provides submillimeter scanning precision and accuracy comparable to those reported in the literature. Further quantitative analysis shows that the proposed system is able to achieve a patient-to-image registration accuracy, described as target registration error, of [Formula: see text]. CONCLUSIONS We believe that the presented approach will lead to a faster integration of LRS-based registration techniques in the surgical environment. Further studies will focus on optimizing scanning time and on the respiratory motion compensation.

A Multiresolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles

In this paper, we seek to expand the use of direct methods in real-time applications by proposing a vision-based strategy for pose estimation of aerial vehicles. The vast majority of approaches make use of features to estimate motion. Conversely, the strategy we propose is based on a MR (Multi-Resolution) implementation of an image registration technique (Inverse Compositional Image Alignment ICIA) using direct methods. An on-board camera in a downwards-looking configuration, and the assumption of planar scenes, are the bases of the algorithm. The motion between frames (rotation and translation) is recovered by decomposing the frame-to-frame homography obtained by the ICIA algorithm applied to a patch that covers around the 80% of the image. When the visual estimation is required (e.g. GPS drop-out), this motion is integrated with the previous known estimation of the vehicles' state, obtained from the on-board sensors (GPS/IMU), and the subsequent estimations are based only on the vision-based motion estimations. The proposed strategy is tested with real flight data in representative stages of a flight: cruise, landing, and take-off, being two of those stages considered critical: take-off and landing. The performance of the pose estimation strategy is analyzed by comparing it with the GPS/IMU estimations. Results show correlation between the visual estimation obtained with the MR-ICIA and the GPS/IMU data, that demonstrate that the visual estimation can be used to provide a good approximation of the vehicle's state when it is required (e.g. GPS drop-outs). In terms of performance, the proposed strategy is able to maintain an estimation of the vehicle's state for more than one minute, at real-time frame rates based, only on visual information.

Computational methods to create and analyze a digital gene expression atlas of embryo development from microscopy images

Abstract The creation of atlases, or digital models where information from different subjects can be combined, is a field of increasing interest in biomedical imaging. When a single image does not contain enough information to appropriately describe the organism under study, it is then necessary to acquire images of several individuals, each of them containing complementary data with respect to the rest of the components in the cohort. This approach allows creating digital prototypes, ranging from anatomical atlases of human patients and organs, obtained for instance from Magnetic Resonance Imaging, to gene expression cartographies of embryo development, typically achieved from Light Microscopy. Within such context, in this PhD Thesis we propose, develop and validate new dedicated image processing methodologies that, based on image registration techniques, bring information from multiple individuals into alignment within a single digital atlas model. We also elaborate a dedicated software visualization platform to explore the resulting wealth of multi-dimensional data and novel analysis algo-rithms to automatically mine the generated resource in search of bio¬logical insights. In particular, this work focuses on gene expression data from developing zebrafish embryos imaged at the cellular resolution level with Two-Photon Laser Scanning Microscopy. Disposing of quantitative measurements relating multiple gene expressions to cell position and their evolution in time is a fundamental prerequisite to understand embryogenesis multi-scale processes. However, the number of gene expressions that can be simultaneously stained in one acquisition is limited due to optical and labeling constraints. These limitations motivate the implementation of atlasing strategies that can recreate a virtual gene expression multiplex. The developed computational tools have been tested in two different scenarios. The first one is the early zebrafish embryogenesis where the resulting atlas constitutes a link between the phenotype and the genotype at the cellular level. The second one is the late zebrafish brain where the resulting atlas allows studies relating gene expression to brain regionalization and neurogenesis. The proposed computational frameworks have been adapted to the requirements of both scenarios, such as the integration of partial views of the embryo into a whole embryo model with cellular resolution or the registration of anatom¬ical traits with deformable transformation models non-dependent on any specific labeling. The software implementation of the atlas generation tool (Match-IT) and the visualization platform (Atlas-IT) together with the gene expression atlas resources developed in this Thesis are to be made freely available to the scientific community. Lastly, a novel proof-of-concept experiment integrates for the first time 3D gene expression atlas resources with cell lineages extracted from live embryos, opening up the door to correlate genetic and cellular spatio-temporal dynamics. La creación de atlas, o modelos digitales, donde la información de distintos sujetos puede ser combinada, es un campo de creciente interés en imagen biomédica. Cuando una sola imagen no contiene suficientes datos como para describir apropiadamente el organismo objeto de estudio, se hace necesario adquirir imágenes de varios individuos, cada una de las cuales contiene información complementaria respecto al resto de componentes del grupo. De este modo, es posible crear prototipos digitales, que pueden ir desde atlas anatómicos de órganos y pacientes humanos, adquiridos por ejemplo mediante Resonancia Magnética, hasta cartografías de la expresión genética del desarrollo de embrionario, típicamente adquiridas mediante Microscopía Optica. Dentro de este contexto, en esta Tesis Doctoral se introducen, desarrollan y validan nuevos métodos de procesado de imagen que, basándose en técnicas de registro de imagen, son capaces de alinear imágenes y datos provenientes de múltiples individuos en un solo atlas digital. Además, se ha elaborado una plataforma de visualization específicamente diseñada para explorar la gran cantidad de datos, caracterizados por su multi-dimensionalidad, que resulta de estos métodos. Asimismo, se han propuesto novedosos algoritmos de análisis y minería de datos que permiten inspeccionar automáticamente los atlas generados en busca de conclusiones biológicas significativas. En particular, este trabajo se centra en datos de expresión genética del desarrollo embrionario del pez cebra, adquiridos mediante Microscopía dos fotones con resolución celular. Disponer de medidas cuantitativas que relacionen estas expresiones genéticas con las posiciones celulares y su evolución en el tiempo es un prerrequisito fundamental para comprender los procesos multi-escala característicos de la morfogénesis. Sin embargo, el número de expresiones genéticos que pueden ser simultáneamente etiquetados en una sola adquisición es reducido debido a limitaciones tanto ópticas como del etiquetado. Estas limitaciones requieren la implementación de estrategias de creación de atlas que puedan recrear un multiplexado virtual de expresiones genéticas. Las herramientas computacionales desarrolladas han sido validadas en dos escenarios distintos. El primer escenario es el desarrollo embrionario temprano del pez cebra, donde el atlas resultante permite constituir un vínculo, a nivel celular, entre el fenotipo y el genotipo de este organismo modelo. El segundo escenario corresponde a estadios tardíos del desarrollo del cerebro del pez cebra, donde el atlas resultante permite relacionar expresiones genéticas con la regionalización del cerebro y la formación de neuronas. La plataforma computacional desarrollada ha sido adaptada a los requisitos y retos planteados en ambos escenarios, como la integración, a resolución celular, de vistas parciales dentro de un modelo consistente en un embrión completo, o el alineamiento entre estructuras de referencia anatómica equivalentes, logrado mediante el uso de modelos de transformación deformables que no requieren ningún marcador específico. Está previsto poner a disposición de la comunidad científica tanto la herramienta de generación de atlas (Match-IT), como su plataforma de visualización (Atlas-IT), así como las bases de datos de expresión genética creadas a partir de estas herramientas. Por último, dentro de la presente Tesis Doctoral, se ha incluido una prueba conceptual innovadora que permite integrar los mencionados atlas de expresión genética tridimensionales dentro del linaje celular extraído de una adquisición in vivo de un embrión. Esta prueba conceptual abre la puerta a la posibilidad de correlar, por primera vez, las dinámicas espacio-temporales de genes y células.

MRI Registration for Human Hip Kinematics

Measurement of joint kinematics can provide knowledge to help improve joint prosthesis design, as well as identify joint motion patterns that may lead to joint degeneration or injury. More investigation into how the hip translates in live human subjects during high amplitude motions is needed. This work presents a design of a non-invasive method using the registration between images from conventional Magnetic Resonance Imaging (MRI) and open MRI to calculate three dimensional hip joint kinematics. The method was tested on a single healthy subject in three different poses. MRI protocols for the conventional gantry, high-resolution MRI and the open gantry, lowresolution MRI were developed. The scan time for the low-resolution protocol was just under 6 minutes. High-resolution meshes and low resolution contours were derived from segmentation of the high-resolution and low-resolution images, respectively. Low-resolution contours described the poses as scanned, whereas the meshes described the bones’ geometries. The meshes and contours were registered to each other, and joint kinematics were calculated. The segmentation and registration were performed for both cortical and sub-cortical bone surfaces. A repeatability study was performed by comparing the kinematic results derived from three users’ segmentations of the sub-cortical bone surfaces from a low-resolution scan. The root mean squared error of all registrations was below 1.92mm. The maximum range between segmenters in translation magnitude was 0.95mm, and the maximum deviation from the average of all orientations was 1.27◦. This work demonstrated that this method for non-invasive measurement of hip kinematics is promising for measuring high-range-of-motion hip motions in vivo.

Open-source virtual bronchoscopy for image guided navigation

This thesis describes the development of an open-source system for virtual bronchoscopy used in combination with electromagnetic instrument tracking. The end application is virtual navigation of the lung for biopsy of early stage cancer nodules. The open-source platform 3D Slicer was used for creating freely available algorithms for virtual bronchscopy. Firstly, the development of an open-source semi-automatic algorithm for prediction of solitary pulmonary nodule malignancy is presented. This approach may help the physician decide whether to proceed with biopsy of the nodule. The user-selected nodule is segmented in order to extract radiological characteristics (i.e., size, location, edge smoothness, calcification presence, cavity wall thickness) which are combined with patient information to calculate likelihood of malignancy. The overall accuracy of the algorithm is shown to be high compared to independent experts' assessment of malignancy. The algorithm is also compared with two different predictors, and our approach is shown to provide the best overall prediction accuracy. The development of an airway segmentation algorithm which extracts the airway tree from surrounding structures on chest Computed Tomography (CT) images is then described. This represents the first fundamental step toward the creation of a virtual bronchoscopy system. Clinical and ex-vivo images are used to evaluate performance of the algorithm. Different CT scan parameters are investigated and parameters for successful airway segmentation are optimized. Slice thickness is the most affecting parameter, while variation of reconstruction kernel and radiation dose is shown to be less critical. Airway segmentation is used to create a 3D rendered model of the airway tree for virtual navigation. Finally, the first open-source virtual bronchoscopy system was combined with electromagnetic tracking of the bronchoscope for the development of a GPS-like system for navigating within the lungs. Tools for pre-procedural planning and for helping with navigation are provided. Registration between the lungs of the patient and the virtually reconstructed airway tree is achieved using a landmark-based approach. In an attempt to reduce difficulties with registration errors, we also implemented a landmark-free registration method based on a balanced airway survey. In-vitro and in-vivo testing showed good accuracy for this registration approach. The centreline of the 3D airway model is extracted and used to compensate for possible registration errors. Tools are provided to select a target for biopsy on the patient CT image, and pathways from the trachea towards the selected targets are automatically created. The pathways guide the physician during navigation, while distance to target information is updated in real-time and presented to the user. During navigation, video from the bronchoscope is streamed and presented to the physician next to the 3D rendered image. The electromagnetic tracking is implemented with 5 DOF sensing that does not provide roll rotation information. An intensity-based image registration approach is implemented to rotate the virtual image according to the bronchoscope's rotations. The virtual bronchoscopy system is shown to be easy to use and accurate in replicating the clinical setting, as demonstrated in the pre-clinical environment of a breathing lung method. Animal studies were performed to evaluate the overall system performance.

MRI Registration for Human Hip Kinematics

Measurement of joint kinematics can provide knowledge to help improve joint prosthesis design, as well as identify joint motion patterns that may lead to joint degeneration or injury. More investigation into how the hip translates in live human subjects during high amplitude motions is needed. This work presents a design of a non-invasive method using the registration between images from conventional Magnetic Resonance Imaging (MRI) and open MRI to calculate three dimensional hip joint kinematics. The method was tested on a single healthy subject in three different poses. MRI protocols for the conventional gantry, high-resolution MRI and the open gantry, lowresolution MRI were developed. The scan time for the low-resolution protocol was just under 6 minutes. High-resolution meshes and low resolution contours were derived from segmentation of the high-resolution and low-resolution images, respectively. Low-resolution contours described the poses as scanned, whereas the meshes described the bones’ geometries. The meshes and contours were registered to each other, and joint kinematics were calculated. The segmentation and registration were performed for both cortical and sub-cortical bone surfaces. A repeatability study was performed by comparing the kinematic results derived from three users’ segmentations of the sub-cortical bone surfaces from a low-resolution scan. The root mean squared error of all registrations was below 1.92mm. The maximum range between segmenters in translation magnitude was 0.95mm, and the maximum deviation from the average of all orientations was 1.27◦. This work demonstrated that this method for non-invasive measurement of hip kinematics is promising for measuring high-range-of-motion hip motions in vivo.

Registration-by-regression of coronary CTA and X-ray angiography

We evaluate the integration of 3D preoperative computed tomography angiography of the coronary arteries with intraoperative 2D X-ray angiographies by a recently proposed novel registration-by-regression method. The method relates image features of 2D projection images to the transformation parameters of the 3D image. We compared different sets of features and studied the influence of preprocessing the training set. For the registration evaluation, a gold standard was developed from eight X-ray angiography sequences from six different patients. The alignment quality was measured using the 3D mean target registration error (mTRE). The registration-by-regression method achieved moderate accuracy (median mTRE of 15 mm) on real images. It does therefore not provide yet a complete solution to the 3D–2D registration problem but it could be used as an initialisation method to eliminate the need for manual initialisation.

Three-dimensional, in-silico breast phantom for multimodality image simulations

We report on the construction of anatomically realistic three-dimensional in-silico breast phantoms with adjustable sizes, shapes and morphologic features. The concept of multiscale spatial resolution is implemented for generating breast tissue images from multiple modalities. Breast epidermal boundary and subcutaneous fat layer is generated by fitting an ellipsoid and 2nd degree polynomials to reconstructive surgical data and ultrasound imaging data. Intraglandular fat is simulated by randomly distributing and orienting adipose ellipsoids within a fibrous region immediately within the dermal layer. Cooper’s ligaments are simulated as fibrous ellipsoidal shells distributed within the subcutaneous fat layer. Individual ductal lobes are simulated following a random binary tree model which is generated based upon probabilistic branching conditions described by ramification matrices, as originally proposed by Bakic et al [3, 4]. The complete ductal structure of the breast is simulated from multiple lobes that extend from the base of the nipple and branch towards the chest wall. As lobe branching progresses, branches are reduced in height and radius and terminal branches are capped with spherical lobular clusters. Biophysical parameters are mapped onto the complete anatomical model and synthetic multimodal images (Mammography, Ultrasound, CT) are generated for phantoms of different adipose percentages (40%, 50%, 60%, and 70%) and are analytically compared with clinical examples. Results demonstrate that the in-silico breast phantom has applications in imaging performance evaluation and, specifically, great utility for solving image registration issues in multimodality imaging.

Image Processing Algorithms applied to Radio SLAM using MIMO Radars

Radio Simultaneous Location and Mapping (SLAM) consists of the simultaneous tracking of the target and estimation of the surrounding environment, to build a map and estimate the target movements within it. It is an increasingly exploited technique for automotive applications, in order to improve the localization of obstacles and the target relative movement with respect to them, for emergency situations, for example when it is necessary to explore (with a drone or a robot) environments with a limited visibility, or for personal radar applications, thanks to its versatility and cheapness. Until today, these systems were based on light detection and ranging (lidar) or visual cameras, high-accuracy and expensive approaches that are limited to specific environments and weather conditions. Instead, in case of smoke, fog or simply darkness, radar-based systems can operate exactly in the same way. In this thesis activity, the Fourier-Mellin algorithm is analyzed and implemented, to verify the applicability to Radio SLAM, in which the radar frames can be treated as images and the radar motion between consecutive frames can be covered with registration. Furthermore, a simplified version of that algorithm is proposed, in order to solve the problems of the Fourier-Mellin algorithm when working with real radar images and improve the performance. The INRAS RBK2, a MIMO 2x16 mmWave radar, is used for experimental acquisitions, consisting of multiple tests performed in Lab-E of the Cesena Campus, University of Bologna. The different performances of Fourier-Mellin and its simplified version are compared also with the MatchScan algorithm, a classic algorithm for SLAM systems.

White matter microstructure underlying default mode network connectivity in the human brain

Resting state functional magnetic resonance imaging (fMRI) reveals a distinct network of correlated brain function representing a default mode state of the human brain The underlying structural basis of this functional connectivity pattern is still widely unexplored We combined fractional anisotropy measures of fiber tract integrity derived from diffusion tensor imaging (DTI) and resting state fMRI data obtained at 3 Tesla from 20 healthy elderly subjects (56 to 83 years of age) to determine white matter microstructure e 7 underlying default mode connectivity We hypothesized that the functional connectivity between the posterior cingulate and hippocampus from resting state fMRI data Would be associated with the white matter microstructure in the cingulate bundle and fiber tracts connecting posterior cingulate gyrus With lateral temporal lobes, medial temporal lobes, and precuneus This was demonstrated at the p<0001 level using a voxel-based multivariate analysis of covariance (MANCOVA) approach In addition, we used a data-driven technique of joint independent component analysis (ICA) that uncovers spatial pattern that are linked across modalities. It revealed a pattern of white matter tracts including cingulate bundle and associated fiber tracts resembling the findings from the hypothesis-driven analysis and was linked to the pattern of default mode network (DMN) connectivity in the resting state fMRI data Out findings support the notion that the functional connectivity between the posterior cingulate and hippocampus and the functional connectivity across the entire DMN is based oil distinct pattern of anatomical connectivity within the cerebral white matter (C) 2009 Elsevier Inc All rights reserved

The Cholinergic System in Mild Cognitive Impairment and Alzheimer`s Disease: An In Vivo MRI and DTI study

Few studies have investigated in vivo changes of the cholinergic basal forebrain in Alzheimer`s disease (AD) and amnestic mild cognitive impairment (MCI), an at risk stage of AD. Even less is known about alterations of cortical projecting fiber tracts associated with basal forebrain atrophy. In this study, we determined regional atrophy within the basal forebrain in 21 patients with AD and 16 subjects with MCI compared to 20 healthy elderly subjects using deformation-based morphometry of MRI scans. We assessed effects of basal forebrain atrophy on fiber tracts derived from high-resolution diffusion tensor imaging (DTI) using tract-based spatial statistics. We localized significant effects relative to a map of cholinergic nuclei in MRI standard space as determined from a postmortem brain. Patients with AD and MCI subjects showed reduced volumes in basal forebrain areas corresponding to anterior medial and lateral, intermediate and posterior nuclei of the Nucleus basalis of Meynert (NbM) as well as in the diagonal band of Broca nuclei (P < 0.01). Effects in MCI subjects were spatially more restricted than in AD, but occurred at similar locations. The volume of the right antero-lateral NbM nucleus was correlated with intracortical projecting fiber tract integrity such as the corpus callosum, cingulate, and the superior longitudinal, inferior longitudinal, inferior fronto-occipital, and uncinate fasciculus (P < 0.05, corrected for multiple comparisons). Our findings suggest that a multimodal MRI-DTI approach is supportive to determine atrophy of cholinergic nuclei and its effect on intracortical projecting fiber tracts in AD. Hum Brain Mapp 32: 1349-1362, 2011. (C) 2010 Wiley-Liss, Inc.

Three-dimensional space-borne synthetic aperture radar (SAR) imaging with multiple pass processing

Three-dimensional (3D) synthetic aperture radar (SAR) imaging via multiple-pass processing is an extension of interferometric SAR imaging. It exploits more than two flight passes to achieve a desired resolution in elevation. In this paper, a novel approach is developed to reconstruct a 3D space-borne SAR image with multiple-pass processing. It involves image registration, phase correction and elevational imaging. An image model matching is developed for multiple image registration, an eigenvector method is proposed for the phase correction and the elevational imaging is conducted using a Fourier transform or a super-resolution method for enhancement of elevational resolution. 3D SAR images are obtained by processing simulated data and real data from the first European Remote Sensing satellite (ERS-1) with the proposed approaches.