Multimodal non-rigid image registration methods for therapy and surgery planning

La planificación pre-operatoria se ha convertido en una tarea esencial en cirugías y terapias de marcada complejidad, especialmente aquellas relacionadas con órgano blando. Un ejemplo donde la planificación preoperatoria tiene gran interés es la cirugía hepática. Dicha planificación comprende la detección e identificación precisa de las lesiones individuales y vasos así como la correcta segmentación y estimación volumétrica del hígado funcional. Este proceso es muy importante porque determina tanto si el paciente es un candidato adecuado para terapia quirúrgica como la definición del abordaje a seguir en el procedimiento. La radioterapia de órgano blando es un segundo ejemplo donde la planificación se requiere tanto para la radioterapia externa convencional como para la radioterapia intraoperatoria. La planificación comprende la segmentación de tumor y órganos vulnerables y la estimación de la dosimetría. La segmentación de hígado funcional y la estimación volumétrica para planificación de la cirugía se estiman habitualmente a partir de imágenes de tomografía computarizada (TC). De igual modo, en la planificación de radioterapia, los objetivos de la radiación se delinean normalmente sobre TC. Sin embargo, los avances en las tecnologías de imagen de resonancia magnética (RM) están ofreciendo progresivamente ventajas adicionales. Por ejemplo, se ha visto que el ratio de detección de metástasis hepáticas es significativamente superior en RM con contraste Gd–EOB–DTPA que en TC. Por tanto, recientes estudios han destacado la importancia de combinar la información de TC y RM para conseguir el mayor nivel posible de precisión en radioterapia y para facilitar una descripción precisa de las lesiones del hígado. Con el objetivo de mejorar la planificación preoperatoria en ambos escenarios se precisa claramente de un algoritmo de registro no rígido de imagen. Sin embargo, la gran mayoría de sistemas comerciales solo proporcionan métodos de registro rígido. Las medidas de intensidad de voxel han demostrado ser criterios de similitud de imágenes robustos, y, entre ellas, la Información Mutua (IM) es siempre la primera elegida en registros multimodales. Sin embargo, uno de los principales problemas de la IM es la ausencia de información espacial y la asunción de que las relaciones estadísticas entre las imágenes son homogéneas a lo largo de su domino completo. La hipótesis de esta tesis es que la incorporación de información espacial de órganos al proceso de registro puede mejorar la robustez y calidad del mismo, beneficiándose de la disponibilidad de las segmentaciones clínicas. En este trabajo, se propone y valida un esquema de registro multimodal no rígido 3D usando una nueva métrica llamada Información Mutua Centrada en el Órgano (Organ-Focused Mutual Information metric (OF-MI)) y se compara con la formulación clásica de la Información Mutua. Esto permite mejorar los resultados del registro en áreas problemáticas incorporando información regional al criterio de similitud, beneficiándose de la disponibilidad real de segmentaciones en protocolos estándares clínicos, y permitiendo que la dependencia estadística entre las dos modalidades de imagen difiera entre órganos o regiones. El método propuesto se ha aplicado al registro de TC y RM con contraste Gd–EOB–DTPA así como al registro de imágenes de TC y MR para planificación de radioterapia intraoperatoria rectal. Adicionalmente, se ha desarrollado un algoritmo de apoyo de segmentación 3D basado en Level-Sets para la incorporación de la información de órgano en el registro. El algoritmo de segmentación se ha diseñado específicamente para la estimación volumétrica de hígado sano funcional y ha demostrado un buen funcionamiento en un conjunto de imágenes de TC abdominales. Los resultados muestran una mejora estadísticamente significativa de OF-MI comparada con la Información Mutua clásica en las medidas de calidad de los registros; tanto con datos simulados (p<0.001) como con datos reales en registro hepático de TC y RM con contraste Gd– EOB–DTPA y en registro para planificación de radioterapia rectal usando OF-MI multi-órgano (p<0.05). Adicionalmente, OF-MI presenta resultados más estables con menor dispersión que la Información Mutua y un comportamiento más robusto con respecto a cambios en la relación señal-ruido y a la variación de parámetros. La métrica OF-MI propuesta en esta tesis presenta siempre igual o mayor precisión que la clásica Información Mutua y consecuentemente puede ser una muy buena alternativa en aplicaciones donde la robustez del método y la facilidad en la elección de parámetros sean particularmente importantes. Abstract Pre-operative planning has become an essential task in complex surgeries and therapies, especially for those affecting soft tissue. One example where soft tissue preoperative planning is of high interest is liver surgery. It involves the accurate detection and identification of individual liver lesions and vessels as well as the proper functional liver segmentation and volume estimation. This process is very important because it determines whether the patient is a suitable candidate for surgical therapy and the type of procedure. Soft tissue radiation therapy is a second example where planning is required for both conventional external and intraoperative radiotherapy. It involves the segmentation of the tumor target and vulnerable organs and the estimation of the planned dose. Functional liver segmentations and volume estimations for surgery planning are commonly estimated from computed tomography (CT) images. Similarly, in radiation therapy planning, targets to be irradiated and healthy and vulnerable tissues to be protected from irradiation are commonly delineated on CT scans. However, developments in magnetic resonance imaging (MRI) technology are progressively offering advantages. For instance, the hepatic metastasis detection rate has been found to be significantly higher in Gd–EOB–DTPAenhanced MRI than in CT. Therefore, recent studies highlight the importance of combining the information from CT and MRI to achieve the highest level of accuracy in radiotherapy and to facilitate accurate liver lesion description. In order to improve those two soft tissue pre operative planning scenarios, an accurate nonrigid image registration algorithm is clearly required. However, the vast majority of commercial systems only provide rigid registration. Voxel intensity measures have been shown to be robust measures of image similarity, and among them, Mutual Information (MI) is always the first candidate in multimodal registrations. However, one of the main drawbacks of Mutual Information is the absence of spatial information and the assumption that statistical relationships between images are the same over the whole domain of the image. The hypothesis of the present thesis is that incorporating spatial organ information into the registration process may improve the registration robustness and quality, taking advantage of the clinical segmentations availability. In this work, a multimodal nonrigid 3D registration framework using a new Organ- Focused Mutual Information metric (OF-MI) is proposed, validated and compared to the classical formulation of the Mutual Information (MI). It allows improving registration results in problematic areas by adding regional information into the similitude criterion taking advantage of actual segmentations availability in standard clinical protocols and allowing the statistical dependence between the two modalities differ among organs or regions. The proposed method is applied to CT and T1 weighted delayed Gd–EOB–DTPA-enhanced MRI registration as well as to register CT and MRI images in rectal intraoperative radiotherapy planning. Additionally, a 3D support segmentation algorithm based on Level-Sets has been developed for the incorporation of the organ information into the registration. The segmentation algorithm has been specifically designed for the healthy and functional liver volume estimation demonstrating good performance in a set of abdominal CT studies. Results show a statistical significant improvement of registration quality measures with OF-MI compared to MI with both simulated data (p<0.001) and real data in liver applications registering CT and Gd–EOB–DTPA-enhanced MRI and in registration for rectal radiotherapy planning using multi-organ OF-MI (p<0.05). Additionally, OF-MI presents more stable results with smaller dispersion than MI and a more robust behavior with respect to SNR changes and parameters variation. The proposed OF-MI always presents equal or better accuracy than the classical MI and consequently can be a very convenient alternative within applications where the robustness of the method and the facility to choose the parameters are particularly important.

Efficient Model-based 3D Tracking by Using Direct Image Registration

This thesis deals with the problem of efficiently tracking 3D objects in sequences of images. We tackle the efficient 3D tracking problem by using direct image registration. This problem is posed as an iterative optimization procedure that minimizes a brightness error norm. We review the most popular iterative methods for image registration in the literature, turning our attention to those algorithms that use efficient optimization techniques. Two forms of efficient registration algorithms are investigated. The first type comprises the additive registration algorithms: these algorithms incrementally compute the motion parameters by linearly approximating the brightness error function. We centre our attention on Hager and Belhumeur’s factorization-based algorithm for image registration. We propose a fundamental requirement that factorization-based algorithms must satisfy to guarantee good convergence, and introduce a systematic procedure that automatically computes the factorization. Finally, we also bring out two warp functions to register rigid and nonrigid 3D targets that satisfy the requirement. The second type comprises the compositional registration algorithms, where the brightness function error is written by using function composition. We study the current approaches to compositional image alignment, and we emphasize the importance of the Inverse Compositional method, which is known to be the most efficient image registration algorithm. We introduce a new algorithm, the Efficient Forward Compositional image registration: this algorithm avoids the necessity of inverting the warping function, and provides a new interpretation of the working mechanisms of the inverse compositional alignment. By using this information, we propose two fundamental requirements that guarantee the convergence of compositional image registration methods. Finally, we support our claims by using extensive experimental testing with synthetic and real-world data. We propose a distinction between image registration and tracking when using efficient algorithms. We show that, depending whether the fundamental requirements are hold, some efficient algorithms are eligible for image registration but not for tracking.

Cloud Computing Service for Managing Large Medical Image Data-Sets Using Balanced Collaborative Agents

Managing large medical image collections is an increasingly demanding important issue in many hospitals and other medical settings. A huge amount of this information is daily generated, which requires robust and agile systems. In this paper we present a distributed multi-agent system capable of managing very large medical image datasets. In this approach, agents extract low-level information from images and store them in a data structure implemented in a relational database. The data structure can also store semantic information related to images and particular regions. A distinctive aspect of our work is that a single image can be divided so that the resultant sub-images can be stored and managed separately by different agents to improve performance in data accessing and processing. The system also offers the possibility of applying some region-based operations and filters on images, facilitating image classification. These operations can be performed directly on data structures in the database.

MIA - a free and open source software for gray scale medical image analysis

Background Gray scale images make the bulk of data in bio-medical image analysis, and hence, the main focus of many image processing tasks lies in the processing of these monochrome images. With ever improving acquisition devices, spatial and temporal image resolution increases, and data sets become very large. Various image processing frameworks exists that make the development of new algorithms easy by using high level programming languages or visual programming. These frameworks are also accessable to researchers that have no background or little in software development because they take care of otherwise complex tasks. Specifically, the management of working memory is taken care of automatically, usually at the price of requiring more it. As a result, processing large data sets with these tools becomes increasingly difficult on work station class computers. One alternative to using these high level processing tools is the development of new algorithms in a languages like C++, that gives the developer full control over how memory is handled, but the resulting workflow for the prototyping of new algorithms is rather time intensive, and also not appropriate for a researcher with little or no knowledge in software development. Another alternative is in using command line tools that run image processing tasks, use the hard disk to store intermediate results, and provide automation by using shell scripts. Although not as convenient as, e.g. visual programming, this approach is still accessable to researchers without a background in computer science. However, only few tools exist that provide this kind of processing interface, they are usually quite task specific, and don’t provide an clear approach when one wants to shape a new command line tool from a prototype shell script. Results The proposed framework, MIA, provides a combination of command line tools, plug-ins, and libraries that make it possible to run image processing tasks interactively in a command shell and to prototype by using the according shell scripting language. Since the hard disk becomes the temporal storage memory management is usually a non-issue in the prototyping phase. By using string-based descriptions for filters, optimizers, and the likes, the transition from shell scripts to full fledged programs implemented in C++ is also made easy. In addition, its design based on atomic plug-ins and single tasks command line tools makes it easy to extend MIA, usually without the requirement to touch or recompile existing code. Conclusion In this article, we describe the general design of MIA, a general purpouse framework for gray scale image processing. We demonstrated the applicability of the software with example applications from three different research scenarios, namely motion compensation in myocardial perfusion imaging, the processing of high resolution image data that arises in virtual anthropology, and retrospective analysis of treatment outcome in orthognathic surgery. With MIA prototyping algorithms by using shell scripts that combine small, single-task command line tools is a viable alternative to the use of high level languages, an approach that is especially useful when large data sets need to be processed.

Robust Image Registration with Global Intensity Transformation

This paper presents a registration method for images with global illumination variations. The method is based on a joint iterative optimization (geometric and photometric) of the L1 norm of the intensity error. Two strategies are compared to directly find the appropriate intensity transformation within each iteration: histogram specification and the solution obtained by analyzing the necessary optimality conditions. Such strategies reduce the search space of the joint optimization to that of the geometric transformation between the images.

Organ-focused mutual information for nonrigid multimodal registration of liver CT and Gd–EOB–DTPA-enhanced MRI

Accurate detection of liver lesions is of great importance in hepatic surgery planning. Recent studies have shown that the detection rate of liver lesions is significantly higher in gadoxetic acid-enhanced magnetic resonance imaging (Gd–EOB–DTPA-enhanced MRI) than in contrast-enhanced portal-phase computed tomography (CT); however, the latter remains essential because of its high specificity, good performance in estimating liver volumes and better vessel visibility. To characterize liver lesions using both the above image modalities, we propose a multimodal nonrigid registration framework using organ-focused mutual information (OF-MI). This proposal tries to improve mutual information (MI) based registration by adding spatial information, benefiting from the availability of expert liver segmentation in clinical protocols. The incorporation of an additional information channel containing liver segmentation information was studied. A dataset of real clinical images and simulated images was used in the validation process. A Gd–EOB–DTPA-enhanced MRI simulation framework is presented. To evaluate results, warping index errors were calculated for the simulated data, and landmark-based and surface-based errors were calculated for the real data. An improvement of the registration accuracy for OF-MI as compared with MI was found for both simulated and real datasets. Statistical significance of the difference was tested and confirmed in the simulated dataset (p < 0.01).

Methodology for registration of distended recutms in pelvic CT studies

Purpose: Accurate delineation of the rectum is of high importance in off-line adaptive radiation therapy since it is a major dose-limiting organ in prostate cancer radiotherapy. The intensity-based deformable image registration (DIR) methods cannot create a correct spatial transformation if there is no correspondence between the template and the target images. The variation of rectal filling, gas, or feces, creates a noncorrespondence in image intensities that becomes a great obstacle for intensity-based DIR. Methods: In this study the authors have designed and implemented a semiautomatic method to create a rectum mask in pelvic computed tomography (CT) images. The method, that includes a DIR based on the demons algorithm, has been tested in 13 prostate cancer cases, each comprising of two CT scans, for a total of 26 CT scans. Results: The use of the manual segmentation in the planning image and the proposed rectum mask method (RMM) method in the daily image leads to an improvement in the DIR performance in pelvic CT images, obtaining a mean value of overlap volume index = 0.89, close to the values obtained using the manual segmentations in both images. Conclusions: The application of the RMM method in the daily image and the manual segmentations in the planning image during prostate cancer treatments increases the performance of the registration in presence of rectal fillings, obtaining very good agreement with a physician's manual contours.

Methodology for registration of distended rectums in pelvic CT studies

Purpose: Accurate delineation of the rectum is of high importance in off-line adaptive radiation therapy since it is a major dose-limiting organ in prostate cancer radiotherapy. The intensity-based deformable image registration (DIR) methods cannot create a correct spatial transformation if there is no correspondence between the template and the target images. The variation of rectal filling, gas, or feces, creates a noncorrespondence in image intensities that becomes a great obstacle for intensity-based DIR. Methods: In this study the authors have designed and implemented a semiautomatic method to create a rectum mask in pelvic computed tomography (CT) images. The method, that includes a DIR based on the demons algorithm, has been tested in 13 prostate cancer cases, each comprising of two CT scans, for a total of 26 CT scans. Results: The use of the manual segmentation in the planning image and the proposed rectum mask method (RMM) method in the daily image leads to an improvement in the DIR performance in pelvic CT images, obtaining a mean value of overlap volume index = 0.89, close to the values obtained using the manual segmentations in both images. Conclusions: The application of the RMM method in the daily image and the manual segmentations in the planning image during prostate cancer treatments increases the performance of the registration in presence of rectal fillings, obtaining very good agreement with a physician's manual contours.

A Multiresolution Image Alignment Technique Based on Direct Methods for Pose Estimation of Aerial Vehicles

In this paper, we seek to expand the use of direct methods in real-time applications by proposing a vision-based strategy for pose estimation of aerial vehicles. The vast majority of approaches make use of features to estimate motion. Conversely, the strategy we propose is based on a MR (Multi-Resolution) implementation of an image registration technique (Inverse Compositional Image Alignment ICIA) using direct methods. An on-board camera in a downwards-looking configuration, and the assumption of planar scenes, are the bases of the algorithm. The motion between frames (rotation and translation) is recovered by decomposing the frame-to-frame homography obtained by the ICIA algorithm applied to a patch that covers around the 80% of the image. When the visual estimation is required (e.g. GPS drop-out), this motion is integrated with the previous known estimation of the vehicles' state, obtained from the on-board sensors (GPS/IMU), and the subsequent estimations are based only on the vision-based motion estimations. The proposed strategy is tested with real flight data in representative stages of a flight: cruise, landing, and take-off, being two of those stages considered critical: take-off and landing. The performance of the pose estimation strategy is analyzed by comparing it with the GPS/IMU estimations. Results show correlation between the visual estimation obtained with the MR-ICIA and the GPS/IMU data, that demonstrate that the visual estimation can be used to provide a good approximation of the vehicle's state when it is required (e.g. GPS drop-outs). In terms of performance, the proposed strategy is able to maintain an estimation of the vehicle's state for more than one minute, at real-time frame rates based, only on visual information.

Automatic motion compensation of free breathing acquired myocardial perfusion data by using independent component analysis

Images acquired during free breathing using first-pass gadolinium-enhanced myocardial perfusion magnetic resonance imaging (MRI) exhibit a quasiperiodic motion pattern that needs to be compensated for if a further automatic analysis of the perfusion is to be executed. In this work, we present a method to compensate this movement by combining independent component analysis (ICA) and image registration: First, we use ICA and a time?frequency analysis to identify the motion and separate it from the intensity change induced by the contrast agent. Then, synthetic reference images are created by recombining all the independent components but the one related to the motion. Therefore, the resulting image series does not exhibit motion and its images have intensities similar to those of their original counterparts. Motion compensation is then achieved by using a multi-pass image registration procedure. We tested our method on 39 image series acquired from 13 patients, covering the basal, mid and apical areas of the left heart ventricle and consisting of 58 perfusion images each. We validated our method by comparing manually tracked intensity profiles of the myocardial sections to automatically generated ones before and after registration of 13 patient data sets (39 distinct slices). We compared linear, non-linear, and combined ICA based registration approaches and previously published motion compensation schemes. Considering run-time and accuracy, a two-step ICA based motion compensation scheme that first optimizes a translation and then for non-linear transformation performed best and achieves registration of the whole series in 32 ± 12 s on a recent workstation. The proposed scheme improves the Pearsons correlation coefficient between manually and automatically obtained time?intensity curves from .84 ± .19 before registration to .96 ± .06 after registration

Technical evaluation of a third generation optical pose tracker for motion analysis and image-guided surgery

Laparoscopic instrument tracking systems are an essential component in image-guided interventions and offer new possibilities to improve and automate objective assessment methods of surgical skills. In this study we present our system design to apply a third generation optical pose tracker (Micron- Tracker®) to laparoscopic practice. A technical evaluation of this design is performed in order to analyze its accuracy in computing the laparoscopic instrument tip position. Results show a stable fluctuation error over the entire analyzed workspace. The relative position errors are 1.776±1.675 mm, 1.817±1.762 mm, 1.854±1.740 mm, 2.455±2.164 mm, 2.545±2.496 mm, 2.764±2.342 mm, 2.512±2.493 mm for distances of 50, 100, 150, 200, 250, 300, and 350 mm, respectively. The accumulated distance error increases with the measured distance. The instrument inclination covered by the system is high, from 90 to 7.5 degrees. The system reports a low positional accuracy for the instrument tip.

Mejoras en el procesamiento de imágenes de TC para el tratamiento con radioterapia adaptativa del cáncer de próstata

El cáncer de próstata es el tipo de cáncer con mayor prevalencia entre los hombres del mundo occidental y, pese a tener una alta tasa de supervivencia relativa, es la segunda mayor causa de muerte por cáncer en este sector de la población. El tratamiento de elección frente al cáncer de próstata es, en la mayoría de los casos, la radioterapia externa. Las técnicas más modernas de radioterapia externa, como la radioterapia modulada en intensidad, permiten incrementar la dosis en el tumor mientras se reduce la dosis en el tejido sano. Sin embargo, la localización del volumen objetivo varía con el día de tratamiento, y se requieren movimientos muy pequeños de los órganos para sacar partes del volumen objetivo fuera de la región terapéutica, o para introducir tejidos sanos críticos dentro. Para evitar esto se han desarrollado técnicas más avanzadas, como la radioterapia guiada por imagen, que se define por un manejo más preciso de los movimientos internos mediante una adaptación de la planificación del tratamiento basada en la información anatómica obtenida de imágenes de tomografía computarizada (TC) previas a la sesión terapéutica. Además, la radioterapia adaptativa añade la información dosimétrica de las fracciones previas a la información anatómica. Uno de los fundamentos de la radioterapia adaptativa es el registro deformable de imágenes, de gran utilidad a la hora de modelar los desplazamientos y deformaciones de los órganos internos. Sin embargo, su utilización conlleva nuevos retos científico-tecnológicos en el procesamiento de imágenes, principalmente asociados a la variabilidad de los órganos, tanto en localización como en apariencia. El objetivo de esta tesis doctoral es mejorar los procesos clínicos de delineación automática de contornos y de cálculo de dosis acumulada para la planificación y monitorización de tratamientos con radioterapia adaptativa, a partir de nuevos métodos de procesamiento de imágenes de TC (1) en presencia de contrastes variables, y (2) cambios de apariencia del recto. Además, se pretende (3) proveer de herramientas para la evaluación de la calidad de los contornos obtenidos en el caso del gross tumor volumen (GTV). Las principales contribuciones de esta tesis doctoral son las siguientes: _ 1. La adaptación, implementación y evaluación de un algoritmo de registro basado en el flujo óptico de la fase de la imagen como herramienta para el cálculo de transformaciones no-rígidas en presencia de cambios de intensidad, y su aplicabilidad a tratamientos de radioterapia adaptativa en cáncer de próstata con uso de agentes de contraste radiológico. Los resultados demuestran que el algoritmo seleccionado presenta mejores resultados cualitativos en presencia de contraste radiológico en la vejiga, y no distorsiona la imagen forzando deformaciones poco realistas. 2. La definición, desarrollo y validación de un nuevo método de enmascaramiento de los contenidos del recto (MER), y la evaluación de su influencia en el procedimiento de radioterapia adaptativa en cáncer de próstata. Las segmentaciones obtenidas mediante el MER para la creación de máscaras homogéneas en las imágenes de sesión permiten mejorar sensiblemente los resultados de los algoritmos de registro en la región rectal. Así, el uso de la metodología propuesta incrementa el índice de volumen solapado entre los contornos manuales y automáticos del recto hasta un valor del 89%, cercano a los resultados obtenidos usando máscaras manuales para el registro de las dos imágenes. De esta manera se pueden corregir tanto el cálculo de los nuevos contornos como el cálculo de la dosis acumulada. 3. La definición de una metodología de evaluación de la calidad de los contornos del GTV, que permite la representación de la distribución espacial del error, adaptándola a volúmenes no-convexos como el formado por la próstata y las vesículas seminales. Dicha metodología de evaluación, basada en un nuevo algoritmo de reconstrucción tridimensional y una nueva métrica de cuantificación, presenta resultados precisos con una gran resolución espacial en un tiempo despreciable frente al tiempo de registro. Esta nueva metodología puede ser una herramienta útil para la comparación de distintos algoritmos de registro deformable orientados a la radioterapia adaptativa en cáncer de próstata. En conclusión, el trabajo realizado en esta tesis doctoral corrobora las hipótesis de investigación postuladas, y pretende servir como cimiento de futuros avances en el procesamiento de imagen médica en los tratamientos de radioterapia adaptativa en cáncer de próstata. Asimismo, se siguen abriendo nuevas líneas de aplicación futura de métodos de procesamiento de imágenes médicas con el fin de mejorar los procesos de radioterapia adaptativa en presencia de cambios de apariencia de los órganos, e incrementar la seguridad del paciente. I.2 Inglés Prostate cancer is the most prevalent cancer amongst men in the Western world and, despite having a relatively high survival rate, is the second leading cause of cancer death in this sector of the population. The treatment of choice against prostate cancer is, in most cases, external beam radiation therapy. The most modern techniques of external radiotherapy, as intensity modulated radiotherapy, allow increasing the dose to the tumor whilst reducing the dose to healthy tissue. However, the location of the target volume varies with the day of treatment, and very small movements of the organs are required to pull out parts of the target volume outside the therapeutic region, or to introduce critical healthy tissues inside. Advanced techniques, such as the image-guided radiotherapy (IGRT), have been developed to avoid this. IGRT is defined by more precise handling of internal movements by adapting treatment planning based on the anatomical information obtained from computed tomography (CT) images prior to the therapy session. Moreover, the adaptive radiotherapy adds dosimetric information of previous fractions to the anatomical information. One of the fundamentals of adaptive radiotherapy is deformable image registration, very useful when modeling the displacements and deformations of the internal organs. However, its use brings new scientific and technological challenges in image processing, mainly associated to the variability of the organs, both in location and appearance. The aim of this thesis is to improve clinical processes of automatic contour delineation and cumulative dose calculation for planning and monitoring of adaptive radiotherapy treatments, based on new methods of CT image processing (1) in the presence of varying contrasts, and (2) rectum appearance changes. It also aims (3) to provide tools for assessing the quality of contours obtained in the case of gross tumor volume (GTV). The main contributions of this PhD thesis are as follows: 1. The adaptation, implementation and evaluation of a registration algorithm based on the optical flow of the image phase as a tool for the calculation of non-rigid transformations in the presence of intensity changes, and its applicability to adaptive radiotherapy treatment in prostate cancer with use of radiological contrast agents. The results demonstrate that the selected algorithm shows better qualitative results in the presence of radiological contrast agents in the urinary bladder, and does not distort the image forcing unrealistic deformations. 2. The definition, development and validation of a new method for masking the contents of the rectum (MER, Spanish acronym), and assessing their impact on the process of adaptive radiotherapy in prostate cancer. The segmentations obtained by the MER for the creation of homogenous masks in the session CT images can improve significantly the results of registration algorithms in the rectal region. Thus, the use of the proposed methodology increases the volume overlap index between manual and automatic contours of the rectum to a value of 89%, close to the results obtained using manual masks for both images. In this way, both the calculation of new contours and the calculation of the accumulated dose can be corrected. 3. The definition of a methodology for assessing the quality of the contours of the GTV, which allows the representation of the spatial distribution of the error, adapting it to non-convex volumes such as that formed by the prostate and seminal vesicles. Said evaluation methodology, based on a new three-dimensional reconstruction algorithm and a new quantification metric, presents accurate results with high spatial resolution in a time negligible compared to the registration time. This new approach may be a useful tool to compare different deformable registration algorithms oriented to adaptive radiotherapy in prostate cancer In conclusion, this PhD thesis corroborates the postulated research hypotheses, and is intended to serve as a foundation for future advances in medical image processing in adaptive radiotherapy treatment in prostate cancer. In addition, it opens new future applications for medical image processing methods aimed at improving the adaptive radiotherapy processes in the presence of organ’s appearance changes, and increase the patient safety.

Higher-order regularization and morphological techniques for image segmentation

La segmentación de imágenes es un campo importante de la visión computacional y una de las áreas de investigación más activas, con aplicaciones en comprensión de imágenes, detección de objetos, reconocimiento facial, vigilancia de vídeo o procesamiento de imagen médica. La segmentación de imágenes es un problema difícil en general, pero especialmente en entornos científicos y biomédicos, donde las técnicas de adquisición imagen proporcionan imágenes ruidosas. Además, en muchos de estos casos se necesita una precisión casi perfecta. En esta tesis, revisamos y comparamos primero algunas de las técnicas ampliamente usadas para la segmentación de imágenes médicas. Estas técnicas usan clasificadores a nivel de pixel e introducen regularización sobre pares de píxeles que es normalmente insuficiente. Estudiamos las dificultades que presentan para capturar la información de alto nivel sobre los objetos a segmentar. Esta deficiencia da lugar a detecciones erróneas, bordes irregulares, configuraciones con topología errónea y formas inválidas. Para solucionar estos problemas, proponemos un nuevo método de regularización de alto nivel que aprende información topológica y de forma a partir de los datos de entrenamiento de una forma no paramétrica usando potenciales de orden superior. Los potenciales de orden superior se están popularizando en visión por computador, pero la representación exacta de un potencial de orden superior definido sobre muchas variables es computacionalmente inviable. Usamos una representación compacta de los potenciales basada en un conjunto finito de patrones aprendidos de los datos de entrenamiento que, a su vez, depende de las observaciones. Gracias a esta representación, los potenciales de orden superior pueden ser convertidos a potenciales de orden 2 con algunas variables auxiliares añadidas. Experimentos con imágenes reales y sintéticas confirman que nuestro modelo soluciona los errores de aproximaciones más débiles. Incluso con una regularización de alto nivel, una precisión exacta es inalcanzable, y se requeire de edición manual de los resultados de la segmentación automática. La edición manual es tediosa y pesada, y cualquier herramienta de ayuda es muy apreciada. Estas herramientas necesitan ser precisas, pero también lo suficientemente rápidas para ser usadas de forma interactiva. Los contornos activos son una buena solución: son buenos para detecciones precisas de fronteras y, en lugar de buscar una solución global, proporcionan un ajuste fino a resultados que ya existían previamente. Sin embargo, requieren una representación implícita que les permita trabajar con cambios topológicos del contorno, y esto da lugar a ecuaciones en derivadas parciales (EDP) que son costosas de resolver computacionalmente y pueden presentar problemas de estabilidad numérica. Presentamos una aproximación morfológica a la evolución de contornos basada en un nuevo operador morfológico de curvatura que es válido para superficies de cualquier dimensión. Aproximamos la solución numérica de la EDP de la evolución de contorno mediante la aplicación sucesiva de un conjunto de operadores morfológicos aplicados sobre una función de conjuntos de nivel. Estos operadores son muy rápidos, no sufren de problemas de estabilidad numérica y no degradan la función de los conjuntos de nivel, de modo que no hay necesidad de reinicializarlo. Además, su implementación es mucho más sencilla que la de las EDP, ya que no requieren usar sofisticados algoritmos numéricos. Desde un punto de vista teórico, profundizamos en las conexiones entre operadores morfológicos y diferenciales, e introducimos nuevos resultados en este área. Validamos nuestra aproximación proporcionando una implementación morfológica de los contornos geodésicos activos, los contornos activos sin bordes, y los turbopíxeles. En los experimentos realizados, las implementaciones morfológicas convergen a soluciones equivalentes a aquéllas logradas mediante soluciones numéricas tradicionales, pero con ganancias significativas en simplicidad, velocidad y estabilidad. ABSTRACT Image segmentation is an important field in computer vision and one of its most active research areas, with applications in image understanding, object detection, face recognition, video surveillance or medical image processing. Image segmentation is a challenging problem in general, but especially in the biological and medical image fields, where the imaging techniques usually produce cluttered and noisy images and near-perfect accuracy is required in many cases. In this thesis we first review and compare some standard techniques widely used for medical image segmentation. These techniques use pixel-wise classifiers and introduce weak pairwise regularization which is insufficient in many cases. We study their difficulties to capture high-level structural information about the objects to segment. This deficiency leads to many erroneous detections, ragged boundaries, incorrect topological configurations and wrong shapes. To deal with these problems, we propose a new regularization method that learns shape and topological information from training data in a nonparametric way using high-order potentials. High-order potentials are becoming increasingly popular in computer vision. However, the exact representation of a general higher order potential defined over many variables is computationally infeasible. We use a compact representation of the potentials based on a finite set of patterns learned fromtraining data that, in turn, depends on the observations. Thanks to this representation, high-order potentials can be converted into pairwise potentials with some added auxiliary variables and minimized with tree-reweighted message passing (TRW) and belief propagation (BP) techniques. Both synthetic and real experiments confirm that our model fixes the errors of weaker approaches. Even with high-level regularization, perfect accuracy is still unattainable, and human editing of the segmentation results is necessary. The manual edition is tedious and cumbersome, and tools that assist the user are greatly appreciated. These tools need to be precise, but also fast enough to be used in real-time. Active contours are a good solution: they are good for precise boundary detection and, instead of finding a global solution, they provide a fine tuning to previously existing results. However, they require an implicit representation to deal with topological changes of the contour, and this leads to PDEs that are computationally costly to solve and may present numerical stability issues. We present a morphological approach to contour evolution based on a new curvature morphological operator valid for surfaces of any dimension. We approximate the numerical solution of the contour evolution PDE by the successive application of a set of morphological operators defined on a binary level-set. These operators are very fast, do not suffer numerical stability issues, and do not degrade the level set function, so there is no need to reinitialize it. Moreover, their implementation is much easier than their PDE counterpart, since they do not require the use of sophisticated numerical algorithms. From a theoretical point of view, we delve into the connections between differential andmorphological operators, and introduce novel results in this area. We validate the approach providing amorphological implementation of the geodesic active contours, the active contours without borders, and turbopixels. In the experiments conducted, the morphological implementations converge to solutions equivalent to those achieved by traditional numerical solutions, but with significant gains in simplicity, speed, and stability.

Optimización de una energía mediante cortes de grafos. Segmentación de imágenes

La segmentación de imágenes puede plantearse como un problema de minimización de una energía discreta. Nos enfrentamos así a una doble cuestión: definir una energía cuyo mínimo proporcione la segmentación buscada y, una vez definida la energía, encontrar un mínimo absoluto de la misma. La primera parte de esta tesis aborda el segundo problema, y la segunda parte, en un contexto más aplicado, el primero. Las técnicas de minimización basadas en cortes de grafos permiten obtener el mínimo de una energía discreta en tiempo polinomial mediante algoritmos de tipo min-cut/max-flow. Sin embargo, estas técnicas solo pueden aplicarse a energías que son representabas por grafos. Un importante reto es estudiar qué energías son representabas así como encontrar un grafo que las represente, lo que equivale a encontrar una función gadget con variables adicionales. En la primera parte de este trabajo se estudian propiedades de las funciones gadgets que permiten acotar superiormente el número de variables adicionales. Además se caracterizan las energías con cuatro variables que son representabas, definiendo gadgets con dos variables adicionales. En la segunda parte, más práctica, se aborda el problema de segmentación de imágenes médicas, base en muchas ocasiones para la diagnosis y el seguimiento de terapias. La segmentación multi-atlas es una potente técnica de segmentación automática de imágenes médicas, con tres aspectos importantes a destacar: el tipo de registro entre los atlas y la imagen objetivo, la selección de atlas y el método de fusión de etiquetas. Este último punto puede formularse como un problema de minimización de una energía. A este respecto introducimos dos nuevas energías representables. La primera, de orden dos, se utiliza en la segmentación en hígado y fondo de imágenes abdominales obtenidas mediante tomografía axial computarizada. La segunda, de orden superior, se utiliza en la segmentación en hipocampos y fondo de imágenes cerebrales obtenidas mediante resonancia magnética. ABSTRACT The image segmentation can be described as the problem of minimizing a discrete energy. We face two problems: first, to define an energy whose minimum provides the desired segmentation and, second, once the energy is defined we must find its global minimum. The first part of this thesis addresses the second problem, and the second part, in a more applied context, the first problem. Minimization techniques based on graph cuts find the minimum of a discrete energy in polynomial time via min-cut/max-flow algorithms. Nevertheless, these techniques can only be applied to graph-representable energies. An important challenge is to study which energies are graph-representable and to construct graphs which represent these energies. This is the same as finding a gadget function with additional variables. In the first part there are studied the properties of gadget functions which allow the number of additional variables to be bounded from above. Moreover, the graph-representable energies with four variables are characterised and gadgets with two additional variables are defined for these. The second part addresses the application of these ideas to medical image segmentation. This is often the first step in computer-assisted diagnosis and monitoring therapy. Multiatlas segmentation is a powerful automatic segmentation technique for medical images, with three important aspects that are highlighted here: the registration between the atlas and the target image, the atlas selection, and the label fusion method. We formulate the label fusion method as a minimization problem and we introduce two new graph-representable energies. The first is a second order energy and it is used for the segmentation of the liver in computed tomography (CT) images. The second energy is a higher order energy and it is used for the segmentation of the hippocampus in magnetic resonance images (MRI).

Nonrigid Motion Compensation of Free Breathing Acquired Myocardial Perfusion Data

In this work, we present a novel method to compensate the movement in images acquired during free breathing using first-pass gadolinium enhanced, myocardial perfusion magnetic resonance imaging (MRI). First, we use independent component analysis (ICA) to identify the optimal number of independent components (ICs) that separate the breathing motion from the intensity change induced by the contrast agent. Then, synthetic images are created by recombining the ICs, but other then in previously published work (Milles et al. 2008), we omit the component related to motion, and therefore, the resulting reference image series is free of motion. Motion compensation is then achieved by using a multi-pass non-rigid image registration scheme. We tested our method on 15 distinct image series (5 patients) consisting of 58 images each and we validated our method by comparing manually tracked intensity profiles of the myocardial sections to automatically generated ones before and after registration. The average correlation to the manually obtained curves before registration 0:89 0:11 was increased to 0:98 0:02