Multimodal non-rigid image registration methods for therapy and surgery planning

La planificación pre-operatoria se ha convertido en una tarea esencial en cirugías y terapias de marcada complejidad, especialmente aquellas relacionadas con órgano blando. Un ejemplo donde la planificación preoperatoria tiene gran interés es la cirugía hepática. Dicha planificación comprende la detección e identificación precisa de las lesiones individuales y vasos así como la correcta segmentación y estimación volumétrica del hígado funcional. Este proceso es muy importante porque determina tanto si el paciente es un candidato adecuado para terapia quirúrgica como la definición del abordaje a seguir en el procedimiento. La radioterapia de órgano blando es un segundo ejemplo donde la planificación se requiere tanto para la radioterapia externa convencional como para la radioterapia intraoperatoria. La planificación comprende la segmentación de tumor y órganos vulnerables y la estimación de la dosimetría. La segmentación de hígado funcional y la estimación volumétrica para planificación de la cirugía se estiman habitualmente a partir de imágenes de tomografía computarizada (TC). De igual modo, en la planificación de radioterapia, los objetivos de la radiación se delinean normalmente sobre TC. Sin embargo, los avances en las tecnologías de imagen de resonancia magnética (RM) están ofreciendo progresivamente ventajas adicionales. Por ejemplo, se ha visto que el ratio de detección de metástasis hepáticas es significativamente superior en RM con contraste Gd–EOB–DTPA que en TC. Por tanto, recientes estudios han destacado la importancia de combinar la información de TC y RM para conseguir el mayor nivel posible de precisión en radioterapia y para facilitar una descripción precisa de las lesiones del hígado. Con el objetivo de mejorar la planificación preoperatoria en ambos escenarios se precisa claramente de un algoritmo de registro no rígido de imagen. Sin embargo, la gran mayoría de sistemas comerciales solo proporcionan métodos de registro rígido. Las medidas de intensidad de voxel han demostrado ser criterios de similitud de imágenes robustos, y, entre ellas, la Información Mutua (IM) es siempre la primera elegida en registros multimodales. Sin embargo, uno de los principales problemas de la IM es la ausencia de información espacial y la asunción de que las relaciones estadísticas entre las imágenes son homogéneas a lo largo de su domino completo. La hipótesis de esta tesis es que la incorporación de información espacial de órganos al proceso de registro puede mejorar la robustez y calidad del mismo, beneficiándose de la disponibilidad de las segmentaciones clínicas. En este trabajo, se propone y valida un esquema de registro multimodal no rígido 3D usando una nueva métrica llamada Información Mutua Centrada en el Órgano (Organ-Focused Mutual Information metric (OF-MI)) y se compara con la formulación clásica de la Información Mutua. Esto permite mejorar los resultados del registro en áreas problemáticas incorporando información regional al criterio de similitud, beneficiándose de la disponibilidad real de segmentaciones en protocolos estándares clínicos, y permitiendo que la dependencia estadística entre las dos modalidades de imagen difiera entre órganos o regiones. El método propuesto se ha aplicado al registro de TC y RM con contraste Gd–EOB–DTPA así como al registro de imágenes de TC y MR para planificación de radioterapia intraoperatoria rectal. Adicionalmente, se ha desarrollado un algoritmo de apoyo de segmentación 3D basado en Level-Sets para la incorporación de la información de órgano en el registro. El algoritmo de segmentación se ha diseñado específicamente para la estimación volumétrica de hígado sano funcional y ha demostrado un buen funcionamiento en un conjunto de imágenes de TC abdominales. Los resultados muestran una mejora estadísticamente significativa de OF-MI comparada con la Información Mutua clásica en las medidas de calidad de los registros; tanto con datos simulados (p<0.001) como con datos reales en registro hepático de TC y RM con contraste Gd– EOB–DTPA y en registro para planificación de radioterapia rectal usando OF-MI multi-órgano (p<0.05). Adicionalmente, OF-MI presenta resultados más estables con menor dispersión que la Información Mutua y un comportamiento más robusto con respecto a cambios en la relación señal-ruido y a la variación de parámetros. La métrica OF-MI propuesta en esta tesis presenta siempre igual o mayor precisión que la clásica Información Mutua y consecuentemente puede ser una muy buena alternativa en aplicaciones donde la robustez del método y la facilidad en la elección de parámetros sean particularmente importantes. Abstract Pre-operative planning has become an essential task in complex surgeries and therapies, especially for those affecting soft tissue. One example where soft tissue preoperative planning is of high interest is liver surgery. It involves the accurate detection and identification of individual liver lesions and vessels as well as the proper functional liver segmentation and volume estimation. This process is very important because it determines whether the patient is a suitable candidate for surgical therapy and the type of procedure. Soft tissue radiation therapy is a second example where planning is required for both conventional external and intraoperative radiotherapy. It involves the segmentation of the tumor target and vulnerable organs and the estimation of the planned dose. Functional liver segmentations and volume estimations for surgery planning are commonly estimated from computed tomography (CT) images. Similarly, in radiation therapy planning, targets to be irradiated and healthy and vulnerable tissues to be protected from irradiation are commonly delineated on CT scans. However, developments in magnetic resonance imaging (MRI) technology are progressively offering advantages. For instance, the hepatic metastasis detection rate has been found to be significantly higher in Gd–EOB–DTPAenhanced MRI than in CT. Therefore, recent studies highlight the importance of combining the information from CT and MRI to achieve the highest level of accuracy in radiotherapy and to facilitate accurate liver lesion description. In order to improve those two soft tissue pre operative planning scenarios, an accurate nonrigid image registration algorithm is clearly required. However, the vast majority of commercial systems only provide rigid registration. Voxel intensity measures have been shown to be robust measures of image similarity, and among them, Mutual Information (MI) is always the first candidate in multimodal registrations. However, one of the main drawbacks of Mutual Information is the absence of spatial information and the assumption that statistical relationships between images are the same over the whole domain of the image. The hypothesis of the present thesis is that incorporating spatial organ information into the registration process may improve the registration robustness and quality, taking advantage of the clinical segmentations availability. In this work, a multimodal nonrigid 3D registration framework using a new Organ- Focused Mutual Information metric (OF-MI) is proposed, validated and compared to the classical formulation of the Mutual Information (MI). It allows improving registration results in problematic areas by adding regional information into the similitude criterion taking advantage of actual segmentations availability in standard clinical protocols and allowing the statistical dependence between the two modalities differ among organs or regions. The proposed method is applied to CT and T1 weighted delayed Gd–EOB–DTPA-enhanced MRI registration as well as to register CT and MRI images in rectal intraoperative radiotherapy planning. Additionally, a 3D support segmentation algorithm based on Level-Sets has been developed for the incorporation of the organ information into the registration. The segmentation algorithm has been specifically designed for the healthy and functional liver volume estimation demonstrating good performance in a set of abdominal CT studies. Results show a statistical significant improvement of registration quality measures with OF-MI compared to MI with both simulated data (p<0.001) and real data in liver applications registering CT and Gd–EOB–DTPA-enhanced MRI and in registration for rectal radiotherapy planning using multi-organ OF-MI (p<0.05). Additionally, OF-MI presents more stable results with smaller dispersion than MI and a more robust behavior with respect to SNR changes and parameters variation. The proposed OF-MI always presents equal or better accuracy than the classical MI and consequently can be a very convenient alternative within applications where the robustness of the method and the facility to choose the parameters are particularly important.

Single optical surface imaging designs with unconstrained object to image mapping

In this work, novel imaging designs with a single optical surface (either refractive or reflective) are presented. In some of these designs, both object and image shapes are given but mapping from object to image is obtained as a result of the design. In other designs, not only the mapping is obtained in the design process, but also the shape of the object is found. In the examples considered, the image is virtual and located at infinity and is seen from known pupil, which can emulate a human eye. In the first introductory part, 2D designs have been done using three different design methods: a SMS design, a compound Cartesian oval surface, and a differential equation method for the limit case of small pupil. At the point-size pupil limit, it is proven that these three methods coincide. In the second part, previous 2D designs are extended to 3D by rotation and the astigmatism of the image has been studied. As an advanced variation, the differential equation method is used to provide the freedom to control the tangential rays and sagittal rays simultaneously. As a result, designs without astigmatism (at the small pupil limit) on a curved object surface have been obtained. Finally, this anastigmatic differential equation method has been extended to 3D for the general case, in which freeform surfaces are designed.

MBIS: Multivariate Bayesian Image Segmentation Tool

We present MBIS (Multivariate Bayesian Image Segmentation tool), a clustering tool based on the mixture of multivariate normal distributions model. MBIS supports multi-channel bias field correction based on a B-spline model. A second methodological novelty is the inclusion of graph-cuts optimization for the stationary anisotropic hidden Markov random field model. Along with MBIS, we release an evaluation framework that contains three different experiments on multi-site data. We first validate the accuracy of segmentation and the estimated bias field for each channel. MBIS outperforms a widely used segmentation tool in a cross-comparison evaluation. The second experiment demonstrates the robustness of results on atlas-free segmentation of two image sets from scan-rescan protocols on 21 healthy subjects. Multivariate segmentation is more replicable than the monospectral counterpart on T1-weighted images. Finally, we provide a third experiment to illustrate how MBIS can be used in a large-scale study of tissue volume change with increasing age in 584 healthy subjects. This last result is meaningful as multivariate segmentation performs robustly without the need for prior knowledge.

Segunda vista del teatro de Sagunto [Material gráfico] = Seconde vue du théâtre de Sagonte = Second view of the theatre of Sagonta

Estampado junto a: "Vista de la Ciudadela"

A study of feature combination for vehicle detection based on image processing

Video analytics play a critical role in most recent traffic monitoring and driver assistance systems. In this context, the correct detection and classification of surrounding vehicles through image analysis has been the focus of extensive research in the last years. Most of the pieces of work reported for image-based vehicle verification make use of supervised classification approaches and resort to techniques, such as histograms of oriented gradients (HOG), principal component analysis (PCA), and Gabor filters, among others. Unfortunately, existing approaches are lacking in two respects: first, comparison between methods using a common body of work has not been addressed; second, no study of the combination potentiality of popular features for vehicle classification has been reported. In this study the performance of the different techniques is first reviewed and compared using a common public database. Then, the combination capabilities of these techniques are explored and a methodology is presented for the fusion of classifiers built upon them, taking into account also the vehicle pose. The study unveils the limitations of single-feature based classification and makes clear that fusion of classifiers is highly beneficial for vehicle verification.

The second STE12 homologue of Cryptococcus neoformans is MATa-specific and plays an important role in virulence

Cryptococcus neoformans STE12α, a homologue of Saccharomyces cerevisiae STE12, exists only in MATα strains. We identified another STE12 homologue, STE12a, which is MATa specific. As in the case with Δste12α, the mating efficiency for Δste12a was reduced significantly. The Δste12a strains surprisingly still mated with Δste12α strains. In MATα strains, STE12a functionally complemented STE12α for mating efficacy, haploid fruiting, and regulation of capsule size in the mouse brain. Furthermore, when STE12a was replaced with two copies of STE12α, the resulting MATa strain produced hyphae on filament agar. STE12a regulates mRNA levels of several genes that are important for virulence including CNLAC1 and CAP genes. STE12a also modulates enzyme activities of phospholipase and superoxide dismutase. Importantly, deletion of STE12a markedly reduced the virulence in mice, as is the case with STE12α. Brain smears of mice infected with the Δste12a strain showed yeast cells with a considerable reduction in capsule size compared with those infected with STE12a strains. When the disrupted locus of ste12a was replaced with a wild-type STE12a gene, both in vivo and in vitro mutant phenotypes were reversed. These results suggest that STE12a and STE12α have similar functions, and that the mating type of the cells influences the alleles to exert their biological effects. C. neoformans, thus, is the first fungal species that contains a mating-type-specific STE12 homologue in each mating type. Our results demonstrate that mating-type-specific genes are not only important for saprobic reproduction but also play an important role for survival of the organism in host tissue.

Three-dimensional image reconstruction of dephosphorylated smooth muscle heavy meromyosin reveals asymmetry in the interaction between myosin heads and placement of subfragment 2

Regulation of the actin-activated ATPase of smooth muscle myosin II is known to involve an interaction between the two heads that is controlled by phosphorylation of the regulatory light chain. However, the three-dimensional structure of this inactivated form has been unknown. We have used a lipid monolayer to obtain two-dimensional crystalline arrays of the unphosphorylated inactive form of smooth muscle heavy meromyosin suitable for structural studies by electron cryomicroscopy of unstained, frozen-hydrated specimens. The three-dimensional structure reveals an asymmetric interaction between the two myosin heads. The ATPase activity of one head is sterically “blocked” because part of its actin-binding interface is positioned onto the converter domain of the second head. ATPase activity of the second head, which can bind actin, appears to be inhibited through stabilization of converter domain movements needed to release phosphate and achieve strong actin binding. When the subfragment 2 domain of heavy meromyosin is oriented as it would be in an actomyosin filament lattice, the position of the heads is very different from that needed to bind actin, suggesting an additional contribution to ATPase inhibition in situ.

High speed image techniques for construction safety net monitoring in outdoor conditions

Póster presentado en SPIE Photonics Europe, Brussels, 16-19 April 2012.

Image Noise Removal on Heterogeneous CPU-GPU Configurations

A parallel algorithm to remove impulsive noise in digital images using heterogeneous CPU/GPU computing is proposed. The parallel denoising algorithm is based on the peer group concept and uses an Euclidean metric. In order to identify the amount of pixels to be allocated in multi-core and GPUs, a performance analysis using large images is presented. A comparison of the parallel implementation in multi-core, GPUs and a combination of both is performed. Performance has been evaluated in terms of execution time and Megapixels/second. We present several optimization strategies especially effective for the multi-core environment, and demonstrate significant performance improvements. The main advantage of the proposed noise removal methodology is its computational speed, which enables efficient filtering of color images in real-time applications.

Targetless image-based method for measuring displacements and strains on concrete surfaces with a consumer camera

Measurement of concrete strain through non-invasive methods is of great importance in civil engineering and structural analysis. Traditional methods use laser speckle and high quality cameras that may result too expensive for many applications. Here we present a method for measuring concrete deformations with a standard reflex camera and image processing for tracking objects in the concretes surface. Two different approaches are presented here. In the first one, on-purpose objects are drawn on the surface, while on the second one we track small defects on the surface due to air bubbles in the hardening process. The method has been tested on a concrete sample under several loading/unloading cycles. A stop-motion sequence of the process has been captured and analyzed. Results have been successfully compared with the values given by a strain gauge. Accuracy of our methods in tracking objects is below 8 μm, in the order of more expensive commercial devices.

New Hampshire, 1816 (Image 1 of 6) (Raster Image)

This layer is a georeferenced raster image of the historic paper map entitled: New Hampshire by recent survey : made under the supreme authority and published according to law by Philip Carrigain ; J.J. Barralet, del. ; W. Harrison, sct., Philada. It was published by Philip Carrigain in 1816. Scale [ca. 1:200,000]. This layer is image 1 of 6 total images, representing the northeast portion of the six sheet source map. The image inside the map neatline is georeferenced to the surface of the earth and fit to the New Hampshire State Plane coordinate system (NAD 1983 in Feet) (Fipszone 2800). All map collar and inset information is also available as part of the raster image, including any inset maps, profiles, statistical tables, directories, text, illustrations, index maps, legends, or other information associated with the principal map. This map shows features such as roads, drainage, public buildings, schools, churches, industry locations (e.g. mills, factories, mines, etc.), selected private buildings with names of property owners, town boundaries, land grants, and more. Relief shown pictorially and by hachures. Includes area notes, text, and table of population. Also includes illustrations: View of the Great Boars Head and Hampton Beach -- The Cap of the White Mountains -- View of the White Mountains from Shelburne; inset maps: States of the Union east of the Hudson with the adjacent British colonies. Scale [ca. 1:1,920,000] -- The middle, southern and western sections of the United States with the territories. Scale [ca. 1:4,900,000]. Includes: ms. additions with updated county boundary & township names.This layer is part of a selection of digitally scanned and georeferenced historic maps of New England from the Harvard Map Collection. These maps typically portray both natural and manmade features. The selection represents a range of regions, originators, ground condition dates, scales, and map purposes.

Nares Strait region, Canada and Greenland, 1853 (Raster Image)

This layer is a georeferenced raster image of the historic paper map entitled: Chart exhibiting the discoveries of the second American-Grinnell-Expedition in search of Sir John Franklin : unrevised from the original material and projected on the spot by E.K. Kane. It was published by Lith of J. Bien in [1855]. Scale [ca. 1:400,000]. Covers the Nares Strait region, Greenland and Canada. The image inside the map neatline is georeferenced to the surface of the earth and fit to the 'NAD 1983 CSRS UTM Zone 19 North' projection. All map collar and inset information is also available as part of the raster image, including any inset maps, profiles, statistical tables, directories, text, illustrations, index maps, legends, or other information associated with the principal map. This map shows coastal features such as drainage, islands, capes, bays, tides, lines of ice, camps, and more. Relief shown by hachures. This layer is part of a selection of digitally scanned and georeferenced historic maps from the Harvard Map Collection and the Harvard University Library as part of the Open Collections Program at Harvard University project: Organizing Our World: Sponsored Exploration and Scientific Discovery in the Modern Age. Maps selected for the project correspond to various expeditions and represent a range of regions, originators, ground condition dates, scales, and purposes.

Post Route Map of New York, 1871 (Image 1 of 2) (Raster Image)

This layer is a georeferenced raster image of the historic paper map entitled: Post route map of the state of New York : and parts of Vermont, Massachusetts, Connecticut, New Jersey and Pennsylvania, showing also the adjacent portions of the dominion of Canada, designed and constructed under the orders of Postmaster General, Alex. W. Randall, and second asst. Postmaster General, Geo. Wm. McLellan, by W.L. Nicholson, topographer of P.O. Dept. It was published by U.S. Post Office Dept. in 1868. Ink notation: "The service on this diagram brought up to date of Jan. 1st, 1871." Scale [1:380,160]. This layer is image 1 of 2 total images of the two sheet source map. The image inside the map neatline is georeferenced to the surface of the earth and fit to the Universal Transverse Mercator (UTM) Zone 18N NAD83 projection. All map collar and inset information is also available as part of the raster image, including any inset maps, profiles, statistical tables, directories, text, illustrations, index maps, legends, or other information associated with the principal map. This map shows features such as roads, railroads, postal routes (with delivery schedule frequency and distances), drainage, cities and towns, counties, and more. Includes insets: Postal service of Long Island with the principal mail connections of the city of New York. Scale [1:380,160] -- [Parts of the provinces of Québec and Ontario bordering New York and Vermont]. Includes tables of distances and statistics. This layer is part of a selection of digitally scanned and georeferenced historic maps from The Harvard Map Collection as part of the Imaging the Urban Environment project. Maps selected for this project represent major urban areas and cities of the world, at various time periods. These maps typically portray both natural and manmade features at a large scale. The selection represents a range of regions, originators, ground condition dates, scales, and purposes.

Post Route Map of New York, 1871 (Image 2 of 2) (Raster Image)

This layer is a georeferenced raster image of the historic paper map entitled: Post route map of the state of New York : and parts of Vermont, Massachusetts, Connecticut, New Jersey and Pennsylvania, showing also the adjacent portions of the dominion of Canada, designed and constructed under the orders of Postmaster General, Alex. W. Randall, and second asst. Postmaster General, Geo. Wm. McLellan, by W.L. Nicholson, topographer of P.O. Dept. It was published by U.S. Post Office Dept. in 1868. Ink notation: "The service on this diagram brought up to date of Jan. 1st, 1871." Scale [1:380,160]. This layer is image 2 of 2 total images of the two sheet source map. The image inside the map neatline is georeferenced to the surface of the earth and fit to the Universal Transverse Mercator (UTM) Zone 18N NAD83 projection. All map collar and inset information is also available as part of the raster image, including any inset maps, profiles, statistical tables, directories, text, illustrations, index maps, legends, or other information associated with the principal map. This map shows features such as roads, railroads, postal routes (with delivery schedule frequency and distances), drainage, cities and towns, counties, and more. Includes insets: Postal service of Long Island with the principal mail connections of the city of New York. Scale [1:380,160] -- [Parts of the provinces of Québec and Ontario bordering New York and Vermont]. Includes tables of distances and statistics. This layer is part of a selection of digitally scanned and georeferenced historic maps from The Harvard Map Collection as part of the Imaging the Urban Environment project. Maps selected for this project represent major urban areas and cities of the world, at various time periods. These maps typically portray both natural and manmade features at a large scale. The selection represents a range of regions, originators, ground condition dates, scales, and purposes.

Oak Lane, Philadelphia, Pennsylvania, 1900 (Raster Image)

This layer is a georeferenced raster image of the historic paper map entitled: For sale : Lawnton and Oak Lane building lots : twenty-second ward : estate of E. M. Davis, deceased. It was published by J.T. Jackson & Co. ca. 1900. Scale [ca. 1:528,000]. Covers the Oak Lane neighborhood of Philadelphia, Pennsylvania. The image inside the map neatline is georeferenced to the surface of the earth and fit to the Pennsylvania South State Plane Coordinate System NAD83 (in Feet) (Fipszone 3702). All map collar and inset information is also available as part of the raster image, including any inset maps, profiles, statistical tables, directories, text, illustrations, index maps, legends, or other information associated with the principal map. Cadastral map showing buildings, property boundaries and dimensions, names of landowners, streets names, railroad line and station stops, and more. Relief is shown by countours and spot heights. Includes sales and advertisement notice. This layer is part of a selection of digitally scanned and georeferenced historic maps from The Harvard Map Collection as part of the Imaging the Urban Environment project. Maps selected for this project represent major urban areas and cities of the world, at various time periods. These maps typically portray both natural and manmade features at a large scale. The selection represents a range of regions, originators, ground condition dates, scales, and purposes.