Domain Adaptation in remote sensing: increasing the portability of land-cover classifiers

Among the types of remote sensing acquisitions, optical images are certainly one of the most widely relied upon data sources for Earth observation. They provide detailed measurements of the electromagnetic radiation reflected or emitted by each pixel in the scene. Through a process termed supervised land-cover classification, this allows to automatically yet accurately distinguish objects at the surface of our planet. In this respect, when producing a land-cover map of the surveyed area, the availability of training examples representative of each thematic class is crucial for the success of the classification procedure. However, in real applications, due to several constraints on the sample collection process, labeled pixels are usually scarce. When analyzing an image for which those key samples are unavailable, a viable solution consists in resorting to the ground truth data of other previously acquired images. This option is attractive but several factors such as atmospheric, ground and acquisition conditions can cause radiometric differences between the images, hindering therefore the transfer of knowledge from one image to another. The goal of this Thesis is to supply remote sensing image analysts with suitable processing techniques to ensure a robust portability of the classification models across different images. The ultimate purpose is to map the land-cover classes over large spatial and temporal extents with minimal ground information. To overcome, or simply quantify, the observed shifts in the statistical distribution of the spectra of the materials, we study four approaches issued from the field of machine learning. First, we propose a strategy to intelligently sample the image of interest to collect the labels only in correspondence of the most useful pixels. This iterative routine is based on a constant evaluation of the pertinence to the new image of the initial training data actually belonging to a different image. Second, an approach to reduce the radiometric differences among the images by projecting the respective pixels in a common new data space is presented. We analyze a kernel-based feature extraction framework suited for such problems, showing that, after this relative normalization, the cross-image generalization abilities of a classifier are highly increased. Third, we test a new data-driven measure of distance between probability distributions to assess the distortions caused by differences in the acquisition geometry affecting series of multi-angle images. Also, we gauge the portability of classification models through the sequences. In both exercises, the efficacy of classic physically- and statistically-based normalization methods is discussed. Finally, we explore a new family of approaches based on sparse representations of the samples to reciprocally convert the data space of two images. The projection function bridging the images allows a synthesis of new pixels with more similar characteristics ultimately facilitating the land-cover mapping across images.

Surface reconstruction from microscopic images in optical lithography.

This paper presents a method to reconstruct 3D surfaces of silicon wafers from 2D images of printed circuits taken with a scanning electron microscope. Our reconstruction method combines the physical model of the optical acquisition system with prior knowledge about the shapes of the patterns in the circuit; the result is a shape-from-shading technique with a shape prior. The reconstruction of the surface is formulated as an optimization problem with an objective functional that combines a data-fidelity term on the microscopic image with two prior terms on the surface. The data term models the acquisition system through the irradiance equation characteristic of the microscope; the first prior is a smoothness penalty on the reconstructed surface, and the second prior constrains the shape of the surface to agree with the expected shape of the pattern in the circuit. In order to account for the variability of the manufacturing process, this second prior includes a deformation field that allows a nonlinear elastic deformation between the expected pattern and the reconstructed surface. As a result, the minimization problem has two unknowns, and the reconstruction method provides two outputs: 1) a reconstructed surface and 2) a deformation field. The reconstructed surface is derived from the shading observed in the image and the prior knowledge about the pattern in the circuit, while the deformation field produces a mapping between the expected shape and the reconstructed surface that provides a measure of deviation between the circuit design models and the real manufacturing process.

Optical spectroscopy of the bladder washout fluid to optimize fluorescence cystoscopy with Hexvix®.

Fluorescence cystoscopy enhances detection of early bladder cancer. Water used to inflate the bladder during the procedure rapidly contains urine, which may contain fluorochromes. This frequently degradesfluorescence images. Samples of bladder washout fluid (BWF) or urine were collected (15 subjects). We studiedtheir fluorescence properties and assessed changes induced by pH (4 to 9) and temperature (15°C to 41°C).A typical fluorescence spectrum of BWF features a main peak (excitation/emission: 320∕420 nm, FWHM =50∕100 nm) and a weaker (5% to 20% of main peak intensity), secondary peak (excitation/emission: 455∕525 nm, FWHM = 80∕50 nm). Interpatient fluctuations of fluorescence intensity are observed. Fluorescence intensity decreases when temperature increases (max 30%) or pH values vary (max 25%). Neither approach is compatible with clinical settings. Fluorescence lifetime measurements suggest that 4-pyridoxic acid/riboflavin is the most likely molecule responsible for urine's main/secondary fluorescence peak. Our measurements give an insight into the spectroscopy of the detrimental background fluorescence. This should be included in the optical design of fluorescence cystoscopes. We estimate that restricting the excitation range from 370-430 nm to 395-415 nm would reduce the BWF background by a factor 2.

Kernel-based methods for change detection in remote sensing images

Nowadays, the joint exploitation of images acquired daily by remote sensing instruments and of images available from archives allows a detailed monitoring of the transitions occurring at the surface of the Earth. These modifications of the land cover generate spectral discrepancies that can be detected via the analysis of remote sensing images. Independently from the origin of the images and of type of surface change, a correct processing of such data implies the adoption of flexible, robust and possibly nonlinear method, to correctly account for the complex statistical relationships characterizing the pixels of the images. This Thesis deals with the development and the application of advanced statistical methods for multi-temporal optical remote sensing image processing tasks. Three different families of machine learning models have been explored and fundamental solutions for change detection problems are provided. In the first part, change detection with user supervision has been considered. In a first application, a nonlinear classifier has been applied with the intent of precisely delineating flooded regions from a pair of images. In a second case study, the spatial context of each pixel has been injected into another nonlinear classifier to obtain a precise mapping of new urban structures. In both cases, the user provides the classifier with examples of what he believes has changed or not. In the second part, a completely automatic and unsupervised method for precise binary detection of changes has been proposed. The technique allows a very accurate mapping without any user intervention, resulting particularly useful when readiness and reaction times of the system are a crucial constraint. In the third, the problem of statistical distributions shifting between acquisitions is studied. Two approaches to transform the couple of bi-temporal images and reduce their differences unrelated to changes in land cover are studied. The methods align the distributions of the images, so that the pixel-wise comparison could be carried out with higher accuracy. Furthermore, the second method can deal with images from different sensors, no matter the dimensionality of the data nor the spectral information content. This opens the doors to possible solutions for a crucial problem in the field: detecting changes when the images have been acquired by two different sensors.

Atlas-based segmentation of pathological brain MR images

We propose a method for brain atlas deformation inpresence of large space-occupying tumors, based on an apriori model of lesion growth that assumes radialexpansion of the lesion from its starting point. First,an affine registration brings the atlas and the patientinto global correspondence. Then, the seeding of asynthetic tumor into the brain atlas provides a templatefor the lesion. Finally, the seeded atlas is deformed,combining a method derived from optical flow principlesand a model of lesion growth (MLG). Results show that themethod can be applied to the automatic segmentation ofstructures and substructures in brains with grossdeformation, with important medical applications inneurosurgery, radiosurgery and radiotherapy.

Atlas-based segmentation of pathological MR brain images using a model of lesion growth.

We propose a method for brain atlas deformation in the presence of large space-occupying tumors, based on an a priori model of lesion growth that assumes radial expansion of the lesion from its starting point. Our approach involves three steps. First, an affine registration brings the atlas and the patient into global correspondence. Then, the seeding of a synthetic tumor into the brain atlas provides a template for the lesion. The last step is the deformation of the seeded atlas, combining a method derived from optical flow principles and a model of lesion growth. Results show that a good registration is performed and that the method can be applied to automatic segmentation of structures and substructures in brains with gross deformation, with important medical applications in neurosurgery, radiosurgery, and radiotherapy.

Estimating the apparent transverse relaxation time (R2(*)) from images with different contrasts (ESTATICS) reduces motion artifacts.

Relaxation rates provide important information about tissue microstructure. Multi-parameter mapping (MPM) estimates multiple relaxation parameters from multi-echo FLASH acquisitions with different basic contrasts, i.e., proton density (PD), T1 or magnetization transfer (MT) weighting. Motion can particularly affect maps of the apparent transverse relaxation rate R2(*), which are derived from the signal of PD-weighted images acquired at different echo times. To address the motion artifacts, we introduce ESTATICS, which robustly estimates R2(*) from images even when acquired with different basic contrasts. ESTATICS extends the fitted signal model to account for inherent contrast differences in the PDw, T1w and MTw images. The fit was implemented as a conventional ordinary least squares optimization and as a robust fit with a small or large confidence interval. These three different implementations of ESTATICS were tested on data affected by severe motion artifacts and data with no prominent motion artifacts as determined by visual assessment or fast optical motion tracking. ESTATICS improved the quality of the R2(*) maps and reduced the coefficient of variation for both types of data-with average reductions of 30% when severe motion artifacts were present. ESTATICS can be applied to any protocol comprised of multiple 2D/3D multi-echo FLASH acquisitions as used in the general research and clinical setting.

Images in clinical urology. Nail of glans penis.

A 56-year-old man presented with a "nail" growing at the base of his glans penis. The tumor was locally excised, and microscopic examination revealed papillomatosis and hyperkeratosis of the malpighian epithelium, with a strong inflammatory reaction of the chorion and signs of local microinvasion, as well as the presence of well-differentiated squamous epithelial cells. The surgical margins were negative. The differential diagnosis was made between a benign papillomatous proliferation and verrucous carcinoma.

Détectabilité extra-fovéale sur des images anatomiques

Contexte¦- Les métastases hépatiques hypovasculaires sont parfois difficile à détecter car très polymorphiques et fréquemment irrégulières. Leurs contrastes sur CT scan hépatique sont souvent faibles.¦- Lors d'un diagnostic, le radiologue ne fixe pas sa vision fovéale sur chaque pixel de l'image. Les expériences de psychophysique avec eye-tracker montrent en effet que le radiologue se concentre sur quelques points spécifiques de l'image appelés fixations. Dans ce travail, nous nous intéresserons aux capacités de détection de l'oeil lorsque l'observateur effectue une saccade entre deux points de fixation. Plus particulièrement, nous nous intéresserons à caractériser les capacités de l'oeil à détecter les signaux se trouvant en dehors de sa vision fovéale, dans ce qu'on appelle, la vision périphérique.¦Objectifs¦- Caractériser l'effet de l'excentricité de la vision sur la détectabilité des contrastes dans le cas de métastases hépatiques hypovasculaires.¦- Récolter des données expérimentales en vue de créer un modèle mathématique qui permettra, à terme, de qualifier le système d'imagerie.¦- → objectifs du TM en soit :¦o prendre en main l'eyetracker¦o traduire une problématique médicale en une expérience scientifique reproductible, quantifiable et qualifiable.¦Méthode¦Nous effectuons une expérience 2AFC (2 Alternative Forced-Choice experiment) afin d'estimer la détectabilité du signal. Pour cela, nous forcerons l'observateur à maintenir son point de fixation à un endroit défini et vérifié par l'eye-tracker. La position del'excentricité du signal tumoral généré sur une coupe de CT hépatique sera le paramètre varié. L'observateur se verra présenté tour à tour deux coupes de CT hépatique, l'une comportant le signal tumoral standardisé et l'autre ne comportant pas le signal. L'observateur devra déterminer quelle image contient la pathologie avec la plus grande probabilité.¦- Cette expérience est un modèle simplifié de la réalité. En effet, le radiologue ne fixe pas un seul point lors de sa recherche mais effectue un "scanpath". Une seconde expérience, dite en free search sera effectuée dans la mesure du temps à disposition. Lors de cette expérience, le signal standardisé sera connu de l'observateur et il n'y aura plus de point de fixation forcée. L'eyetracker suivra le scanpath effectué par l'oeil de l'observateur lors de la recherche du signal sur une coupe de CT scan hépatique. L'intérêt de cette expérience réside dans l'observation de la corrélation entre les saccades et la découverte du signal. Elle permet aussi de vérifier les résultats obtenus lors de la première expérience.¦Résultats escomptés¦- Exp1 : Quantifier l'importance de l'excentricité en radiologie et aider à améliorer la performance de recherche.¦- Exp 2 : tester la validité des résultats obtenus par la première expérience.¦Plus value escomptée¦- Récolte de données pour créer un modèle mathématique capable de déterminer la qualité de l'image radiologique.¦- Possibilité d'extension à la recherche dans les trois dimensions du CT scan hépatique.

Three Generation Family With Pattern Dystrophy and a Successful Treatment of Subfoveal Cnv Related to Pattern Dystrophy With Anti-vegf Intravitreal Injections

Purpose: To phenotype a large 3 generation Swiss family with pattern dystrophy and to report a successful result of treatment with ranibizumab of a subfoveal choroidal neovascularisation (CNV) associated with pattern dystrophy in 1 patient Patients and methods: 4 affected and 3 unaffected patients (3 female 4 male, age range: 19 - 80 years) were assessed with a complete ophthalmologic examination. AF images were taken using Heidelberg Retina Angiograph and the digital color photos, fluorescein angiogragraphy (FFA) using the same TOPCON 501 camera. Electroretinogram (full-field and multifocal) was performed in 1 affected patient. One 48 years old patient developed a subfoveal CNV, which was treated with 2 injections of ranibizumab, at 3 months interval. Blood sample was taken for molecular analysis (screening of the gene RDS). Results: Two patients had a typical fundoscopic appearance of pattern dystrophy with butterfly shaped deposit at the fovea and some peripheral flecks, as shown with AF imaging.. Two others affected patients had a more unusual appearance with some macular atrophy in one or both eyes, surrounded by flecks. The visual acuity ranged from 1.0 to 0.1 according to Snellen EDTRS chart. The patient with subfoveal CNV presented a drop of vision form 1.0 to 0.6 within 10 days prior to the diagnosis and also reported some metamorphopsia. FFA and optical computerized tomography (OCT) confirmed a classic CNV. After the 1st injection her vision improved to 1.0 but persistent metamorphopsia and fluid on OCT motivated a second injection. One month after the second injection the OCT was flat and the patient had no symptoms. The results of RDS screening will be presented at the meeting. Conclusion: We present a family with pattern dystrophy, with some members having an unusual fundus appearance, which was mistaken for an early onset dry AMD. The AF imaging is a useful tool in diagnosing this condition. A CNV associated with pattern dystrophy a rare. This is the first report of a successful treatment of the CNV with anti-VEGF intravitreal injections.