Deep convolutional neural networks for semantic video object segmentation

In this thesis, we propose to infer pixel-level labelling in video by utilising only object category information, exploiting the intrinsic structure of video data. Our motivation is the observation that image-level labels are much more easily to be acquired than pixel-level labels, and it is natural to find a link between the image level recognition and pixel level classification in video data, which would transfer learned recognition models from one domain to the other one. To this end, this thesis proposes two domain adaptation approaches to adapt the deep convolutional neural network (CNN) image recognition model trained from labelled image data to the target domain exploiting both semantic evidence learned from CNN, and the intrinsic structures of unlabelled video data. Our proposed approaches explicitly model and compensate for the domain adaptation from the source domain to the target domain which in turn underpins a robust semantic object segmentation method for natural videos. We demonstrate the superior performance of our methods by presenting extensive evaluations on challenging datasets comparing with the state-of-the-art methods.

Text segmentation using a cache memory

International audience

Spectral-based mesh segmentation

In design and manufacturing, mesh segmentation is required for FACE construction in boundary representation (BRep), which in turn is central for featurebased design, machining, parametric CAD and reverse engineering, among others -- Although mesh segmentation is dictated by geometry and topology, this article focuses on the topological aspect (graph spectrum), as we consider that this tool has not been fully exploited -- We preprocess the mesh to obtain a edgelength homogeneous triangle set and its Graph Laplacian is calculated -- We then produce a monotonically increasing permutation of the Fiedler vector (2nd eigenvector of Graph Laplacian) for encoding the connectivity among part feature submeshes -- Within the mutated vector, discontinuities larger than a threshold (interactively set by a human) determine the partition of the original mesh -- We present tests of our method on large complex meshes, which show results which mostly adjust to BRep FACE partition -- The achieved segmentations properly locate most manufacturing features, although it requires human interaction to avoid over segmentation -- Future work includes an iterative application of this algorithm to progressively sever features of the mesh left from previous submesh removals

Location analysis of city sections: socio-demographic segmentation and restaurant potentiality estimation : a case study of Lisbon

One of the objectives of this study is to perform classification of socio-demographic components for the level of city section in City of Lisbon. In order to accomplish suitable platform for the restaurant potentiality map, the socio-demographic components were selected to produce a map of spatial clusters in accordance to restaurant suitability. Consequently, the second objective is to obtain potentiality map in terms of underestimation and overestimation in number of restaurants. To the best of our knowledge there has not been found identical methodology for the estimation of restaurant potentiality. The results were achieved with combination of SOM (Self-Organized Map) which provides a segmentation map and GAM (Generalized Additive Model) with spatial component for restaurant potentiality. Final results indicate that the highest influence in restaurant potentiality is given to tourist sites, spatial autocorrelation in terms of neighboring restaurants (spatial component), and tax value, where lower importance is given to household with 1 or 2 members and employed population, respectively. In addition, an important conclusion is that the most attractive market sites have shown no change or moderate underestimation in terms of restaurants potentiality.

Red wine sweetness preference according to consumer segmentation

The present work aims to evaluate the acceptance and preference for sweet taste in red wine, according to consumer segmentation in age, gender, personality type, tasting sensitivity and consumer experience in wine. A hundred and fourteen wine tasters were invited to the wine tasting, and the average age was 27 years. An addition of sugar was made with equal concentrations of glucose and fructose to the wine at 2g/L, 4g/L, 8g/L, 16g/L and 32g/L. Five pairs of glasses were presented for the subjects to taste containing each a control wine and a spiked sample. Pairs were presented in order of concentration, from 2g/L to 32g/l. The subjects were also asked to answer two online questionnaires at the end of the tasting, on the personality types and vinotype, which is related to mouth sensitivity. ISO-5495 paired comparison tests were used for sensorial analysis. The objective was to assess if any of the nine segmentation factors had influence on preference or rejection for spiked samples and to establish whether this preference was statistically significant. We concluded that it would be important to have subjects with an age average higher than 27 years and more experienced in wine drinking, mostly because the data relative to preferences in novices shows some dispersion and lack of attention. A panel of older and more experienced wine tasters is likely to be more attentive and focused and therefore yield differentiated results. It was also concluded that more research is required to extend this investigation to other wine styles because the differences in preferences can depend on other reasons, such as preferring a wine with more or less sugar according to the type of wine. Finally it was concluded also that some variables influence preference for sweet taste in red wine, such as gender, vinotype and category of experience

Local optimal scale in a hierarchical segmentation method for satellite image: an OBIA approach for the agricultural landscape

Overrecentdecades,remotesensinghasemergedasaneffectivetoolforimprov- ing agriculture productivity. In particular, many works have dealt with the problem of identifying characteristics or phenomena of crops and orchards on different scales using remote sensed images. Since the natural processes are scale dependent and most of them are hierarchically structured, the determination of optimal study scales is mandatory in understanding these processes and their interactions. The concept of multi-scale/multi- resolution inherent to OBIA methodologies allows the scale problem to be dealt with. But for that multi-scale and hierarchical segmentation algorithms are required. The question that remains unsolved is to determine the suitable scale segmentation that allows different objects and phenomena to be characterized in a single image. In this work, an adaptation of the Simple Linear Iterative Clustering (SLIC) algorithm to perform a multi-scale hierarchi- cal segmentation of satellite images is proposed. The selection of the optimal multi-scale segmentation for different regions of the image is carried out by evaluating the intra- variability and inter-heterogeneity of the regions obtained on each scale with respect to the parent-regions defined by the coarsest scale. To achieve this goal, an objective function, that combines weighted variance and the global Moran index, has been used. Two different kinds of experiment have been carried out, generating the number of regions on each scale through linear and dyadic approaches. This methodology has allowed, on the one hand, the detection of objects on different scales and, on the other hand, to represent them all in a sin- gle image. Altogether, the procedure provides the user with a better comprehension of the land cover, the objects on it and the phenomena occurring.

Superpixel-based roughness measure for multispectral satellite image segmentation

The new generation of artificial satellites is providing a huge amount of Earth observation images whose exploitation can report invaluable benefits, both economical and environmental. However, only a small fraction of this data volume has been analyzed, mainly due to the large human resources needed for that task. In this sense, the development of unsupervised methodologies for the analysis of these images is a priority. In this work, a new unsupervised segmentation algorithm for satellite images is proposed. This algorithm is based on the rough-set theory, and it is inspired by a previous segmentation algorithm defined in the RGB color domain. The main contributions of the new algorithm are: (i) extending the original algorithm to four spectral bands; (ii) the concept of the superpixel is used in order to define the neighborhood similarity of a pixel adapted to the local characteristics of each image; (iii) and two new region merged strategies are proposed and evaluated in order to establish the final number of regions in the segmented image. The experimental results show that the proposed approach improves the results provided by the original method when both are applied to satellite images with different spectral and spatial resolutions.

Exchange, know-hows, and interpersonal segmentation : an assessment of the archaic component of the Gaudreau (BkEu-8) site, Weedon, Quebec

Le site Gaudreau est un site perturbé et à occupations multiples situé dans le sud-est du Québec, et présente des occupations datant du Paléoindien Récent jusqu’à la période historique. Les occupations Archaïques du site, noté par la présence de bifaces diagnostiques de l’Archaïque Supérieur et de l’Archaïque Terminal et par des Macrooutils de l’Archaïque Moyen et de l’Archaïque Supérieur, sont le sujet principal de ce mémoire. Puisqu’aucune occupation ne peut être différencié horizontalement ni verticalement, et qu’aucun objet non-diagnostique ne peut être associé avec certitude, seul un échantillon de 32 objets ont été observés. Étant donné la faible taille de l’échantillon analysé, il est fort probable qu’un plus grand nombre de sources de matières premières aient été utilisés durant les occupations de l’Archaïque. Toutefois, un réseau de matières premières lithiques similaire à ceux des sites du Lac Mégantic a été observé, avec une forte représentation de la rhyolite Kineo-Traveller et des cherts Appalachiens. Des cherts des Grands Lacs et le quartzite de Cheshire sont aussi présents. Le mudstone silicifié d’origine locale et le quartz sont par contre faiblement représentés dans l’échantillon, probablement dû à un biais de proximité de source. L’analyse technique de l’échantillon, sans contrôle pour les pratiques techno-économiques, dénote plusieurs récurrences techniques à l’intérieur des unités typologiques, sans toutefois appuyer des différences récurrentes significatives entre les matières premières de régions différentes. À cause de la taille de l’échantillon et du contexte perturbé, la pertinence des fortes similarités entre certains objets est douteuse. La segmentation interpersonnelle des chaînes opératoires ne pouvait être déterminée dans l’échantillon. Cependant, les résultats incitent plutôt à croire que les matières premières devaient circuler sous diverses formes. Il peut être considéré que, en dehors des matières premières locales, les occupants Archaïques du site Gaudreau n’avaient pas d’accès direct aux matières premières exogènes.

Positron Emission Tomography (PET) Tumor Segmentation and Quantification: Development of New Algorithms

Tumor functional volume (FV) and its mean activity concentration (mAC) are the quantities derived from positron emission tomography (PET). These quantities are used for estimating radiation dose for a therapy, evaluating the progression of a disease and also use it as a prognostic indicator for predicting outcome. PET images have low resolution, high noise and affected by partial volume effect (PVE). Manually segmenting each tumor is very cumbersome and very hard to reproduce. To solve the above problem I developed an algorithm, called iterative deconvolution thresholding segmentation (IDTS) algorithm; the algorithm segment the tumor, measures the FV, correct for the PVE and calculates mAC. The algorithm corrects for the PVE without the need to estimate camera’s point spread function (PSF); also does not require optimizing for a specific camera. My algorithm was tested in physical phantom studies, where hollow spheres (0.5-16 ml) were used to represent tumors with a homogeneous activity distribution. It was also tested on irregular shaped tumors with a heterogeneous activity profile which were acquired using physical and simulated phantom. The physical phantom studies were performed with different signal to background ratios (SBR) and with different acquisition times (1-5 min). The algorithm was applied on ten clinical data where the results were compared with manual segmentation and fixed percentage thresholding method called T50 and T60 in which 50% and 60% of the maximum intensity respectively is used as threshold. The average error in FV and mAC calculation was 30% and -35% for 0.5 ml tumor. The average error FV and mAC calculation were ~5% for 16 ml tumor. The overall FV error was ~10% for heterogeneous tumors in physical and simulated phantom data. The FV and mAC error for clinical image compared to manual segmentation was around -17% and 15% respectively. In summary my algorithm has potential to be applied on data acquired from different cameras as its not dependent on knowing the camera’s PSF. The algorithm can also improve dose estimation and treatment planning.

Segmentation and kinematics of the North America-Caribbean plate boundary offshore Hispaniola

We explored the submarine portions of the Enriquillo–Plantain Garden Fault zone (EPGFZ) and the Septentrional–Oriente Fault zone (SOFZ) along the Northern Caribbean plate boundary using high-resolution multibeam echo-sounding and shallow seismic reflection. The bathymetric data shed light on poorly documented or previously unknown submarine fault zones running over 200 km between Haiti and Jamaica (EPGFZ) and 300 km between the Dominican Republic and Cuba (SOFZ). The primary plate-boundary structures are a series of strike-slip fault segments associated with pressure ridges, restraining bends, step overs and dogleg offsets indicating very active tectonics. Several distinct segments 50–100 km long cut across pre-existing structures inherited from former tectonic regimes or bypass recent morphologies formed under the current strike-slip regime. Along the most recent trace of the SOFZ, we measured a strike-slip offset of 16.5 km, which indicates steady activity for the past ~1.8 Ma if its current GPS-derived motion of 9.8 ± 2 mm a−1 has remained stable during the entire Quaternary.

Exchange, know-hows, and interpersonal segmentation : an assessment of the archaic component of the Gaudreau (BkEu-8) site, Weedon, Quebec

Le site Gaudreau est un site perturbé et à occupations multiples situé dans le sud-est du Québec, et présente des occupations datant du Paléoindien Récent jusqu’à la période historique. Les occupations Archaïques du site, noté par la présence de bifaces diagnostiques de l’Archaïque Supérieur et de l’Archaïque Terminal et par des Macrooutils de l’Archaïque Moyen et de l’Archaïque Supérieur, sont le sujet principal de ce mémoire. Puisqu’aucune occupation ne peut être différencié horizontalement ni verticalement, et qu’aucun objet non-diagnostique ne peut être associé avec certitude, seul un échantillon de 32 objets ont été observés. Étant donné la faible taille de l’échantillon analysé, il est fort probable qu’un plus grand nombre de sources de matières premières aient été utilisés durant les occupations de l’Archaïque. Toutefois, un réseau de matières premières lithiques similaire à ceux des sites du Lac Mégantic a été observé, avec une forte représentation de la rhyolite Kineo-Traveller et des cherts Appalachiens. Des cherts des Grands Lacs et le quartzite de Cheshire sont aussi présents. Le mudstone silicifié d’origine locale et le quartz sont par contre faiblement représentés dans l’échantillon, probablement dû à un biais de proximité de source. L’analyse technique de l’échantillon, sans contrôle pour les pratiques techno-économiques, dénote plusieurs récurrences techniques à l’intérieur des unités typologiques, sans toutefois appuyer des différences récurrentes significatives entre les matières premières de régions différentes. À cause de la taille de l’échantillon et du contexte perturbé, la pertinence des fortes similarités entre certains objets est douteuse. La segmentation interpersonnelle des chaînes opératoires ne pouvait être déterminée dans l’échantillon. Cependant, les résultats incitent plutôt à croire que les matières premières devaient circuler sous diverses formes. Il peut être considéré que, en dehors des matières premières locales, les occupants Archaïques du site Gaudreau n’avaient pas d’accès direct aux matières premières exogènes.

Méthodes de segmentation du ventricule gauche basée sur l'algorithme graph cut pour les images par résonance magnétique et échocardiographiques

L’échocardiographie et l’imagerie par résonance magnétique sont toutes deux des techniques non invasives utilisées en clinique afin de diagnostiquer ou faire le suivi de maladies cardiaques. La première mesure un délai entre l’émission et la réception d’ultrasons traversant le corps, tandis que l’autre mesure un signal électromagnétique généré par des protons d’hydrogène présents dans le corps humain. Les résultats des acquisitions de ces deux modalités d’imagerie sont fondamentalement différents, mais contiennent dans les deux cas de l’information sur les structures du coeur humain. La segmentation du ventricule gauche consiste à délimiter les parois internes du muscle cardiaque, le myocarde, afin d’en calculer différentes métriques cliniques utiles au diagnostic et au suivi de différentes maladies cardiaques, telle la quantité de sang qui circule à chaque battement de coeur. Suite à un infarctus ou autre condition, les performances ainsi que la forme du coeur en sont affectées. L’imagerie du ventricule gauche est utilisée afin d’aider les cardiologues à poser les bons diagnostics. Cependant, dessiner les tracés manuels du ventricule gauche requiert un temps non négligeable aux cardiologues experts, d’où l’intérêt pour une méthode de segmentation automatisée fiable et rapide. Ce mémoire porte sur la segmentation du ventricule gauche. La plupart des méthodes existantes sont spécifiques à une seule modalité d’imagerie. Celle proposée dans ce document permet de traiter rapidement des acquisitions provenant de deux modalités avec une précision de segmentation équivalente au tracé manuel d’un expert. Pour y parvenir, elle opère dans un espace anatomique, induisant ainsi une forme a priori implicite. L’algorithme de Graph Cut, combiné avec des stratégies telles les cartes probabilistes et les enveloppes convexes régionales, parvient à générer des résultats qui équivalent (ou qui, pour la majorité des cas, surpassent) l’état de l’art ii Sommaire au moment de la rédaction de ce mémoire. La performance de la méthode proposée, quant à l’état de l’art, a été démontrée lors d’un concours international. Elle est également validée exhaustivement via trois bases de données complètes en se comparant aux tracés manuels de deux experts et des tracés automatisés du logiciel Syngovia. Cette recherche est un projet collaboratif avec l’Université de Bourgogne, en France.

Field-oriented assessment of agricultural crops through temporal segmentation of modis VI data.

Monitoring agricultural crops constitutes a vital task for the general understanding of land use spatio-temporal dynamics. This paper presents an approach for the enhancement of current crop monitoring capabilities on a regional scale, in order to allow for the analysis of environmental and socio-economic drivers and impacts of agricultural land use. This work discusses the advantages and current limitations of using 250m VI data from the Moderate Resolution Imaging Spectroradiometer (MODIS) for this purpose, with emphasis in the difficulty of correctly analyzing pixels whose temporal responses are disturbed due to certain sources of interference such as mixed or heterogeneous land cover. It is shown that the influence of noisy or disturbed pixels can be minimized, and a much more consistent and useful result can be attained, if individual agricultural fields are identified and each field's pixels are analyzed in a collective manner. As such, a method is proposed that makes use of image segmentation techniques based on MODIS temporal information in order to identify portions of the study area that agree with actual agricultural field borders. The pixels of each portion or segment are then analyzed individually in order to estimate the reliability of the temporal signal observed and the consequent relevance of any estimation of land use from that data. The proposed method was applied in the state of Mato Grosso, in mid-western Brazil, where extensive ground truth data was available. Experiments were carried out using several supervised classification algorithms as well as different subsets of land cover classes, in order to test the methodology in a comprehensive way. Results show that the proposed method is capable of consistently improving classification results not only in terms of overall accuracy but also qualitatively by allowing a better understanding of the land use patterns detected. It thus provides a practical and straightforward procedure for enhancing crop-mapping capabilities using temporal series of moderate resolution remote sensing data.

Machine Learning methods for hepatocellular malignancies segmentation and MVI prediction

The aim of this thesis project is to automatically localize HCC tumors in the human liver and subsequently predict if the tumor will undergo microvascular infiltration (MVI), the initial stage of metastasis development. The input data for the work have been partially supplied by Sant'Orsola Hospital and partially downloaded from online medical databases. Two Unet models have been implemented for the automatic segmentation of the livers and the HCC malignancies within it. The segmentation models have been evaluated with the Intersection-over-Union and the Dice Coefficient metrics. The outcomes obtained for the liver automatic segmentation are quite good (IOU = 0.82; DC = 0.35); the outcomes obtained for the tumor automatic segmentation (IOU = 0.35; DC = 0.46) are, instead, affected by some limitations: it can be state that the algorithm is almost always able to detect the location of the tumor, but it tends to underestimate its dimensions. The purpose is to achieve the CT images of the HCC tumors, necessary for features extraction. The 14 Haralick features calculated from the 3D-GLCM, the 120 Radiomic features and the patients' clinical information are collected to build a dataset of 153 features. Now, the goal is to build a model able to discriminate, based on the features given, the tumors that will undergo MVI and those that will not. This task can be seen as a classification problem: each tumor needs to be classified either as “MVI positive” or “MVI negative”. Techniques for features selection are implemented to identify the most descriptive features for the problem at hand and then, a set of classification models are trained and compared. Among all, the models with the best performances (around 80-84% ± 8-15%) result to be the XGBoost Classifier, the SDG Classifier and the Logist Regression models (without penalization and with Lasso, Ridge or Elastic Net penalization).

Supervised and weakly supervised counting-by-segmentation: the fluorescent microscopy use case

This thesis focuses on automating the time-consuming task of manually counting activated neurons in fluorescent microscopy images, which is used to study the mechanisms underlying torpor. The traditional method of manual annotation can introduce bias and delay the outcome of experiments, so the author investigates a deep-learning-based procedure to automatize this task. The author explores two of the main convolutional-neural-network (CNNs) state-of-the-art architectures: UNet and ResUnet family model, and uses a counting-by-segmentation strategy to provide a justification of the objects considered during the counting process. The author also explores a weakly-supervised learning strategy that exploits only dot annotations. The author quantifies the advantages in terms of data reduction and counting performance boost obtainable with a transfer-learning approach and, specifically, a fine-tuning procedure. The author released the dataset used for the supervised use case and all the pre-training models, and designed a web application to share both the counting process pipeline developed in this work and the models pre-trained on the dataset analyzed in this work.