Essays on Machine Learning and Hedonic Models

Thesis (Ph.D.)--University of Washington, 2016-08

Rilevamento della qualità del manto stradale: un approccio sperimentale tramite Machine Learning

Il riconoscimento delle condizioni del manto stradale partendo esclusivamente dai dati raccolti dallo smartphone di un ciclista a bordo del suo mezzo è un ambito di ricerca finora poco esplorato. Per lo sviluppo di questa tesi è stata sviluppata un'apposita applicazione, che combinata a script Python permette di riconoscere differenti tipologie di asfalto. L’applicazione raccoglie i dati rilevati dai sensori di movimento integrati nello smartphone, che registra i movimenti mentre il ciclista è alla guida del suo mezzo. Lo smartphone è fissato in un apposito holder fissato sul manubrio della bicicletta e registra i dati provenienti da giroscopio, accelerometro e magnetometro. I dati sono memorizzati su file CSV, che sono elaborati fino ad ottenere un unico DataSet contenente tutti i dati raccolti con le features estratte mediante appositi script Python. A ogni record sarà assegnato un cluster deciso in base ai risultati prodotti da K-means, risultati utilizzati in seguito per allenare algoritmi Supervised. Lo scopo degli algoritmi è riconoscere la tipologia di manto stradale partendo da questi dati. Per l’allenamento, il DataSet è stato diviso in due parti: il training set dal quale gli algoritmi imparano a classificare i dati e il test set sul quale gli algoritmi applicano ciò che hanno imparato per dare in output la classificazione che ritengono idonea. Confrontando le previsioni degli algoritmi con quello che i dati effettivamente rappresentano si ottiene la misura dell’accuratezza dell’algoritmo.

Developing a radiomic based machine learning model to identify patients with resected non-small-cell lung cancer at high risk of relapse

Background There is a wide variation of recurrence risk of Non-small-cell lung cancer (NSCLC) within the same Tumor Node Metastasis (TNM) stage, suggesting that other parameters are involved in determining this probability. Radiomics allows extraction of quantitative information from images that can be used for clinical purposes. The primary objective of this study is to develop a radiomic prognostic model that predicts a 3 year disease free-survival (DFS) of resected Early Stage (ES) NSCLC patients. Material and Methods 56 pre-surgery non contrast Computed Tomography (CT) scans were retrieved from the PACS of our institution and anonymized. Then they were automatically segmented with an open access deep learning pipeline and reviewed by an experienced radiologist to obtain 3D masks of the NSCLC. Images and masks underwent to resampling normalization and discretization. From the masks hundreds Radiomic Features (RF) were extracted using Py-Radiomics. Hence, RF were reduced to select the most representative features. The remaining RF were used in combination with Clinical parameters to build a DFS prediction model using Leave-one-out cross-validation (LOOCV) with Random Forest. Results and Conclusion A poor agreement between the radiologist and the automatic segmentation algorithm (DICE score of 0.37) was found. Therefore, another experienced radiologist manually segmented the lesions and only stable and reproducible RF were kept. 50 RF demonstrated a high correlation with the DFS but only one was confirmed when clinicopathological covariates were added: Busyness a Neighbouring Gray Tone Difference Matrix (HR 9.610). 16 clinical variables (which comprised TNM) were used to build the LOOCV model demonstrating a higher Area Under the Curve (AUC) when RF were included in the analysis (0.67 vs 0.60) but the difference was not statistically significant (p=0,5147).

Synthetic data generation for the assessment of antimicrobial resistance through machine learning

As a consequence of the diffusion of next generation sequencing techniques, metagenomics databases have become one of the most promising repositories of information about features and behavior of microorganisms. One of the subjects that can be studied from those data are bacteria populations. Next generation sequencing techniques allow to study the bacteria population within an environment by sampling genetic material directly from it, without the needing of culturing a similar population in vitro and observing its behavior. As a drawback, it is quite complex to extract information from those data and usually there is more than one way to do that; AMR is no exception. In this study we will discuss how the quantified AMR, which regards the genotype of the bacteria, can be related to the bacteria phenotype and its actual level of resistance against the specific substance. In order to have a quantitative information about bacteria genotype, we will evaluate the resistome from the read libraries, aligning them against CARD database. With those data, we will test various machine learning algorithms for predicting the bacteria phenotype. The samples that we exploit should resemble those that could be obtained from a natural context, but are actually produced by a read libraries simulation tool. In this way we are able to design the populations with bacteria of known genotype, so that we can relay on a secure ground truth for training and testing our algorithms.

Meta-learning for informed model searches in Automated Machine Learning

Day by day, machine learning is changing our lives in ways we could not have imagined just 5 years ago. ML expertise is more and more requested and needed, though just a limited number of ML engineers are available on the job market, and their knowledge is always limited by an inherent characteristic of theirs: they are humans. This thesis explores the possibilities offered by meta-learning, a new field in ML that takes learning a level higher: models are trained on other models' training data, starting from features of the dataset they were trained on, inference times, obtained performances, to try to understand the relationship between a good model and the way it was obtained. The so-called metamodel was trained on data collected by OpenML, the largest ML metadata platform that's publicly available today. Datasets were analyzed to obtain meta-features that describe them, which were then tied to model performances in a regression task. The obtained metamodel predicts the expected performances of a given model type (e.g., a random forest) on a given ML task (e.g., classification on the UCI census dataset). This research was then integrated into a custom-made AutoML framework, to show how meta-learning is not an end in itself, but it can be used to further progress our ML research. Encoding ML engineering expertise in a model allows better, faster, and more impactful ML applications across the whole world, while reducing the cost that is inevitably tied to human engineers.

tecniche di machine learning per la predizione del rating delle applicazioni all’interno di un play store

Il mio progetto di tesi ha come obiettivo quello di creare un modello in grado di predire il rating delle applicazioni presenti all’interno del Play Store, uno dei più grandi servizi di distribuzione digitale Android. A tale scopo ho utilizzato il linguaggio Python, che grazie alle sue librerie, alla sua semplicità e alla sua versatilità è certamen- te uno dei linguaggi più usati nel campo dell’intelligenza artificiale. Il punto di partenza del mio studio è stato il Dataset (Insieme di dati strutturati in forma relazionale) “Google Play Store Apps” reperibile su Kaggle al seguente indirizzo: https://www.kaggle.com/datasets/lava18/google-play-store-apps, contenente 10841 osservazioni e 13 attributi. Dopo una prima parte relativa al caricamen- to, alla visualizzazione e alla preparazione dei dati su cui lavorare, ho applica- to quattro di↵erenti tecniche di Machine Learning per la stima del rating delle applicazioni. In particolare, sono state utilizzate:https://www.kaggle.com/datasets/lava18/google-play-store-apps, contenente 10841 osservazioni e 13 attributi. Dopo una prima parte relativa al caricamento, alla visualizzazione e alla preparazione dei dati su cui lavorare, ho applicato quattro differenti tecniche di Machine Learning per la stima del rating delle applicazioni: Ridje, Regressione Lineare, Random Forest e SVR. Tali algoritmi sono stati applicati attuando due tipi diversi di trasformazioni (Label Encoding e One Hot Encoding) sulla variabile ‘Category’, con lo scopo di analizzare come le suddette trasformazioni riescano a influire sulla bontà del modello. Ho confrontato poi l’errore quadratico medio (MSE), l’errore medio as- soluto (MAE) e l’errore mediano assoluto (MdAE) con il fine di capire quale sia l’algoritmo più efficiente.

ICE Emissions Estimation through Machine-learning techniques to support real-world R&D experiments

The emissions estimation, both during homologation and standard driving, is one of the new challenges that automotive industries have to face. The new European and American regulation will allow a lower and lower quantity of Carbon Monoxide emission and will require that all the vehicles have to be able to monitor their own pollutants production. Since numerical models are too computationally expensive and approximated, new solutions based on Machine Learning are replacing standard techniques. In this project we considered a real V12 Internal Combustion Engine to propose a novel approach pushing Random Forests to generate meaningful prediction also in extreme cases (extrapolation, very high frequency peaks, noisy instrumentation etc.). The present work proposes also a data preprocessing pipeline for strongly unbalanced datasets and a reinterpretation of the regression problem as a classification problem in a logarithmic quantized domain. Results have been evaluated for two different models representing a pure interpolation scenario (more standard) and an extrapolation scenario, to test the out of bounds robustness of the model. The employed metrics take into account different aspects which can affect the homologation procedure, so the final analysis will focus on combining all the specific performances together to obtain the overall conclusions.

Machine Learning in Automatic Tabulation for Constraint Model Reformulation

Combinatorial decision and optimization problems belong to numerous applications, such as logistics and scheduling, and can be solved with various approaches. Boolean Satisfiability and Constraint Programming solvers are some of the most used ones and their performance is significantly influenced by the model chosen to represent a given problem. This has led to the study of model reformulation methods, one of which is tabulation, that consists in rewriting the expression of a constraint in terms of a table constraint. To apply it, one should identify which constraints can help and which can hinder the solving process. So far this has been performed by hand, for example in MiniZinc, or automatically with manually designed heuristics, in Savile Row. Though, it has been shown that the performances of these heuristics differ across problems and solvers, in some cases helping and in others hindering the solving procedure. However, recent works in the field of combinatorial optimization have shown that Machine Learning (ML) can be increasingly useful in the model reformulation steps. This thesis aims to design a ML approach to identify the instances for which Savile Row’s heuristics should be activated. Additionally, it is possible that the heuristics miss some good tabulation opportunities, so we perform an exploratory analysis for the creation of a ML classifier able to predict whether or not a constraint should be tabulated. The results reached towards the first goal show that a random forest classifier leads to an increase in the performances of 4 different solvers. The experimental results in the second task show that a ML approach could improve the performance of a solver for some problem classes.

Molecular And Morphological Evidence Reveals A New Species In The Phyllomedusa Hypochondrialis Group (hylidae, Phyllomedusinae) From The Atlantic Forest Of The Highlands Of Southern Brazil.

The taxonomic status of a disjunctive population of Phyllomedusa from southern Brazil was diagnosed using molecular, chromosomal, and morphological approaches, which resulted in the recognition of a new species of the P. hypochondrialis group. Here, we describe P. rustica sp. n. from the Atlantic Forest biome, found in natural highland grassland formations on a plateau in the south of Brazil. Phylogenetic inferences placed P. rustica sp. n. in a subclade that includes P. rhodei + all the highland species of the clade. Chromosomal morphology is conservative, supporting the inference of homologies among the karyotypes of the species of this genus. Phyllomedusa rustica is apparently restricted to its type-locality, and we discuss the potential impact on the strategies applied to the conservation of the natural grassland formations found within the Brazilian Atlantic Forest biome in southern Brazil. We suggest that conservation strategies should be modified to guarantee the preservation of this species.

Experiences From The Brazilian Atlantic Forest: Ecological Findings And Conservation Initiatives.

The Brazilian Atlantic Forest hosts one of the world's most diverse and threatened tropical forest biota. In many ways, its history of degradation describes the fate experienced by tropical forests around the world. After five centuries of human expansion, most Atlantic Forest landscapes are archipelagos of small forest fragments surrounded by open-habitat matrices. This 'natural laboratory' has contributed to a better understanding of the evolutionary history and ecology of tropical forests and to determining the extent to which this irreplaceable biota is susceptible to major human disturbances. We share some of the major findings with respect to the responses of tropical forests to human disturbances across multiple biological levels and spatial scales and discuss some of the conservation initiatives adopted in the past decade. First, we provide a short description of the Atlantic Forest biota and its historical degradation. Secondly, we offer conceptual models describing major shifts experienced by tree assemblages at local scales and discuss landscape ecological processes that can help to maintain this biota at larger scales. We also examine potential plant responses to climate change. Finally, we propose a research agenda to improve the conservation value of human-modified landscapes and safeguard the biological heritage of tropical forests.

All Green, But Equal? Morphological Traits And Ecological Implications On Spores Of Three Species Of Mosses In The Brazilian Atlantic Forest.

Spores of the tropical mosses Pyrrhobryum spiniforme, Neckeropsis undulata and N. disticha were characterized regarding size, number per capsule and viability. Chemical substances were analyzed for P. spiniforme and N. undulata spores. Length of sporophyte seta (spore dispersal ability) was analyzed for P. spiniforme. Four to six colonies per species in each site (lowland and highland areas of an Atlantic Forest; Serra do Mar State Park, Brazil) were visited for the collection of capsules (2008 - 2009). Neckeropsis undulata in the highland area produced the largest spores (ca. 19 µm) with the highest viability. The smallest spores were found in N. disticha in the lowland (ca. 13 µm). Pyrrhobryum spiniforme produced more spores per capsule in the highland (ca. 150,000) than in lowland (ca. 40,000); longer sporophytic setae in the lowland (ca. 64 mm) than in the highland (ca. 43 mm); and similar sized spores in both areas (ca. 16 µm). Spores of N. undulata and P. spiniforme contained lipids and proteins in the cytoplasm, and acid/neutral lipids and pectins in the wall. Lipid bodies were larger in N. undulata than in P. spiniforme. No starch was recorded for spores. Pyrrhobryum spiniforme in the highland area, different from lowland, was characterized by low reproductive effort, but presented many spores per capsule.

The Neuromuscular Activity Of Bothriopsis Bilineata Smaragdina (forest Viper) Venom And Its Toxin Bbil-tx (asp49 Phospholipase A2) On Isolated Mouse Nerve-muscle Preparations.

The presynaptic action of Bothriopsis bilineata smaragdina (forest viper) venom and Bbil-TX, an Asp49 PLA2 from this venom, was examined in detail in mouse phrenic nerve-muscle (PND) preparations in vitro and in a neuroblastoma cell line (SK-N-SH) in order to gain a better insight into the mechanism of action of the venom and associated Asp49 PLA2. In low Ca(2+) solution, venom (3μg/ml) caused a quadriphasic response in PND twitch height whilst at 10μg/ml the venom additionally induced an abrupt and marked initial contracture followed by neuromuscular facilitation, rhythmic oscillations of nerve-evoked twitches, alterations in baseline and progressive blockade. The venom slowed the relaxation phase of muscle twitches. In low Ca(2+), Bbil-TX [210nM (3μg/ml)] caused a progressive increase in PND twitch amplitude but no change in the decay time constant. Venom (10μg/ml) and Bbil-TX (210nM) caused minor changes in the compound action potential (CAP) amplitude recorded from sciatic nerve preparations, with no significant effect on rise time and latency; tetrodotoxin (3.1nM) blocked the CAP at the end of the experiments. In mouse triangularis sterni nerve-muscle (TSn-m) preparations, venom (10μg/ml) and Bbil-TX (210nM) significantly reduced the perineural waveform associated with the outward K(+) current while the amplitude of the inward Na(+) current was not significantly affected. Bbil-TX (210nM) caused a progressive increase in the quantal content of TSn-m preparations maintained in low Ca(2+) solution. Venom (3μg/ml) and toxin (210nM) increased the calcium fluorescence in SK-N-SH neuroblastoma cells loaded with Fluo3 AM and maintained in low or normal Ca(2+) solution. In normal Ca(2+), the increase in fluorescence amplitude was accompanied by irregular and frequent calcium transients. In TSn-m preparations loaded with Fluo4 AM, venom (10μg/ml) caused an immediate increase in intracellular Ca(2+) followed by oscillations in fluorescence and muscle contracture; Bbil-TX did not change the calcium fluorescence in TSn-m preparations. Immunohistochemical analysis of toxin-treated PND preparations revealed labeling of junctional ACh receptors but a loss of the presynaptic proteins synaptophysin and SNAP25. Together, these data confirm the presynaptic action of Bbil-TX and show that it involves modulation of K(+) channel activity and presynaptic protein expression.

Environmental Controls In The Water Use Patterns Of A Tropical Cloud Forest Tree Species, Drimys Brasiliensis (winteraceae).

Trees from tropical montane cloud forest (TMCF) display very dynamic patterns of water use. They are capable of downwards water transport towards the soil during leaf-wetting events, likely a consequence of foliar water uptake (FWU), as well as high rates of night-time transpiration (Enight) during drier nights. These two processes might represent important sources of water losses and gains to the plant, but little is known about the environmental factors controlling these water fluxes. We evaluated how contrasting atmospheric and soil water conditions control diurnal, nocturnal and seasonal dynamics of sap flow in Drimys brasiliensis (Miers), a common Neotropical cloud forest species. We monitored the seasonal variation of soil water content, micrometeorological conditions and sap flow of D. brasiliensis trees in the field during wet and dry seasons. We also conducted a greenhouse experiment exposing D. brasiliensis saplings under contrasting soil water conditions to deuterium-labelled fog water. We found that during the night D. brasiliensis possesses heightened stomatal sensitivity to soil drought and vapour pressure deficit, which reduces night-time water loss. Leaf-wetting events had a strong suppressive effect on tree transpiration (E). Foliar water uptake increased in magnitude with drier soil and during longer leaf-wetting events. The difference between diurnal and nocturnal stomatal behaviour in D. brasiliensis could be attributed to an optimization of carbon gain when leaves are dry, as well as minimization of nocturnal water loss. The leaf-wetting events on the other hand seem important to D. brasiliensis water balance, especially during soil droughts, both by suppressing tree transpiration (E) and as a small additional water supply through FWU. Our results suggest that decreases in leaf-wetting events in TMCF might increase D. brasiliensis water loss and decrease its water gains, which could compromise its ecophysiological performance and survival during dry periods.

Tree structure and richness in an Atlantic Forest fragment: distance from anthropogenic and natural edges

Approximately 7.2% of the Atlantic rainforest remains in Brazil, with only 16% of this forest remaining in the State of Rio de Janeiro, all of it distributed in fragments. This forest fragmentation can produce biotic and abiotic differences between edges and the fragment interior. In this study, we compared the structure and richness of tree communities in three habitats - an anthropogenic edge (AE), a natural edge (NE) and the fragment interior (FI) - of a fragment of Atlantic forest in the State of Rio de Janeiro, Brazil (22°50'S and 42°28'W). One thousand and seventy-six trees with a diameter at breast height > 4.8 cm, belonging to 132 morphospecies and 39 families, were sampled in a total study area of 0.75 ha. NE had the greatest basal area and the trees in this habitat had the greatest diameter:height allometric coefficient, whereas AE had a lower richness and greater variation in the height of the first tree branch. Tree density, diameter, height and the proportion of standing dead trees did not differ among the habitats. There was marked heterogeneity among replicates within each habitat. These results indicate that the forest interior and the fragment edges (natural or anthropogenic) do not differ markedly considering the studied parameters. Other factors, such as the age from the edge, type of matrix and proximity of gaps, may play a more important role in plant community structure than the proximity from edges.

Seed dispersal of Solanum thomasiifolium Sendtner (Solanaceae) in the Linhares Forest, Espírito Santo state, Brazil

The aim of this study was to analyse seed dispersal and establishment of Solanum thomasiifolium in an area of nativo vegetation in Espirito Santo state on the southeastern Brazilian coast. Ten species of birds, the crab-eating fox (Cerdocyon thous), and one species of lizard (Tropidurus torquatus) fed on S. thomasiifolium fruits and dispersed viable seeds in their faeces. The proportional contribution of each of these groups to seed dispersal was 77% (birds), 19% (crab-eating fox) and 4% (lizards). Ants also contributed to seed dispersal. More seeds were deposited in vegetation islands than in the surrounding open areas. Germination rates of seeds collected directly from fruit (control), bird droppings, the faeces of crab-eating foxes and lizards were, respectively, 64, 64, 53, and 80 %. Differences among these rates were all significant, except between birds and control. Lizards were important as seed carriers between nearby islands and they expelled a higher proportion of viable seeds. Birds and the crab-eating foxes did not enhance seed germination, but promoted seed dispersal over a wider area. Plant architecture, fruit productivity, fruit characteristics and the diversity of frugivores are important for the success of S. thomasiifolium in habitat colonization.