Machine learning and convolutional neural networks for melanomas identification

Il machine learning negli ultimi anni ha acquisito una crescente popolarità nell’ambito della ricerca scientifica e delle sue applicazioni. Lo scopo di questa tesi è stato quello di studiare il machine learning nei suoi aspetti generali e applicarlo a problemi di computer vision. La tesi ha affrontato le difficoltà del dover spiegare dal punto di vista teorico gli algoritmi alla base delle reti neurali convoluzionali e ha successivamente trattato due problemi concreti di riconoscimento immagini: il dataset MNIST (immagini di cifre scritte a mano) e un dataset che sarà chiamato ”MELANOMA dataset” (immagini di melanomi e nevi sani). Utilizzando le tecniche spiegate nella sezione teorica si sono riusciti ad ottenere risultati soddifacenti per entrambi i dataset ottenendo una precisione del 98% per il MNIST e del 76.8% per il MELANOMA dataset

Methods for monitoring Machine Learning models

The following thesis aims to investigate the issues concerning the maintenance of a Machine Learning model over time, both about the versioning of the model itself and the data on which it is trained and about data monitoring tools and their distribution. The themes of Data Drift and Concept Drift were then explored and the performance of some of the most popular techniques in the field of Anomaly detection, such as VAE, PCA, and Monte Carlo Dropout, were evaluated.

ICE Emissions Estimation through Machine-learning techniques to support real-world R&D experiments

The emissions estimation, both during homologation and standard driving, is one of the new challenges that automotive industries have to face. The new European and American regulation will allow a lower and lower quantity of Carbon Monoxide emission and will require that all the vehicles have to be able to monitor their own pollutants production. Since numerical models are too computationally expensive and approximated, new solutions based on Machine Learning are replacing standard techniques. In this project we considered a real V12 Internal Combustion Engine to propose a novel approach pushing Random Forests to generate meaningful prediction also in extreme cases (extrapolation, very high frequency peaks, noisy instrumentation etc.). The present work proposes also a data preprocessing pipeline for strongly unbalanced datasets and a reinterpretation of the regression problem as a classification problem in a logarithmic quantized domain. Results have been evaluated for two different models representing a pure interpolation scenario (more standard) and an extrapolation scenario, to test the out of bounds robustness of the model. The employed metrics take into account different aspects which can affect the homologation procedure, so the final analysis will focus on combining all the specific performances together to obtain the overall conclusions.

Machine Learning in Automatic Tabulation for Constraint Model Reformulation

Combinatorial decision and optimization problems belong to numerous applications, such as logistics and scheduling, and can be solved with various approaches. Boolean Satisfiability and Constraint Programming solvers are some of the most used ones and their performance is significantly influenced by the model chosen to represent a given problem. This has led to the study of model reformulation methods, one of which is tabulation, that consists in rewriting the expression of a constraint in terms of a table constraint. To apply it, one should identify which constraints can help and which can hinder the solving process. So far this has been performed by hand, for example in MiniZinc, or automatically with manually designed heuristics, in Savile Row. Though, it has been shown that the performances of these heuristics differ across problems and solvers, in some cases helping and in others hindering the solving procedure. However, recent works in the field of combinatorial optimization have shown that Machine Learning (ML) can be increasingly useful in the model reformulation steps. This thesis aims to design a ML approach to identify the instances for which Savile Row’s heuristics should be activated. Additionally, it is possible that the heuristics miss some good tabulation opportunities, so we perform an exploratory analysis for the creation of a ML classifier able to predict whether or not a constraint should be tabulated. The results reached towards the first goal show that a random forest classifier leads to an increase in the performances of 4 different solvers. The experimental results in the second task show that a ML approach could improve the performance of a solver for some problem classes.

Implementazione di algoritmi Tiny Machine Learning per il monitoraggio strutturale basato su emissioni acustiche

Il monitoraggio basato su emissioni acustiche (AE) guidate si è confermato tra le tecniche più affidabili nel campo del Non-Destructive Testing delle strutture planari, vista anche la sua semplicità implementativa, i bassi costi che lo caratterizzano, la non invasività e la possibilità di realizzare un sistema che agisca in maniera continuativa ed in tempo reale sfruttando reti di sensori permanentemente installati, senza la necessità di ispezioni periodiche. In tale contesto, è possibile sfruttare l’abilità dell’apprendimento automatico nell’individuazione dei pattern nascosti all’interno dei segnali grezzi registrati, ottenendo così informazioni utili ai fini dell’applicazione considerata. L’esecuzione on-edge dei modelli, ovvero sul punto di acquisizione, consente di superare le limitazioni imposte dal processamento centralizzato dei dati, con notevoli vantaggi in termini di consumo energetico, tempestività nella risposta ed integrità degli stessi. A questo scopo, si rivela però necessario sviluppare modelli compatibili con le stringenti risorse hardware dei dispositivi a basso costo tipicamente impiegati. In questo elaborato verranno prese in esame alcune tipologie di reti neurali artificiali per l’estrazione dell’istante di arrivo (ToA) di un’emissione acustica all’interno di una sequenza temporale, in particolare quelle convoluzionali (CNNs) ed una loro variante più recente, le CapsNet basate su rounting by agreement. L’individuazione dei ToA relativi al medesimo evento su segnali acquisiti in diverse posizioni spaziali consente infatti di localizzare la sorgente da cui esso è scaturito. Le dimensioni di questi modelli permettono di eseguire l’inferenza direttamente su edge-device. I risultati ottenuti confermano la maggiore robustezza delle tecniche di apprendimento profondo rispetto ai metodi statistici tradizionali nel far fronte a diverse tipologie di disturbo, in particolare negli scenari più critici dal punto di vista del rapporto segnale-rumore.

Machine Learning algorithm on seat comfort level measurement, mapping pressure values with body regions

The comfort level of the seat has a major effect on the usage of a vehicle; thus, car manufacturers have been working on elevating car seat comfort as much as possible. However, still, the testing and evaluation of comfort are done using exhaustive trial and error testing and evaluation of data. In this thesis, we resort to machine learning and Artificial Neural Networks (ANN) to develop a fully automated approach. Even though this approach has its advantages in minimizing time and using a large set of data, it takes away the degree of freedom of the engineer on making decisions. The focus of this study is on filling the gap in a two-step comfort level evaluation which used pressure mapping with body regions to evaluate the average pressure supported by specific body parts and the Self-Assessment Exam (SAE) questions on evaluation of the person’s interest. This study has created a machine learning algorithm that works on giving a degree of freedom to the engineer in making a decision when mapping pressure values with body regions using ANN. The mapping is done with 92% accuracy and with the help of a Graphical User Interface (GUI) that facilitates the process during the testing time of comfort level evaluation of the car seat, which decreases the duration of the test analysis from days to hours.

Sviluppo di un sistema di climatizzazione automatico distribuito con l'utilizzo di tecniche di machine learning

Nella sede dell’azienda ospitante Alexide, si è ravvisata la mancanza di un sistema di controllo automatico da remoto dell’intero impianto di climatizzazione HVAC (Heating, Ventilation and Air Conditioning) utilizzato, e la soluzione migliore è risultata quella di attuare un processo di trasformazione della struttura in uno smart building. Ho quindi eseguito questa procedura di trasformazione digitale progettando e sviluppando un sistema distribuito in grado di gestire una serie di dati provenienti in tempo reale da sensori ambientali. L’architettura del sistema progettato è stata sviluppata in C# su ambiente dotNET, dove sono stati collezionati i dati necessari per il funzionamento del modello di predizione. Nella fattispecie sono stati utilizzati i dati provenienti dall’HVAC, da un sensore di temperatura interna dell'edificio e dal fotovoltaico installato nella struttura. La comunicazione tra il sistema distribuito e l’entità dell’HVAC avviene mediante il canale di comunicazione ModBus, mentre per quanto riguarda i dati della temperatura interna e del fotovoltaico questi vengono collezionati da sensori che inviano le informazioni sfruttando un canale di comunicazione che utilizza il protocollo MQTT, e lo stesso viene utilizzato come principale metodo di comunicazione all’interno del sistema, appoggiandosi ad un broker di messaggistica con modello publish/subscribe. L'automatizzazione del sistema è dovuta anche all'utilizzo di un modello di predizione con lo scopo di predire in maniera quanto più accurata possibile la temperatura interna all'edificio delle ore future. Per quanto riguarda il modello di predizione da me implementato e integrato nel sistema la scelta è stata quella di ispirarmi ad un modello ideato da Google nel 2014 ovvero il Sequence to Sequence. Il modello sviluppato si struttura come un encoder-decoder che utilizza le RNN, in particolare le reti LSTM.

Aberrant lipid metabolism in hepatocellular carcinoma revealed by plasma metabolomics and lipid profiling

There has been limited analysis of the effects of hepatocellular carcinoma (HCC) on liver metabolism and circulating endogenous metabolites. Here, we report the findings of a plasma metabolomic investigation of HCC patients by ultraperformance liquid chromatography-electrospray ionization-quadrupole time-of-flight mass spectrometry (UPLC-ESI-QTOFMS), random forests machine learning algorithm, and multivariate data analysis. Control subjects included healthy individuals as well as patients with liver cirrhosis or acute myeloid leukemia. We found that HCC was associated with increased plasma levels of glycodeoxycholate, deoxycholate 3-sulfate, and bilirubin. Accurate mass measurement also indicated upregulation of biliverdin and the fetal bile acids 7α-hydroxy-3-oxochol-4-en-24-oic acid and 3-oxochol-4,6-dien-24-oic acid in HCC patients. A quantitative lipid profiling of patient plasma was also conducted by ultraperformance liquid chromatography-electrospray ionization-triple quadrupole mass spectrometry (UPLC-ESI-TQMS). By this method, we found that HCC was also associated with reduced levels of lysophosphocholines and in 4 of 20 patients with increased levels of lysophosphatidic acid [LPA(16:0)], where it correlated with plasma α-fetoprotein levels. Interestingly, when fatty acids were quantitatively profiled by gas chromatography-mass spectrometry (GC-MS), we found that lignoceric acid (24:0) and nervonic acid (24:1) were virtually absent from HCC plasma. Overall, this investigation illustrates the power of the new discovery technologies represented in the UPLC-ESI-QTOFMS platform combined with the targeted, quantitative platforms of UPLC-ESI-TQMS and GC-MS for conducting metabolomic investigations that can engender new insights into cancer pathobiology.

Treatment of non-resectable hepatocellular carcinoma with autologous tumor-pulsed dendritic cells

Background: The response of hepatocellular carcinoma (HCC) to therapy is often disappointing and new modalities of treatment are clearly needed. Active immunotherapy based on the injection of autologous dendritic cells (DC) co-cultured ex vivo with tumor antigens has been used in pilot studies in various malignancies such as melanoma and lymphoma with encouraging results. Methods: In the present paper, the preparation and exposure of patient DC to autologous HCC antigens and re-injection in an attempt to elicit antitumor immune responses are described. Results: Therapy was given to two patients, one with hepatitis C and one with hepatitis B, who had large, multiple HCC and for whom no other therapy was available. No significant side-effects were observed. The clinical course was unchanged in one patient, who died a few months later. The other patient, whose initial prognosis was considered poor, is still alive and well more than 3 years later with evidence of slowing of tumor growth based on organ imaging. Conclusions: It is concluded that HCC may be a malignancy worthy of DC trials and sufficient details in the present paper are given for the protocol to be copied or modified. (C) 2002 Blackwell Publishing Asia Pty Ltd.

The impact of neural invasion severity in gastrointestinal malignancies: a clinicopathological study.

OBJECTIVES Because neural invasion (NI) is still inconsistently reported and not well characterized within gastrointestinal malignancies (GIMs), our aim was to determine the exact prevalence and severity of NI and to elucidate the true impact of NI on patient's prognosis. BACKGROUND The union internationale contre le cancer (UICC) recently added NI as a novel parameter in the current TNM classification. However, there are only a few existing studies with specific focus on NI, so that the distinct role of NI in GIMs is still uncertain. MATERIALS AND METHODS NI was characterized in approximately 16,000 hematoxylin and eosin tissue sections from 2050 patients with adenocarcinoma of the esophagogastric junction (AEG)-I-III, squamous cell carcinoma (SCC) of the esophagus, gastric cancer (GC), colon cancer (CC), rectal cancer (RC), cholangiocellular cancer (CCC), hepatocellular cancer (HCC), and pancreatic cancer (PC). NI prevalence and severity was determined and related to patient's prognosis and survival. RESULTS NI prevalence largely varied between HCC/6%, CC/28%, RC/34%, AEG-I/36% and AEG-II/36%, SCC/37%, GC/38%, CCC/58%, and AEG-III/65% to PC/100%. NI severity score was uppermost in PC (24.9±1.9) and lowest in AEG-I (0.8±0.3). Multivariable analyses including age, sex, TNM stage, and grading revealed that the prevalence of NI was significantly associated with diminished survival in AEG-II/III, GC, and RC. However, increasing NI severity impaired survival in AEG-II/III and PC only. CONCLUSIONS NI prevalence and NI severity strongly vary within GIMs. Determination of NI severity in GIMs is a more precise tool than solely recording the presence of NI and revealed dismal prognostic impact on patients with AEG-II/III and PC. Evidently, NI is not a concomitant side feature in GIMs and, therefore, deserves special attention for improved patient stratification and individualized therapy after surgery.

Information extraction from clinical and radiology notes for liver cancer staging

Thesis (Ph.D.)--University of Washington, 2016-06

Extreme learning machine for predicting HLA-peptide binding

Machine learning techniques have been recognized as powerful tools for learning from data. One of the most popular learning techniques, the Back-Propagation (BP) Artificial Neural Networks, can be used as a computer model to predict peptides binding to the Human Leukocyte Antigens (HLA). The major advantage of computational screening is that it reduces the number of wet-lab experiments that need to be performed, significantly reducing the cost and time. A recently developed method, Extreme Learning Machine (ELM), which has superior properties over BP has been investigated to accomplish such tasks. In our work, we found that the ELM is as good as, if not better than, the BP in term of time complexity, accuracy deviations across experiments, and most importantly - prevention from over-fitting for prediction of peptide binding to HLA.

Supporting Scientific Research Through Machine and Deep Learning: Fluorescence Microscopy and Operational Intelligence Use Cases

Although the debate of what data science is has a long history and has not reached a complete consensus yet, Data Science can be summarized as the process of learning from data. Guided by the above vision, this thesis presents two independent data science projects developed in the scope of multidisciplinary applied research. The first part analyzes fluorescence microscopy images typically produced in life science experiments, where the objective is to count how many marked neuronal cells are present in each image. Aiming to automate the task for supporting research in the area, we propose a neural network architecture tuned specifically for this use case, cell ResUnet (c-ResUnet), and discuss the impact of alternative training strategies in overcoming particular challenges of our data. The approach provides good results in terms of both detection and counting, showing performance comparable to the interpretation of human operators. As a meaningful addition, we release the pre-trained model and the Fluorescent Neuronal Cells dataset collecting pixel-level annotations of where neuronal cells are located. In this way, we hope to help future research in the area and foster innovative methodologies for tackling similar problems. The second part deals with the problem of distributed data management in the context of LHC experiments, with a focus on supporting ATLAS operations concerning data transfer failures. In particular, we analyze error messages produced by failed transfers and propose a Machine Learning pipeline that leverages the word2vec language model and K-means clustering. This provides groups of similar errors that are presented to human operators as suggestions of potential issues to investigate. The approach is demonstrated on one full day of data, showing promising ability in understanding the message content and providing meaningful groupings, in line with previously reported incidents by human operators.

Does hepatocellular carcinoma in non-alcoholic steatohepatitis exist in cirrhotic and non-cirrhotic patients?

Non-alcoholic steatohepatitis (NASH) has been associated with hepatocellular carcinoma (HCC) often arising in histologically advanced disease when steatohepatitis is not active (cryptogenic cirrhosis). Our objective was to characterize patients with HCC and active, histologically defined steatohepatitis. Among 394 patients with HCC detected by ultrasound imaging over 8 years and staged by the Barcelona Clinic Liver Cancer (BCLC) criteria, we identified 7 cases (1.7%) with HCC occurring in the setting of active biopsy-proven NASH. All were negative for other liver diseases such as hepatitis C, hepatitis B, autoimmune hepatitis, Wilson disease, and hemochromatosis. The patients (4 males and 3 females, age 63 ± 13 years) were either overweight (4) or obese (3); 57% were diabetic and 28.5% had dyslipidemia. Cirrhosis was present in 6 of 7 patients, but 1 patient had well-differentiated HCC in the setting of NASH without cirrhosis (fibrosis stage 1) based on repeated liver biopsies, the absence of portal hypertension by clinical and radiographic evaluations and by direct surgical inspection. Among the cirrhotic patients, 71.4% were clinically staged as Child A and 14.2% as Child B. Tumor size ranged from 1.0 to 5.2 cm and 5 of 7 patients were classified as early stage; 46% of all nodules were hyper-echoic and 57% were <3 cm. HCC was well differentiated in 1/6 and moderately differentiated in 5/6. Alpha-fetoprotein was <100 ng/mL in all patients. HCC in patients with active steatohepatitis is often multifocal, may precede clinically advanced disease and occurs without diagnostic levels of alpha-fetoprotein. Importantly, HCC may occur in NASH in the absence of cirrhosis. More aggressive screening of NASH patients may be warranted.

249 TP53 mutation has high prevalence and is correlated with larger and poorly differentiated HCC in Brazilian patients

Background: Ser-249 TP53 mutation (249(Ser)) is a molecular evidence for aflatoxin-related carcinogenesis in Hepatocellular Carcinoma (HCC) and it is frequent in some African and Asian regions, but it is unusual in Western countries. HBV has been claimed to add a synergic effect on genesis of this particular mutation with aflatoxin. The aim of this study was to investigate the frequency of 249Ser mutation in HCC from patients in Brazil. Methods: We studied 74 HCC formalin fixed paraffin blocks samples of patients whom underwent surgical resection in Brazil. 249Ser mutation was analyzed by RFLP and DNA sequencing. HBV DNA presence was determined by Real-Time PCR. Results: 249Ser mutation was found in 21/74 (28%) samples while HBV DNA was detected in 13/74 (16%). 249Ser mutation was detected in 21/74 samples by RFLP assay, of which 14 were confirmed by 249Ser mutant-specific PCR, and 12 by nucleic acid sequencing. All HCC cases with p53-249ser mutation displayed also wild-type p53 sequences. Poorly differentiated HCC was more likely to have 249Ser mutation (OR = 2.415, 95% CI = 1.001 - 5.824, p = 0.05). The mean size of 249Ser HCC tumor was 9.4 cm versus 5.5 cm on wild type HCC (p = 0.012). HBV DNA detection was not related to 249Ser mutation. Conclusion: Our results indicate that 249Ser mutation is a HCC important factor of carcinogenesis in Brazil and it is associated to large and poorly differentiated tumors.