Methods for integrating machine learning and constrained optimization

In the framework of industrial problems, the application of Constrained Optimization is known to have overall very good modeling capability and performance and stands as one of the most powerful, explored, and exploited tool to address prescriptive tasks. The number of applications is huge, ranging from logistics to transportation, packing, production, telecommunication, scheduling, and much more. The main reason behind this success is to be found in the remarkable effort put in the last decades by the OR community to develop realistic models and devise exact or approximate methods to solve the largest variety of constrained or combinatorial optimization problems, together with the spread of computational power and easily accessible OR software and resources. On the other hand, the technological advancements lead to a data wealth never seen before and increasingly push towards methods able to extract useful knowledge from them; among the data-driven methods, Machine Learning techniques appear to be one of the most promising, thanks to its successes in domains like Image Recognition, Natural Language Processes and playing games, but also the amount of research involved. The purpose of the present research is to study how Machine Learning and Constrained Optimization can be used together to achieve systems able to leverage the strengths of both methods: this would open the way to exploiting decades of research on resolution techniques for COPs and constructing models able to adapt and learn from available data. In the first part of this work, we survey the existing techniques and classify them according to the type, method, or scope of the integration; subsequently, we introduce a novel and general algorithm devised to inject knowledge into learning models through constraints, Moving Target. In the last part of the thesis, two applications stemming from real-world projects and done in collaboration with Optit will be presented.

A machine learning based method to detect genomic imbalances exploiting X chromosome exome reads

Whole Exome Sequencing (WES) is rapidly becoming the first-tier test in clinics, both thanks to its declining costs and the development of new platforms that help clinicians in the analysis and interpretation of SNV and InDels. However, we still know very little on how CNV detection could increase WES diagnostic yield. A plethora of exome CNV callers have been published over the years, all showing good performances towards specific CNV classes and sizes, suggesting that the combination of multiple tools is needed to obtain an overall good detection performance. Here we present TrainX, a ML-based method for calling heterozygous CNVs in WES data using EXCAVATOR2 Normalized Read Counts. We select males and females’ non pseudo-autosomal chromosome X alignments to construct our dataset and train our model, make predictions on autosomes target regions and use HMM to call CNVs. We compared TrainX against a set of CNV tools differing for the detection method (GATK4 gCNV, ExomeDepth, DECoN, CNVkit and EXCAVATOR2) and found that our algorithm outperformed them in terms of stability, as we identified both deletions and duplications with good scores (0.87 and 0.82 F1-scores respectively) and for sizes reaching the minimum resolution of 2 target regions. We also evaluated the method robustness using a set of WES and SNP array data (n=251), part of the Italian cohort of Epi25 collaborative, and were able to retrieve all clinical CNVs previously identified by the SNP array. TrainX showed good accuracy in detecting heterozygous CNVs of different sizes, making it a promising tool to use in a diagnostic setting.

The three-dimensional single-bin-size bin packing problem: combining metaheuristic and machine learning approaches

The Three-Dimensional Single-Bin-Size Bin Packing Problem is one of the most studied problem in the Cutting & Packing category. From a strictly mathematical point of view, it consists of packing a finite set of strongly heterogeneous “small” boxes, called items, into a finite set of identical “large” rectangles, called bins, minimizing the unused volume and requiring that the items are packed without overlapping. The great interest is mainly due to the number of real-world applications in which it arises, such as pallet and container loading, cutting objects out of a piece of material and packaging design. Depending on these real-world applications, more objective functions and more practical constraints could be needed. After a brief discussion about the real-world applications of the problem and a exhaustive literature review, the design of a two-stage algorithm to solve the aforementioned problem is presented. The algorithm must be able to provide the spatial coordinates of the placed boxes vertices and also the optimal boxes input sequence, while guaranteeing geometric, stability, fragility constraints and a reduced computational time. Due to NP-hard complexity of this type of combinatorial problems, a fusion of metaheuristic and machine learning techniques is adopted. In particular, a hybrid genetic algorithm coupled with a feedforward neural network is used. In the first stage, a rich dataset is created starting from a set of real input instances provided by an industrial company and the feedforward neural network is trained on it. After its training, given a new input instance, the hybrid genetic algorithm is able to run using the neural network output as input parameter vector, providing as output the optimal solution. The effectiveness of the proposed works is confirmed via several experimental tests.

Machine learning as a service for high energy physics (MLaaS4HEP): a service for ML-based data analyses

With the CERN LHC program underway, there has been an acceleration of data growth in the High Energy Physics (HEP) field and the usage of Machine Learning (ML) in HEP will be critical during the HL-LHC program when the data that will be produced will reach the exascale. ML techniques have been successfully used in many areas of HEP nevertheless, the development of a ML project and its implementation for production use is a highly time-consuming task and requires specific skills. Complicating this scenario is the fact that HEP data is stored in ROOT data format, which is mostly unknown outside of the HEP community. The work presented in this thesis is focused on the development of a ML as a Service (MLaaS) solution for HEP, aiming to provide a cloud service that allows HEP users to run ML pipelines via HTTP calls. These pipelines are executed by using the MLaaS4HEP framework, which allows reading data, processing data, and training ML models directly using ROOT files of arbitrary size from local or distributed data sources. Such a solution provides HEP users non-expert in ML with a tool that allows them to apply ML techniques in their analyses in a streamlined manner. Over the years the MLaaS4HEP framework has been developed, validated, and tested and new features have been added. A first MLaaS solution has been developed by automatizing the deployment of a platform equipped with the MLaaS4HEP framework. Then, a service with APIs has been developed, so that a user after being authenticated and authorized can submit MLaaS4HEP workflows producing trained ML models ready for the inference phase. A working prototype of this service is currently running on a virtual machine of INFN-Cloud and is compliant to be added to the INFN Cloud portfolio of services.

Design and testing of innovative, artificial intelligence-based techniques to model and control the combustion process and to reduce the Co2 emissions in modern, high-performance, gasoline direct injection engines

This work deals with the development of calibration procedures and control systems to improve the performance and efficiency of modern spark ignition turbocharged engines. The algorithms developed are used to optimize and manage the spark advance and the air-to-fuel ratio to control the knock and the exhaust gas temperature at the turbine inlet. The described work falls within the activity that the research group started in the previous years with the industrial partner Ferrari S.p.a. . The first chapter deals with the development of a control-oriented engine simulator based on a neural network approach, with which the main combustion indexes can be simulated. The second chapter deals with the development of a procedure to calibrate offline the spark advance and the air-to-fuel ratio to run the engine under knock-limited conditions and with the maximum admissible exhaust gas temperature at the turbine inlet. This procedure is then converted into a model-based control system and validated with a Software in the Loop approach using the engine simulator developed in the first chapter. Finally, it is implemented in a rapid control prototyping hardware to manage the combustion in steady-state and transient operating conditions at the test bench. The third chapter deals with the study of an innovative and cheap sensor for the in-cylinder pressure measurement, which is a piezoelectric washer that can be installed between the spark plug and the engine head. The signal generated by this kind of sensor is studied, developing a specific algorithm to adjust the value of the knock index in real-time. Finally, with the engine simulator developed in the first chapter, it is demonstrated that the innovative sensor can be coupled with the control system described in the second chapter and that the performance obtained could be the same reachable with the standard in-cylinder pressure sensors.

Development of machine learning methods for multi-modal biomarkers detection and integration

In medicine, innovation depends on a better knowledge of the human body mechanism, which represents a complex system of multi-scale constituents. Unraveling the complexity underneath diseases proves to be challenging. A deep understanding of the inner workings comes with dealing with many heterogeneous information. Exploring the molecular status and the organization of genes, proteins, metabolites provides insights on what is driving a disease, from aggressiveness to curability. Molecular constituents, however, are only the building blocks of the human body and cannot currently tell the whole story of diseases. This is why nowadays attention is growing towards the contemporary exploitation of multi-scale information. Holistic methods are then drawing interest to address the problem of integrating heterogeneous data. The heterogeneity may derive from the diversity across data types and from the diversity within diseases. Here, four studies conducted data integration using customly designed workflows that implement novel methods and views to tackle the heterogeneous characterization of diseases. The first study devoted to determine shared gene regulatory signatures for onco-hematology and it showed partial co-regulation across blood-related diseases. The second study focused on Acute Myeloid Leukemia and refined the unsupervised integration of genomic alterations, which turned out to better resemble clinical practice. In the third study, network integration for artherosclerosis demonstrated, as a proof of concept, the impact of network intelligibility when it comes to model heterogeneous data, which showed to accelerate the identification of new potential pharmaceutical targets. Lastly, the fourth study introduced a new method to integrate multiple data types in a unique latent heterogeneous-representation that facilitated the selection of important data types to predict the tumour stage of invasive ductal carcinoma. The results of these four studies laid the groundwork to ease the detection of new biomarkers ultimately beneficial to medical practice and to the ever-growing field of Personalized Medicine.

Network analysis and machine learning assist drug repurposing and safety assessment in neurological diseases

In recent decades, two prominent trends have influenced the data modeling field, namely network analysis and machine learning. This thesis explores the practical applications of these techniques within the domain of drug research, unveiling their multifaceted potential for advancing our comprehension of complex biological systems. The research undertaken during this PhD program is situated at the intersection of network theory, computational methods, and drug research. Across six projects presented herein, there is a gradual increase in model complexity. These projects traverse a diverse range of topics, with a specific emphasis on drug repurposing and safety in the context of neurological diseases. The aim of these projects is to leverage existing biomedical knowledge to develop innovative approaches that bolster drug research. The investigations have produced practical solutions, not only providing insights into the intricacies of biological systems, but also allowing the creation of valuable tools for their analysis. In short, the achievements are: • A novel computational algorithm to identify adverse events specific to fixed-dose drug combinations. • A web application that tracks the clinical drug research response to SARS-CoV-2. • A Python package for differential gene expression analysis and the identification of key regulatory "switch genes". • The identification of pivotal events causing drug-induced impulse control disorders linked to specific medications. • An automated pipeline for discovering potential drug repurposing opportunities. • The creation of a comprehensive knowledge graph and development of a graph machine learning model for predictions. Collectively, these projects illustrate diverse applications of data science and network-based methodologies, highlighting the profound impact they can have in supporting drug research activities.

Developing a radiomic based machine learning model to identify patients with resected non-small-cell lung cancer at high risk of relapse

Background There is a wide variation of recurrence risk of Non-small-cell lung cancer (NSCLC) within the same Tumor Node Metastasis (TNM) stage, suggesting that other parameters are involved in determining this probability. Radiomics allows extraction of quantitative information from images that can be used for clinical purposes. The primary objective of this study is to develop a radiomic prognostic model that predicts a 3 year disease free-survival (DFS) of resected Early Stage (ES) NSCLC patients. Material and Methods 56 pre-surgery non contrast Computed Tomography (CT) scans were retrieved from the PACS of our institution and anonymized. Then they were automatically segmented with an open access deep learning pipeline and reviewed by an experienced radiologist to obtain 3D masks of the NSCLC. Images and masks underwent to resampling normalization and discretization. From the masks hundreds Radiomic Features (RF) were extracted using Py-Radiomics. Hence, RF were reduced to select the most representative features. The remaining RF were used in combination with Clinical parameters to build a DFS prediction model using Leave-one-out cross-validation (LOOCV) with Random Forest. Results and Conclusion A poor agreement between the radiologist and the automatic segmentation algorithm (DICE score of 0.37) was found. Therefore, another experienced radiologist manually segmented the lesions and only stable and reproducible RF were kept. 50 RF demonstrated a high correlation with the DFS but only one was confirmed when clinicopathological covariates were added: Busyness a Neighbouring Gray Tone Difference Matrix (HR 9.610). 16 clinical variables (which comprised TNM) were used to build the LOOCV model demonstrating a higher Area Under the Curve (AUC) when RF were included in the analysis (0.67 vs 0.60) but the difference was not statistically significant (p=0,5147).

Sviluppo di un’app mobile con Realtà Aumentata e Machine Learning in contesto di biodiversità

Il volume di tesi ha riguardato lo sviluppo di un'applicazione mobile che sfrutta la Realtà Aumentata e il Machine Learning nel contesto della biodiversità. Nello specifico si è realizzato un modello di AI che permetta la classificazione di immagini di fiori. Tale modello è stato poi integrato in Android, al fine della realizzazione di un'app che riesca a riconoscere specifiche specie di fiori, oltre a individuare gli insetti impollinatori attratti da essi e rappresentarli in Realtà Aumentata.

Machine learning techniques for heavy-flavour baryon production measurements at the LHC

Il quark-gluon plasma (QGP) è uno stato della materia previsto dalla cromodinamica quantistica. L’esperimento ALICE a LHC ha tra i suoi obbiettivi principali lo studio della materia fortemente interagente e le proprietà del QGP attraverso collisioni di ioni pesanti ultra-relativistici. Per un’esaustiva comprensione di tali proprietà, le stesse misure effettuate su sistemi collidenti più piccoli (collisioni protone-protone e protone-ione) sono necessarie come riferimento. Le recenti analisi dei dati raccolti ad ALICE hanno mostrato che la nostra comprensione dei meccanismi di adronizzazione di quark pesanti non è completa, perchè i dati ottenuti in collisioni pp e p-Pb non sono riproducibili utilizzando modelli basati sui risultati ottenuti con collisioni e+e− ed ep. Per questo motivo, nuovi modelli teorici e fenomenologici, in grado di riprodurre le misure sperimentali, sono stati proposti. Gli errori associati a queste nuove misure sperimentali al momento non permettono di verificare in maniera chiara la veridicità dei diversi modelli proposti. Nei prossimi anni sarà quindi fondamentale aumentare la precisione di tali misure sperimentali; d’altra parte, stimare il numero delle diverse specie di particelle prodotte in una collisione può essere estremamente complicato. In questa tesi, il numero di barioni Lc prodotti in un campione di dati è stato ottenuto utilizzando delle tecniche di machine learning, in grado di apprendere pattern e imparare a distinguere candidate di segnale da quelle di fondo. Si sono inoltre confrontate tre diverse implementazioni di un algoritmo di Boosted Decision Trees (BDT) e si è utilizzata quella più performante per ricostruire il barione Lc in collisioni pp raccolte dall’esperimento ALICE.

Machine learning per la previsione sui prezzi delle case

Negli ultimi anni, a causa della crescente tendenza verso i Big Data, l’apprendimento automatico è diventato un approccio di previsione fondamentale perché può prevedere i prezzi delle case in modo accurato in base agli attributi delle abitazioni. In questo elaborato, verranno messe in pratica alcune tecniche di machine learning con l’obiettivo di effettuare previsioni sui prezzi delle abitazioni. Ad esempio, si può pensare all’acquisto di una nuova casa, saranno tanti i fattori di cui si dovrà preoccuparsi, la posizione, i metri quadrati, l’inquinamento dell’aria, il numero di stanze, il numero dei bagni e così via. Tutti questi fattori possono influire in modo più o meno pesante sul prezzo di quell’abitazione. E’ proprio in casi come questi che può essere applicata l’intelligenza artificiale, nello specifico il machine learning, per riuscire a trovare un modello che approssimi nel miglior modo un prezzo, data una serie di caratteristiche. In questa tesi verrà dimostrato come è possibile utilizzare l’apprendimento automatico per effettuare delle stime il più preciso possibile dei prezzi delle case. La tesi è divisa in 5 capitoli, nel primo capitolo verranno introdotti i concetti di base su cui si basa l’elaborato e alcune spiegazioni dei singoli modelli. Nel secondo capitolo, invece, viene trattato l’ambiente di lavoro utilizzato, il linguaggio e le relative librerie utilizzate. Il terzo capitolo contiene un’analisi esplorativa sul dataset utilizzato e vengono effettuate delle operazioni per preparare i dati agli algoritmi che verranno applicati in seguito. Nel capitolo 4 vengono creati i diversi modelli ed effettuate le previsioni sui prezzi mentre nel capitolo 5 vengono analizzati i risultati ottenuti e riportate le conclusioni.

Machine Learning in ambito assicurativo Un caso di studio

Il Machine Learning si sta rivelando una tecnologia dalle incredibili potenzialità nei settori più disparati. Le diverse tecniche e gli algoritmi che vi fanno capo abilitano analisi dei dati molto più efficaci rispetto al passato. Anche l’industria assicurativa sta sperimentando l’adozione di soluzioni di Machine Learning e diverse sono le direzioni di innovamento che ne stanno conseguendo, dall’efficientamento dei processi interni all’offerta di prodotti rispondenti in maniera adattiva alle esigenze del cliente. Questo lavoro di tesi è stato realizzato durante un tirocinio presso Unisalute S.p.A., la prima assicurazione in ambito sanitario in Italia. La criticità intercettata è stata la sovrastima del capitale da destinare a riserva a fronte dell’impegno nei confronti dell’assicurato: questo capitale immobilizzato va a sottrarre risorse ad investimenti più proficui nel medio e lungo termine, per cui è di valore stimarlo appropriatamente. All'interno del settore IT di Unisalute, ho lavorato alla progettazione e implementazione di un modello di Machine Learning che riesca a prevedere se un sinistro appena preso in gestione sarà liquidato o meno. Dotare gli uffici impegnati nella determinazione del riservato di questa stima aggiuntiva basata sui dati, sarebbe di notevole supporto. La progettazione del modello di Machine Learning si è articolata in una Data Pipeline contenente le metodologie più efficienti con riferimento al preprocessamento e alla modellazione dei dati. L’implementazione ha visto Python come linguaggio di programmazione; il dataset, ottenuto a seguito di estrazioni e integrazioni a partire da diversi database Oracle, presenta una cardinalità di oltre 4 milioni di istanze caratterizzate da 32 variabili. A valle del tuning degli iperparamentri e dei vari addestramenti, si è raggiunta un’accuratezza dell’86% che, nel dominio di specie, è ritenuta più che soddisfacente e sono emersi contributi non noti alla liquidabilità dei sinistri.

Texting and Driving: rilevazione di utilizzo dello smartphone alla guida tramite Object Detection e Machine Learning

Questa tesi si ispira a lavori precedentemente portati avanti da altri studenti e si pone il problema della possibilit\`a di riconoscere se uno smartphone \`e utilizzato da un utente mentre esso si trova alla guida di un'autovettura. In essa verranno presentati vari metodi per risolvere questo problema di Machine Learning, ovvero realizzazione di dataset per l'allenamento di modelli e creazione e allenamento di modelli stessi, dediti al riconoscimento di un problema di classificazione binaria e riconoscimento di oggetti tramite Object Detection. Il cercare di riconoscere se l'utente \`e alla guida o meno, avverr\`a tramite l'output della fotocamera frontale dello smartphone, quindi lavoreremo su immagini, video e frame. Arriveremo a riconoscere la posizione della persona rappresentata da questi fotogrammi tramite un modello di Object Detection, che riconosce cintura e finestrino e determina se sono appartenenti al sedile e alla posizione del conducente o del passeggero. Vedremo alla fine, attraverso un'attenta analisi dei risultati ottenuti su ben 8 video diversi che saranno divisi in molti frame, che si ottengono risultati molto interessanti, dai quali si pu\`o prendere spunto per la creazione di un importante sistema di sicurezza alla guida.

Machine learning approach to the extended Hubbard model

The 1d extended Hubbard model with soft-shoulder potential has proved itself to be very difficult to study due its non solvability and to competition between terms of the Hamiltonian. Given this, we tried to investigate its phase diagram for filling n=2/5 and range of soft-shoulder potential r=2 by using Machine Learning techniques. That led to a rich phase diagram; calling U, V the parameters associated to the Hubbard potential and the soft-shoulder potential respectively, we found that for V<5 and U>3 the system is always in Tomonaga Luttinger Liquid phase, then becomes a Cluster Luttinger Liquid for 57, with a quasi-perfect crystal in the U<3V/2 and U>5 region. Finally we found that for U<5 and V>2-3 the system shall maintain the Cluster Luttinger Liquid structure, with a residual in-block single particle mobility.

Synthetic data generation for the assessment of antimicrobial resistance through machine learning

As a consequence of the diffusion of next generation sequencing techniques, metagenomics databases have become one of the most promising repositories of information about features and behavior of microorganisms. One of the subjects that can be studied from those data are bacteria populations. Next generation sequencing techniques allow to study the bacteria population within an environment by sampling genetic material directly from it, without the needing of culturing a similar population in vitro and observing its behavior. As a drawback, it is quite complex to extract information from those data and usually there is more than one way to do that; AMR is no exception. In this study we will discuss how the quantified AMR, which regards the genotype of the bacteria, can be related to the bacteria phenotype and its actual level of resistance against the specific substance. In order to have a quantitative information about bacteria genotype, we will evaluate the resistome from the read libraries, aligning them against CARD database. With those data, we will test various machine learning algorithms for predicting the bacteria phenotype. The samples that we exploit should resemble those that could be obtained from a natural context, but are actually produced by a read libraries simulation tool. In this way we are able to design the populations with bacteria of known genotype, so that we can relay on a secure ground truth for training and testing our algorithms.