Methods for integrating machine learning and constrained optimization

In the framework of industrial problems, the application of Constrained Optimization is known to have overall very good modeling capability and performance and stands as one of the most powerful, explored, and exploited tool to address prescriptive tasks. The number of applications is huge, ranging from logistics to transportation, packing, production, telecommunication, scheduling, and much more. The main reason behind this success is to be found in the remarkable effort put in the last decades by the OR community to develop realistic models and devise exact or approximate methods to solve the largest variety of constrained or combinatorial optimization problems, together with the spread of computational power and easily accessible OR software and resources. On the other hand, the technological advancements lead to a data wealth never seen before and increasingly push towards methods able to extract useful knowledge from them; among the data-driven methods, Machine Learning techniques appear to be one of the most promising, thanks to its successes in domains like Image Recognition, Natural Language Processes and playing games, but also the amount of research involved. The purpose of the present research is to study how Machine Learning and Constrained Optimization can be used together to achieve systems able to leverage the strengths of both methods: this would open the way to exploiting decades of research on resolution techniques for COPs and constructing models able to adapt and learn from available data. In the first part of this work, we survey the existing techniques and classify them according to the type, method, or scope of the integration; subsequently, we introduce a novel and general algorithm devised to inject knowledge into learning models through constraints, Moving Target. In the last part of the thesis, two applications stemming from real-world projects and done in collaboration with Optit will be presented.

Applied deep learning and data science with a human-centric and data-centric approach

The term Artificial intelligence acquired a lot of baggage since its introduction and in its current incarnation is synonymous with Deep Learning. The sudden availability of data and computing resources has opened the gates to myriads of applications. Not all are created equal though, and problems might arise especially for fields not closely related to the tasks that pertain tech companies that spearheaded DL. The perspective of practitioners seems to be changing, however. Human-Centric AI emerged in the last few years as a new way of thinking DL and AI applications from the ground up, with a special attention at their relationship with humans. The goal is designing a system that can gracefully integrate in already established workflows, as in many real-world scenarios AI may not be good enough to completely replace its humans. Often this replacement may even be unneeded or undesirable. Another important perspective comes from, Andrew Ng, a DL pioneer, who recently started shifting the focus of development from “better models” towards better, and smaller, data. He defined his approach Data-Centric AI. Without downplaying the importance of pushing the state of the art in DL, we must recognize that if the goal is creating a tool for humans to use, more raw performance may not align with more utility for the final user. A Human-Centric approach is compatible with a Data-Centric one, and we find that the two overlap nicely when human expertise is used as the driving force behind data quality. This thesis documents a series of case-studies where these approaches were employed, to different extents, to guide the design and implementation of intelligent systems. We found human expertise proved crucial in improving datasets and models. The last chapter includes a slight deviation, with studies on the pandemic, still preserving the human and data centric perspective.

An algorithmic Friedman-Pippenger theorem on tree embeddings and applications

An (n, d)-expander is a graph G = (V, E) such that for every X subset of V with vertical bar X vertical bar <= 2n - 2 we have vertical bar Gamma(G)(X) vertical bar >= (d + 1) vertical bar X vertical bar. A tree T is small if it has at most n vertices and has maximum degree at most d. Friedman and Pippenger (1987) proved that any ( n; d)- expander contains every small tree. However, their elegant proof does not seem to yield an efficient algorithm for obtaining the tree. In this paper, we give an alternative result that does admit a polynomial time algorithm for finding the immersion of any small tree in subgraphs G of (N, D, lambda)-graphs Lambda, as long as G contains a positive fraction of the edges of Lambda and lambda/D is small enough. In several applications of the Friedman-Pippenger theorem, including the ones in the original paper of those authors, the (n, d)-expander G is a subgraph of an (N, D, lambda)-graph as above. Therefore, our result suffices to provide efficient algorithms for such previously non-constructive applications. As an example, we discuss a recent result of Alon, Krivelevich, and Sudakov (2007) concerning embedding nearly spanning bounded degree trees, the proof of which makes use of the Friedman-Pippenger theorem. We shall also show a construction inspired on Wigderson-Zuckerman expander graphs for which any sufficiently dense subgraph contains all trees of sizes and maximum degrees achieving essentially optimal parameters. Our algorithmic approach is based on a reduction of the tree embedding problem to a certain on-line matching problem for bipartite graphs, solved by Aggarwal et al. (1996).

TESTING STATISTICAL HYPOTHESIS ON RANDOM TREES AND APPLICATIONS TO THE PROTEIN CLASSIFICATION PROBLEM

Efficient automatic protein classification is of central importance in genomic annotation. As an independent way to check the reliability of the classification, we propose a statistical approach to test if two sets of protein domain sequences coming from two families of the Pfam database are significantly different. We model protein sequences as realizations of Variable Length Markov Chains (VLMC) and we use the context trees as a signature of each protein family. Our approach is based on a Kolmogorov-Smirnov-type goodness-of-fit test proposed by Balding et at. [Limit theorems for sequences of random trees (2008), DOI: 10.1007/s11749-008-0092-z]. The test statistic is a supremum over the space of trees of a function of the two samples; its computation grows, in principle, exponentially fast with the maximal number of nodes of the potential trees. We show how to transform this problem into a max-flow over a related graph which can be solved using a Ford-Fulkerson algorithm in polynomial time on that number. We apply the test to 10 randomly chosen protein domain families from the seed of Pfam-A database (high quality, manually curated families). The test shows that the distributions of context trees coming from different families are significantly different. We emphasize that this is a novel mathematical approach to validate the automatic clustering of sequences in any context. We also study the performance of the test via simulations on Galton-Watson related processes.

On Building a Memory Evolutive System for Application to Learning and Cognition Modeling

We address here aspects of the implementation of a memory evolutive system (MES), based on the model proposed by A. Ehresmann and J. Vanbremeersch (2007), by means of a simulated network of spiking neurons with time dependent plasticity. We point out the advantages and challenges of applying category theory for the representation of cognition, by using the MES architecture. Then we discuss the issues concerning the minimum requirements that an artificial neural network (ANN) should fulfill in order that it would be capable of expressing the categories and mappings between them, underlying the MES. We conclude that a pulsed ANN based on Izhikevich`s formal neuron with STDP (spike time-dependent plasticity) has sufficient dynamical properties to achieve these requirements, provided it can cope with the topological requirements. Finally, we present some perspectives of future research concerning the proposed ANN topology.

On S-asymptotically omega-periodic functions on Banach spaces and applications

This paper is devoted to the study of the class of continuous and bounded functions f : [0, infinity] -> X for which exists omega > 0 such that lim(t ->infinity) (f (t + omega) - f (t)) = 0 (in the sequel called S-asymptotically omega-periodic functions). We discuss qualitative properties and establish some relationships between this type of functions and the class of asymptotically omega-periodic functions. We also study the existence of S-asymptotically omega-periodic mild solutions of the first-order abstract Cauchy problem in Banach spaces. (C) 2008 Elsevier Inc. All rights reserved.

Computational methods for prediction of T-cell epitopes - a framework for modelling, testing, and applications

Computational models complement laboratory experimentation for efficient identification of MHC-binding peptides and T-cell epitopes. Methods for prediction of MHC-binding peptides include binding motifs, quantitative matrices, artificial neural networks, hidden Markov models, and molecular modelling. Models derived by these methods have been successfully used for prediction of T-cell epitopes in cancer, autoimmunity, infectious disease, and allergy. For maximum benefit, the use of computer models must be treated as experiments analogous to standard laboratory procedures and performed according to strict standards. This requires careful selection of data for model building, and adequate testing and validation. A range of web-based databases and MHC-binding prediction programs are available. Although some available prediction programs for particular MHC alleles have reasonable accuracy, there is no guarantee that all models produce good quality predictions. In this article, we present and discuss a framework for modelling, testing, and applications of computational methods used in predictions of T-cell epitopes. (C) 2004 Elsevier Inc. All rights reserved.

Effects of postnatal protein malnutrition on learning and memory procedures

Protein malnutrition induces structural, neurochemical and functional changes in the central nervous system leading to alterations in cognitive and behavioral development of rats. The aim of this work was to investigate the effects of postnatal protein malnutrition on learning and memory tasks. Previously malnourished (6% protein) and well-nourished rats (16% protein) were tested in three experiments: working memory tasks in the Morris water maze (Experiment I), recognition memory of objects (Experiment II), and working memory in the water T-maze (Experiment III). The results showed higher escape latencies in malnourished animals in Experiment I, lower recognition indexes of malnourished animals in Experiment II, and no differences due to diet in Experiment III. It is suggested that protein malnutrition imposed on early life of rats can produce impairments on both working memory in the Morris maze and recognition memory in the open field tests.

Almost automorphic mild solutions to some partial neutral functional-differential equations and applications

The paper considers the existence and uniqueness of almost automorphic mild solutions to some classes of first-order partial neutral functional-differential equations. Sufficient conditions for the existence and uniqueness of almost automorphic mild solutions to the above-mentioned equations are obtained. As an application, a first-order boundary value problem arising in control systems is considered. (C) 2007 Elsevier Ltd. All fights reserved.

Modulation of Mediotemporal and Ventrostriatal Function in Humans by Delta 9-Tetrahydrocannabinol A Neural Basis for the Effects of Cannabis sativa on Learning and Psychosis

Context: Cannabis sativa use can impair verbal learning, provoke acute psychosis, and increase the risk of schizophrenia. It is unclear where C sativa acts in the human brain to modulate verbal learning and to induce psychotic symptoms. Objectives: To investigate the effects of 2 main psychoactive constituents of C sativa, Delta 9-tetrahydrocannabinol (Delta 9-THC) and cannabidiol, on regional brain function during verbal paired associate learning. Design: Subjects were studied on 3 separate occasions using a block design functional magnetic resonance imaging paradigm while performing a verbal paired associate learning task. Each imaging session was preceded by the ingestion of Delta 9-THC (10 mg), cannabidiol (600 mg), or placebo in a double-blind, randomized, placebo-controlled, repeated-measures, within-subject design. Setting: University research center. Participants: Fifteen healthy, native English-speaking, right-handed men of white race/ethnicity who had used C sativa 15 times or less and had minimal exposure to other illicit drugs in their lifetime. Main Outcome Measures: Regional brain activation ( blood oxygen level-dependent response), performance in a verbal learning task, and objective and subjective ratings of psychotic symptoms, anxiety, intoxication, and sedation. Results: Delta 9-Tetrahydrocannabinol increased psychotic symptoms and levels of anxiety, intoxication, and sedation, whereas no significant effect was noted on these parameters following administration of cannabidiol. Performance in the verbal learning task was not significantly modulated by either drug. Administration of Delta 9-THC augmented activation in the parahippocampal gyrus during blocks 2 and 3 such that the normal linear decrement in activation across repeated encoding blocks was no longer evident. Delta 9-Tetrahydrocannabinol also attenuated the normal time-dependent change in ventrostriatal activation during retrieval of word pairs, which was directly correlated with concurrently induced psychotic symptoms. In contrast, administration of cannabidiol had no such effect. Conclusion: The modulation of mediotemporal and ventrostriatal function by Delta 9-THC may underlie the effects of C sativa on verbal learning and psychotic symptoms, respectively.