Autonomous Pricing using Policy-Gradient Reinforcement Learning

Nella letteratura economica e di teoria dei giochi vi è un dibattito aperto sulla possibilità di emergenza di comportamenti anticompetitivi da parte di algoritmi di determinazione automatica dei prezzi di mercato. L'obiettivo di questa tesi è sviluppare un modello di reinforcement learning di tipo actor-critic con entropy regularization per impostare i prezzi in un gioco dinamico di competizione oligopolistica con prezzi continui. Il modello che propongo esibisce in modo coerente comportamenti cooperativi supportati da meccanismi di punizione che scoraggiano la deviazione dall'equilibrio raggiunto a convergenza. Il comportamento di questo modello durante l'apprendimento e a convergenza avvenuta aiuta inoltre a interpretare le azioni compiute da Q-learning tabellare e altri algoritmi di prezzo in condizioni simili. I risultati sono robusti alla variazione del numero di agenti in competizione e al tipo di deviazione dall'equilibrio ottenuto a convergenza, punendo anche deviazioni a prezzi più alti.

Graph theory and geometric deep learning

La seguente tesi propone un’introduzione al geometric deep learning. Nella prima parte vengono presentati i concetti principali di teoria dei grafi ed introdotta una dinamica di diffusione su grafo, in analogia con l’equazione del calore. A seguire, iniziando dal linear classifier verranno introdotte le architetture che hanno portato all’ideazione delle graph convolutional networks. In conclusione, si analizzano esempi di alcuni algoritmi utilizzati nel geometric deep learning e si mostra una loro implementazione sul Cora dataset, un insieme di dati con struttura a grafo.

Deep learning for computer vision: a comparison between convolutional neural networks and hierarchical temporal memories on object recognition tasks

In recent years, Deep Learning techniques have shown to perform well on a large variety of problems both in Computer Vision and Natural Language Processing, reaching and often surpassing the state of the art on many tasks. The rise of deep learning is also revolutionizing the entire field of Machine Learning and Pattern Recognition pushing forward the concepts of automatic feature extraction and unsupervised learning in general. However, despite the strong success both in science and business, deep learning has its own limitations. It is often questioned if such techniques are only some kind of brute-force statistical approaches and if they can only work in the context of High Performance Computing with tons of data. Another important question is whether they are really biologically inspired, as claimed in certain cases, and if they can scale well in terms of "intelligence". The dissertation is focused on trying to answer these key questions in the context of Computer Vision and, in particular, Object Recognition, a task that has been heavily revolutionized by recent advances in the field. Practically speaking, these answers are based on an exhaustive comparison between two, very different, deep learning techniques on the aforementioned task: Convolutional Neural Network (CNN) and Hierarchical Temporal memory (HTM). They stand for two different approaches and points of view within the big hat of deep learning and are the best choices to understand and point out strengths and weaknesses of each of them. CNN is considered one of the most classic and powerful supervised methods used today in machine learning and pattern recognition, especially in object recognition. CNNs are well received and accepted by the scientific community and are already deployed in large corporation like Google and Facebook for solving face recognition and image auto-tagging problems. HTM, on the other hand, is known as a new emerging paradigm and a new meanly-unsupervised method, that is more biologically inspired. It tries to gain more insights from the computational neuroscience community in order to incorporate concepts like time, context and attention during the learning process which are typical of the human brain. In the end, the thesis is supposed to prove that in certain cases, with a lower quantity of data, HTM can outperform CNN.

Computational study on the asymmetric aminocatalysed Michael addition reaction of cyclohexanone to trans–β–nitrostyrene

Asymmetric organocatalysed reactions are one of the most fascinating synthetic strategies which one can adopt in order to induct a desired chirality into a reaction product. From all the possible practical applications of small organic molecules in catalytic reaction, amine–based catalysis has attracted a lot of attention during the past two decades. The high interest in asymmetric aminocatalytic pathways is to account to the huge variety of carbonyl compounds that can be functionalized by many different reactions of their corresponding chiral–enamine or –iminium ion as activated nucleophile and electrophile, respectively. Starting from the employment of L–Proline, many useful substrates have been proposed in order to further enhance the catalytic performances of these reaction in terms of enantiomeric excess values, yield, conversion of the substrate and turnover number. In particular, in the last decade the use of chiral and quasi–enantiomeric primary amine species has got a lot of attention in the field. Contemporaneously, many studies have been carried out in order to highlight the mechanism through which these kinds of substrates induct chirality into the desired products. In this scenario, computational chemistry has played a crucial role due to the possibility of simulating and studying any kind of reaction and the transition state structures involved. In the present work the transition state geometries of primary amine–catalysed Michael addition reaction of cyclohexanone to trans–β–nitrostyrene with different organic acid cocatalysts has been studied through different computational techniques such as density functional theory based quantum mechanics calculation and force–field directed molecular simulations.

Machine Learning Unsupervised Methods in the Design of an On-board Health Monitoring System for Satellite Applications

The dissertation starts by providing a description of the phenomena related to the increasing importance recently acquired by satellite applications. The spread of such technology comes with implications, such as an increase in maintenance cost, from which derives the interest in developing advanced techniques that favor an augmented autonomy of spacecrafts in health monitoring. Machine learning techniques are widely employed to lay a foundation for effective systems specialized in fault detection by examining telemetry data. Telemetry consists of a considerable amount of information; therefore, the adopted algorithms must be able to handle multivariate data while facing the limitations imposed by on-board hardware features. In the framework of outlier detection, the dissertation addresses the topic of unsupervised machine learning methods. In the unsupervised scenario, lack of prior knowledge of the data behavior is assumed. In the specific, two models are brought to attention, namely Local Outlier Factor and One-Class Support Vector Machines. Their performances are compared in terms of both the achieved prediction accuracy and the equivalent computational cost. Both models are trained and tested upon the same sets of time series data in a variety of settings, finalized at gaining insights on the effect of the increase in dimensionality. The obtained results allow to claim that both models, combined with a proper tuning of their characteristic parameters, successfully comply with the role of outlier detectors in multivariate time series data. Nevertheless, under this specific context, Local Outlier Factor results to be outperforming One-Class SVM, in that it proves to be more stable over a wider range of input parameter values. This property is especially valuable in unsupervised learning since it suggests that the model is keen to adapting to unforeseen patterns.

A deep learning model via thermodynamics and condensed matter physics

Deep Learning architectures give brilliant results in a large variety of fields, but a comprehensive theoretical description of their inner functioning is still lacking. In this work, we try to understand the behavior of neural networks by modelling in the frameworks of Thermodynamics and Condensed Matter Physics. We approach neural networks as in a real laboratory and we measure the frequency spectrum and the entropy of the weights of the trained model. The stochasticity of the training occupies a central role in the dynamics of the weights and makes it difficult to assimilate neural networks to simple physical systems. However, the analogy with Thermodynamics and the introduction of a well defined temperature leads us to an interesting result: if we eliminate from a CNN the "hottest" filters, the performance of the model remains the same, whereas, if we eliminate the "coldest" ones, the performance gets drastically worst. This result could be exploited in the realization of a training loop which eliminates the filters that do not contribute to loss reduction. In this way, the computational cost of the training will be lightened and more importantly this would be done by following a physical model. In any case, beside important practical applications, our analysis proves that a new and improved modeling of Deep Learning systems can pave the way to new and more efficient algorithms.

Computational investigation of a stereoselective synthesis of atropisomeric hydrazides possessing an N–N stereogenic axis

Axially chiral substrates are an interesting and widely studied class of compounds as they can be found in bioactive natural products and are employed as functional materials or as ligands in asymmetric catalytic processes. One branch of this family is the well-known world of the atropisomers. Among them, atropisomeric compounds possessing an N–N stereogenic axis are one truthfully fascinating system but not completely understood yet. In this thesis, we computationally investigated the mechanism of the diastereoselective formation of the N – N chiral axis of a hydrazide under asymmetric phase transfer catalytic conditions. Moreover, during this study, torsional barriers have been calculated for both the reagent and the product at the density functional theory (DFT). These values turned out to suitably match the experimental values and observations. Finally, Electronic Circular Dichroism (ECD) spectra have been simulated in order to assign the chiral absolute configuration to the products.

A biophysical model for valence-based navigational learning: the role of D-Serine

Recent experiments have revealed the fundamental importance of neuromodulatory action on activity-dependent synaptic plasticity underlying behavioral learning and spatial memory formation. Neuromodulators affect synaptic plasticity through the modification of the dynamics of receptors on the synaptic membrane. However, chemical substances other than neuromodulators, such as receptors co-agonists, can influence the receptors' dynamics and thus participate in determining plasticity. Here we focus on D-serine, which has been observed to affect the activity thresholds of synaptic plasticity by co-activating NMDA receptors. We use a computational model for spatial value learning with plasticity between two place cell layers. The D-serine release is CB1R mediated and the model reproduces the impairment of spatial memory due to the astrocytic CB1R knockout for a mouse navigating in the Morris water maze. The addition of path-constraining obstacles shows how performance impairment depends on the environment's topology. The model can explain the experimental evidence and produce useful testable predictions to increase our understanding of the complex mechanisms underlying learning.