Concepts and methods for efficient decentralized learning in federated settings

Deep Neural Networks (DNNs) have revolutionized a wide range of applications beyond traditional machine learning and artificial intelligence fields, e.g., computer vision, healthcare, natural language processing and others. At the same time, edge devices have become central in our society, generating an unprecedented amount of data which could be used to train data-hungry models such as DNNs. However, the potentially sensitive or confidential nature of gathered data poses privacy concerns when storing and processing them in centralized locations. To this purpose, decentralized learning decouples model training from the need of directly accessing raw data, by alternating on-device training and periodic communications. The ability of distilling knowledge from decentralized data, however, comes at the cost of facing more challenging learning settings, such as coping with heterogeneous hardware and network connectivity, statistical diversity of data, and ensuring verifiable privacy guarantees. This Thesis proposes an extensive overview of decentralized learning literature, including a novel taxonomy and a detailed description of the most relevant system-level contributions in the related literature for privacy, communication efficiency, data and system heterogeneity, and poisoning defense. Next, this Thesis presents the design of an original solution to tackle communication efficiency and system heterogeneity, and empirically evaluates it on federated settings. For communication efficiency, an original method, specifically designed for Convolutional Neural Networks, is also described and evaluated against the state-of-the-art. Furthermore, this Thesis provides an in-depth review of recently proposed methods to tackle the performance degradation introduced by data heterogeneity, followed by empirical evaluations on challenging data distributions, highlighting strengths and possible weaknesses of the considered solutions. Finally, this Thesis presents a novel perspective on the usage of Knowledge Distillation as a mean for optimizing decentralized learning systems in settings characterized by data heterogeneity or system heterogeneity. Our vision on relevant future research directions close the manuscript.

La didattica della traduzione specializzata dallo spagnolo in italiano: competenza traduttiva, genere testuale e nuove tecnologie

Il progetto di ricerca che presentiamo nasce dalla virtuosa combinazione di teoria e prassi didattica nello spirito della ricerca-azione. Scopo del presente lavoro è elaborare un percorso didattico di formazione alla traduzione specializzata in ambito medico-scientifico, tecnico ed economico-giuridico per la combinazione linguistica spagnolo-italiano all’interno della cornice istituzionale concreta dell’università italiana oggi. La nostra proposta formativa si fonda su tre elementi: la ricognizione del mercato attuale della traduzione per la combinazione linguistica indicata, l’individuazione degli obiettivi formativi in base al modello di competenza traduttiva scelto, l’elaborazione del percorso didattico per competenze e basato sull’enfoque por tareas di traduzione. Nella progettazione delle modalità didattiche due sono gli aspetti che definiscono il percorso proposto: il concetto di genere testuale specializzato per la traduzione e la gestione delle informazioni mediante le nuove tecnologie (corpora, banche dati terminologiche e fraseologiche, memorie di traduzione, traduzione controllata). Il presente lavoro si articola in due parti: la prima parte (quattro capitoli) presenta l’inquadramento teorico all’interno del quale si sviluppa la riflessione intorno alla didattica della traduzione specializzata; la seconda parte (due capitoli) presenta l’inquadramento metodologico e analitico all’interno del quale si elabora la nostra proposta didattica. Nel primo capitolo si illustrano i rapporti fra traduzione e mondo professionale; nel secondo capitolo si presenta il concetto di competenza traduttiva come ponte tra la formazione e il mondo della traduzione professionale; nel terzo capitolo si ripercorrono le tappe principali dell’evoluzione della didattica della traduzione generale; nel quarto capitolo illustriamo alcune tra le più recenti e complete proposte didattiche per la traduzione specializzata in ambito tecnico, medico-scientifico ed economico-giuridico. Nel quinto capitolo si introduce il concetto di genere testuale specializzato per la traduzione e nel sesto capitolo si illustra la proposta didattica per la traduzione specializzata dallo spagnolo in italiano che ha motivato il presente lavoro.

Machine learning as a service for high energy physics (MLaaS4HEP): a service for ML-based data analyses

With the CERN LHC program underway, there has been an acceleration of data growth in the High Energy Physics (HEP) field and the usage of Machine Learning (ML) in HEP will be critical during the HL-LHC program when the data that will be produced will reach the exascale. ML techniques have been successfully used in many areas of HEP nevertheless, the development of a ML project and its implementation for production use is a highly time-consuming task and requires specific skills. Complicating this scenario is the fact that HEP data is stored in ROOT data format, which is mostly unknown outside of the HEP community. The work presented in this thesis is focused on the development of a ML as a Service (MLaaS) solution for HEP, aiming to provide a cloud service that allows HEP users to run ML pipelines via HTTP calls. These pipelines are executed by using the MLaaS4HEP framework, which allows reading data, processing data, and training ML models directly using ROOT files of arbitrary size from local or distributed data sources. Such a solution provides HEP users non-expert in ML with a tool that allows them to apply ML techniques in their analyses in a streamlined manner. Over the years the MLaaS4HEP framework has been developed, validated, and tested and new features have been added. A first MLaaS solution has been developed by automatizing the deployment of a platform equipped with the MLaaS4HEP framework. Then, a service with APIs has been developed, so that a user after being authenticated and authorized can submit MLaaS4HEP workflows producing trained ML models ready for the inference phase. A working prototype of this service is currently running on a virtual machine of INFN-Cloud and is compliant to be added to the INFN Cloud portfolio of services.

Directly training spiking neural networks for cyber-physical systems: from supervised to reinforcement learning

Spiking Neural Networks (SNNs) are bio-inspired Artificial Neural Networks (ANNs) utilizing discrete spiking signals, akin to neuron communication in the brain, making them ideal for real-time and energy-efficient Cyber-Physical Systems (CPSs). This thesis explores their potential in Structural Health Monitoring (SHM), leveraging low-cost MEMS accelerometers for early damage detection in motorway bridges. The study focuses on Long Short-Term SNNs (LSNNs), although their complex learning processes pose challenges. Comparing LSNNs with other ANN models and training algorithms for SHM, findings indicate LSNNs' effectiveness in damage identification, comparable to ANNs trained using traditional methods. Additionally, an optimized embedded LSNN implementation demonstrates a 54% reduction in execution time, but with longer pre-processing due to spike-based encoding. Furthermore, SNNs are applied in UAV obstacle avoidance, trained directly using a Reinforcement Learning (RL) algorithm with event-based input from a Dynamic Vision Sensor (DVS). Performance evaluation against Convolutional Neural Networks (CNNs) highlights SNNs' superior energy efficiency, showing a 6x decrease in energy consumption. The study also investigates embedded SNN implementations' latency and throughput in real-world deployments, emphasizing their potential for energy-efficient monitoring systems. This research contributes to advancing SHM and UAV obstacle avoidance through SNNs' efficient information processing and decision-making capabilities within CPS domains.