Dynamic Human Robot Interaction Framework Using Deep Learning and Robot Operating System (ROS): a practical approach

Trying to explain to a robot what to do is a difficult undertaking, and only specific types of people have been able to do so far, such as programmers or operators who have learned how to use controllers to communicate with a robot. My internship's goal was to create and develop a framework that would make that easier. The system uses deep learning techniques to recognize a set of hand gestures, both static and dynamic. Then, based on the gesture, it sends a command to a robot. To be as generic as feasible, the communication is implemented using Robot Operating System (ROS). Furthermore, users can add new recognizable gestures and link them to new robot actions; a finite state automaton enforces the users' input verification and correct action sequence. Finally, the users can create and utilize a macro to describe a sequence of actions performable by a robot.

Installazione del navigation stack su rover terrestre e applicazione del kinect nello human-robot interaction.

Progetto SHERPA. Installazione e configurazione del Navigaton Stack su Rover terrestre. Utilizzo e configurazione di LMS151 Sick. Utilizzo e configurazione di Asus Xtion Pro. Progettazione di software per la localizzazione e l'inseguimento di persone tramite camera di profondita.

Robotics and neurodevelopmental disorder: design and implementation of a system of engagement metrics in human-robot interactions.

In the recent decades, robotics has become firmly embedded in areas such as education, teaching, medicine, psychology and many others. We focus here on social robotics; social robots are designed to interact with people in a natural and interpersonal way, often to achieve positive results in different applications. To interact and cooperate with humans in their daily-life activities, robots should exhibit human-like intelligence. The rapid expansion of social robotics and the existence of various kinds of robots on the market have allowed research groups to carry out multiple experiments. The experiments carried out have led to the collections of various kinds of data, which can be used or processed for psychological studies, and studies in other fields. However, there are no tools available in which data can be stored, processed and shared with other research groups. This thesis proposes the design and implementation of visual tool for organizing dataflows in Human Robot Interaction (HRI).

An Embedded In-Cabin Lightweight High-Performance 3D Gaze Estimation System

Gaze estimation has gained interest in recent years for being an important cue to obtain information about the internal cognitive state of humans. Regardless of whether it is the 3D gaze vector or the point of gaze (PoG), gaze estimation has been applied in various fields, such as: human robot interaction, augmented reality, medicine, aviation and automotive. In the latter field, as part of Advanced Driver-Assistance Systems (ADAS), it allows the development of cutting-edge systems capable of mitigating road accidents by monitoring driver distraction. Gaze estimation can be also used to enhance the driving experience, for instance, autonomous driving. It also can improve comfort with augmented reality components capable of being commanded by the driver's eyes. Although, several high-performance real-time inference works already exist, just a few are capable of working with only a RGB camera on computationally constrained devices, such as a microcontroller. This work aims to develop a low-cost, efficient and high-performance embedded system capable of estimating the driver's gaze using deep learning and a RGB camera. The proposed system has achieved near-SOTA performances with about 90% less memory footprint. The capabilities to generalize in unseen environments have been evaluated through a live demonstration, where high performance and near real-time inference were obtained using a webcam and a Raspberry Pi4.

Registrazione e replicazione di procedure Web con una Firefox Extension

In questo lavoro di tesi è stata sviluppata una Firefox Extension per la registrazione e la replicazione di procedure sul Web. Si tratterà a fondo l’ambiente tecnologico nel quale è stata sviluppata l’applicazione e il contesto in cui si inserisce una Firefox Extension. Illustreremo il problema che intendiamo risolvere con la nostra estensione,il contesto applicativo in cui si inserisce e riporteremo una serie di lavori correlati che cercano, con diversi approcci, di risolvere il nostro stesso problema. Illustreremo il lavoro trattando approfonditamente l’approccio da noi utilizzato, mostrandone i vantaggi e i limiti.

Interfaccia grafica per l'elaborazione di immagini microscopiche di colture batteriche geneticamente modificate

Le moderne tecniche di imaging e i recenti sviluppi nel campo della visione computazionale consentono sempre più diffusamente l'utilizzo di metodi di image analysis, specialmente in ambito medico e biologico, permettendo un maggiore supporto sia alla diagnosi, sia alla ricerca. Il lavoro svolto in questa tesi si pone in un contesto di ricerca di carattere interdisciplinare, e riguarda il progetto e la realizzazione di un‘interfaccia grafica per l'analisi di colture batteriche geneticamente modificate, marcate con proteine fluorescenti (GFP), acquisite tramite un microscopio ad epifluorescenza. Nota la funzione di risposta del sistema di acquisizione delle immagini, l'analisi quantitativa delle colture batteriche è effettuata mediante la misurazione di proprietà legate all'intensità della risposta al marcatore fluorescente. L'interfaccia consente un'analisi sia globale dei batteri individuati nell'immagine, sia di singoli gruppi di batteri selezionati dall'utente, fornendo utili informazioni statistiche, sia in forma grafica che numerica. Per la realizzazione dell'interfaccia sono state adottate tecniche di ingegneria del software, con particolare enfasi alla interazione uomo-macchina e seguendo criteri di usability, al fine di consentire un corretto utilizzo dello strumento anche da parte di personale senza conoscenza in campo informatico.

Evoluzione delle user interface: dalla metafora del desktop al Wearable Computing

Negli ultimi decenni abbiamo assistito ad una graduale evoluzione delle interfacce utente e della tecnologia. Sono stati introdotti nuovi dispositivi mobile e wearable che negli ultimi anni hanno subito un incremento tecnologico esponenziale arrivando a fondersi con la vita di tutti i giorni. Le classiche interfacce grafiche WIMP, la metafora del desktop e le linee guida di progettazione fino ad ora sviluppate non risultano ideali per la nuova tecnologia di wearable computing. Il proposito che la tesi vuole andare ad affrontare è proprio quello di indagare lo sviluppo di nuove user inteface basate sulla tecnologia wearable ed in particolare per smart glasses.

Task analysis for HMI design. Level of automation study of aircraft navigation functions

Advancements in technology have enabled increasingly sophisticated automation to be introduced into the flight decks of modern aircraft. Generally, this automation was added to accomplish worthy objectives such as reducing flight crew workload, adding additional capability, or increasing fuel economy. Automation is necessary due to the fact that not all of the functions required for mission accomplishment in today’s complex aircraft are within the capabilities of the unaided human operator, who lacks the sensory capacity to detect much of the information required for flight. To a large extent, these objectives have been achieved. Nevertheless, despite all the benefits from the increasing amounts of highly reliable automation, vulnerabilities do exist in flight crew management of automation and Situation Awareness (SA). Issues associated with flight crew management of automation include: • Pilot understanding of automation’s capabilities, limitations, modes, and operating principles and techniques. • Differing pilot decisions about the appropriate automation level to use or whether to turn automation on or off when they get into unusual or emergency situations. • Human-Machine Interfaces (HMIs) are not always easy to use, and this aspect could be problematic when pilots experience high workload situations. • Complex automation interfaces, large differences in automation philosophy and implementation among different aircraft types, and inadequate training also contribute to deficiencies in flight crew understanding of automation.

Motor Unit Spike Trains Recontruction from sEMG Signals for Gesture Classification on a Low-Power Processing Platform

Hand gesture recognition based on surface electromyography (sEMG) signals is a promising approach for the development of intuitive human-machine interfaces (HMIs) in domains such as robotics and prosthetics. The sEMG signal arises from the muscles' electrical activity, and can thus be used to recognize hand gestures. The decoding from sEMG signals to actual control signals is non-trivial; typically, control systems map sEMG patterns into a set of gestures using machine learning, failing to incorporate any physiological insight. This master thesis aims at developing a bio-inspired hand gesture recognition system based on neuromuscular spike extraction rather than on simple pattern recognition. The system relies on a decomposition algorithm based on independent component analysis (ICA) that decomposes the sEMG signal into its constituent motor unit spike trains, which are then forwarded to a machine learning classifier. Since ICA does not guarantee a consistent motor unit ordering across different sessions, 3 approaches are proposed: 2 ordering criteria based on firing rate and negative entropy, and a re-calibration approach that allows the decomposition model to retain information about previous sessions. Using a multilayer perceptron (MLP), the latter approach results in an accuracy up to 99.4% in a 1-subject, 1-degree of freedom scenario. Afterwards, the decomposition and classification pipeline for inference is parallelized and profiled on the PULP platform, achieving a latency < 50 ms and an energy consumption < 1 mJ. Both the classification models tested (a support vector machine and a lightweight MLP) yielded an accuracy > 92% in a 1-subject, 5-classes (4 gestures and rest) scenario. These results prove that the proposed system is suitable for real-time execution on embedded platforms and also capable of matching the accuracy of state-of-the-art approaches, while also giving some physiological insight on the neuromuscular spikes underlying the sEMG.

Progettazione ed Implementazione Front-End di un Sistema Integrato di Customer Assistance in ambito Industry 4.0

Al giorno d'oggi, l'industry 4.0 è un movimento sempre più prominente che induce ad equipaggiare gli impianti industriali con avanzate infrastrutture tecnologiche digitali, le quali operano sinergicamente con l'impianto, al fine di controllare ed aumentare la produttività, monitorare e prevenire i futuri guasti, ed altro ancora. In questo ambito, gli utenti sono parte integrante della struttura produttiva, in cui ricoprono ruoli strategici e flessibili, collaborano fra loro e con le macchine, con l’obiettivo di affrontare e risolvere proattivamente una vasta gamma di problemi complessi. In particolare, la customer assistance nel settore industriale può certamente variare in relazione a molteplici elementi: il tipo di produzione e le caratteristiche del prodotto; l'organizzazione ed infrastruttura aziendale interna; la quantità di risorse disponibili che possono essere impiegate; il grado di importanza ricoperto dalla customer assistance nel settore industriale di riferimento; altri eventuali fattori appartenenti ad un dominio specifico. Per queste ragioni, si è cercato di individuare e categorizzare nel modo più accurato possibile, il lavoro svolto in questo elaborato ed il contesto nel quale è stato sviluppato. In questa tesi, viene descritta un'applicazione web per erogare assistenza al cliente in ambito di industria 4.0, attraverso il paradigma di ticketing o ticket di supporto/assistenza. Questa applicazione è integrata nel sistema Mentor, il quale è attivo già da anni nel settore industriale 4.0. Il progetto Mentor è una suite di applicazioni cloud-based creata dal gruppo Bucci Industries, una multinazionale attiva nell'industria e nell'automazione con sede a Faenza. In questo caso di studio, si presenta la progettazione ed implementazione della parte front-end del suddetto sistema di assistenza, il quale è integrato ed interconnesso con un paio di applicazioni tipiche di industria 4.0, presenti nella stessa suite di applicazioni.

Integrating an Emotion Recognition Model for the Flobi System

This thesis investigates if emotional states of users interacting with a virtual robot can be recognized reliably and if specific interaction strategy can change the users’ emotional state and affect users’ risk decision. For this investigation, the OpenFace [1] emotion recognition model was intended to be integrated into the Flobi [2] system, to allow the agent to be aware of the current emotional state of the user and to react appropriately. There was an open source ROS [3] bridge available online to integrate OpenFace to the Flobi simulation but it was not consistent with some other projects in Flobi distribution. Then due to technical reasons DeepFace was selected. In a human-agent interaction, the system is compared to a system without using emotion recognition. Evaluation could happen at different levels: evaluation of emotion recognition model, evaluation of the interaction strategy, and evaluation of effect of interaction on user decision. The results showed that the happy emotion induction was 58% and fear emotion induction 77% successful. Risk decision results show that: in happy induction after interaction 16.6% of participants switched to a lower risk decision and 75% of them did not change their decision and the remaining switched to a higher risk decision. In fear inducted participants 33.3% decreased risk 66.6 % did not change their decision The emotion recognition accuracy was and had bias to. The sensitivity and specificity is calculated for each emotion class. The emotion recognition model classifies happy emotions as neutral in most of the time.

Neural Machine Translation in the Field of Law. Analysis and Evaluation against Human Reference of NMT Raw Outputs of an Atto Costitutivo of S.r.l. from Italian into German.

Artificial Intelligence (AI) is gaining ever more ground in every sphere of human life, to the point that it is now even used to pass sentences in courts. The use of AI in the field of Law is however deemed quite controversial, as it could provide more objectivity yet entail an abuse of power as well, given that bias in algorithms behind AI may cause lack of accuracy. As a product of AI, machine translation is being increasingly used in the field of Law too in order to translate laws, judgements, contracts, etc. between different languages and different legal systems. In the legal setting of Company Law, accuracy of the content and suitability of terminology play a crucial role within a translation task, as any addition or omission of content or mistranslation of terms could entail legal consequences for companies. The purpose of the present study is to first assess which neural machine translation system between DeepL and ModernMT produces a more suitable translation from Italian into German of the atto costitutivo of an Italian s.r.l. in terms of accuracy of the content and correctness of terminology, and then to assess which translation proves to be closer to a human reference translation. In order to achieve the above-mentioned aims, two human and automatic evaluations are carried out based on the MQM taxonomy and the BLEU metric. Results of both evaluations show an overall better performance delivered by ModernMT in terms of content accuracy, suitability of terminology, and closeness to a human translation. As emerged from the MQM-based evaluation, its accuracy and terminology errors account for just 8.43% (as opposed to DeepL’s 9.22%), while it obtains an overall BLEU score of 29.14 (against DeepL’s 27.02). The overall performances however show that machines still face barriers in overcoming semantic complexity, tackling polysemy, and choosing domain-specific terminology, which suggests that the discrepancy with human translation may still be remarkable.