Riconoscimento real-time di gesture tramite tecniche di machine learning

Il riconoscimento delle gesture è un tema di ricerca che sta acquisendo sempre più popolarità, specialmente negli ultimi anni, grazie ai progressi tecnologici dei dispositivi embedded e dei sensori. Lo scopo di questa tesi è quello di utilizzare alcune tecniche di machine learning per realizzare un sistema in grado di riconoscere e classificare in tempo reale i gesti delle mani, a partire dai segnali mioelettrici (EMG) prodotti dai muscoli. Inoltre, per consentire il riconoscimento di movimenti spaziali complessi, verranno elaborati anche segnali di tipo inerziale, provenienti da una Inertial Measurement Unit (IMU) provvista di accelerometro, giroscopio e magnetometro. La prima parte della tesi, oltre ad offrire una panoramica sui dispositivi wearable e sui sensori, si occuperà di analizzare alcune tecniche per la classificazione di sequenze temporali, evidenziandone vantaggi e svantaggi. In particolare, verranno considerati approcci basati su Dynamic Time Warping (DTW), Hidden Markov Models (HMM), e reti neurali ricorrenti (RNN) di tipo Long Short-Term Memory (LSTM), che rappresentano una delle ultime evoluzioni nel campo del deep learning. La seconda parte, invece, riguarderà il progetto vero e proprio. Verrà impiegato il dispositivo wearable Myo di Thalmic Labs come caso di studio, e saranno applicate nel dettaglio le tecniche basate su DTW e HMM per progettare e realizzare un framework in grado di eseguire il riconoscimento real-time di gesture. Il capitolo finale mostrerà i risultati ottenuti (fornendo anche un confronto tra le tecniche analizzate), sia per la classificazione di gesture isolate che per il riconoscimento in tempo reale.

Self-regulated learning in computer programming : strategies students adopted during an assignment

The SimProgramming teaching approach has the goal to help students overcome their learning difficulties in the transition from entry-level to advanced computer programming and prepare them for real-world labour environments, adopting learning strategies. It immerses learners in a businesslike learning environment, where students develop a problem-based learning activity with a specific set of tasks, one of which is filling weekly individual forms. We conducted thematic analysis of 401 weekly forms, to identify the students’ strategies for self-regulation of learning during assignment. The students are adopting different strategies in each phase of the approach. The early phases are devoted to organization and planning, later phases focus on applying theoretical knowledge and hands-on programming. Based on the results, we recommend the development of educational practices to help students conduct self-reflection of their performance during tasks.

Self-regulated learning in higher education : strategies adopted by computer programming students

Trabalho apresentado em PAEE/ALE’2016, 8th International Symposium on Project Approaches in Engineering Education (PAEE) and 14th Active Learning in Engineering Education Workshop (ALE)

Comparative study of human age estimation based on hand-crafted and deep face features

In the past few years, human facial age estimation has drawn a lot of attention in the computer vision and pattern recognition communities because of its important applications in age-based image retrieval, security control and surveillance, biomet- rics, human-computer interaction (HCI) and social robotics. In connection with these investigations, estimating the age of a person from the numerical analysis of his/her face image is a relatively new topic. Also, in problems such as Image Classification the Deep Neural Networks have given the best results in some areas including age estimation. In this work we use three hand-crafted features as well as five deep features that can be obtained from pre-trained deep convolutional neural networks. We do a comparative study of the obtained age estimation results with these features.

Valutazione delle tecniche di parallelizzazione nell'addestramento di una CNN per la diagnostica del cancro al seno.

Il cancro è un processo autosufficiente e adattivo che interagisce dinamicamente con il suo microambiente, la cui diagnosi, complessa e dispendiosa in termini di tempo e numero di specialisti impiegati, viene solitamente effettuata valutando l’imaging radiografico oppure effettuando un esame istologico. L'interpretazione di tali immagini risulta generalmente molto complessa, a questo scopo sarebbe molto utile poter addestrare un computer a comprendere tali immagini potendo di fatto affiancarsi allo specialista, senza sostituirlo, al momento della diagnosi. A questo scopo è possibile affidarsi alle tecniche di apprendimento automatico, sistema alla base dell’intelligenza artificiale (AI), le quali permettono di fatto di apprendere automaticamente la rappresentazione delle caratteristiche da immagini campione. Tali tecniche di intelligenza artificiale, hanno però bisogno, per essere addestrate, di grandi quantità di dati in cui il segnale di uscita desiderato è noto, comportando di fatto un aumento delle tempistiche di addestramento. Inoltre, in ambito sanitario, i dati sono distribuiti su più archivi, dislocati sul territorio nazionale, rendendo impossibile l’utilizzo di soluzioni centralizzate. L’obbiettivo di questa trattazione sarà cercare di trovare una soluzione a queste due problematiche, ricorrendo all’utilizzo delle tecniche di parallelizzazione. A seguito dell'introduzione dello scenario biologico e delle tecniche di diagnostica ad esso associato è presentato il percorso di creazione della rete neurale. A seguito del suo addestramento sulla GPU di una singola macchina, ottenendo un'accuratezza dell'83.94% in 5 ore 48 minuti e 43 secondi, è stata introdotto la parallelizzazione ed una sua implementazione. In conclusione, sfruttando il sistema implementato, è stata distribuita la fase di addestramento prima su due macchine e poi su tre, ottenendo una diminuzione del tempo di addestramento rispettivamente del 31.4% e del 50%.

Integrazione di reti neurali con approcci simbolici per problemi di ragionamento deduttivo in linguaggio naturale

Uno degli obiettivi più ambizioni e interessanti dell'informatica, specialmente nel campo dell'intelligenza artificiale, consiste nel raggiungere la capacità di far ragionare un computer in modo simile a come farebbe un essere umano. I più recenti successi nell'ambito delle reti neurali profonde, specialmente nel campo dell'elaborazione del testo in linguaggio naturale, hanno incentivato lo studio di nuove tecniche per affrontare tale problema, a cominciare dal ragionamento deduttivo, la forma più semplice e lineare di ragionamento logico. La domanda fondamentale alla base di questa tesi è infatti la seguente: in che modo una rete neurale basata sull'architettura Transformer può essere impiegata per avanzare lo stato dell'arte nell'ambito del ragionamento deduttivo in linguaggio naturale? Nella prima parte di questo lavoro presento uno studio approfondito di alcune tecnologie recenti che hanno affrontato questo problema con intuizioni vincenti. Da questa analisi emerge come particolarmente efficace l'integrazione delle reti neurali con tecniche simboliche più tradizionali. Nella seconda parte propongo un focus sull'architettura ProofWriter, che ha il pregio di essere relativamente semplice e intuitiva pur presentando prestazioni in linea con quelle dei concorrenti. Questo approfondimento mette in luce la capacità dei modelli T5, con il supporto del framework HuggingFace, di produrre più risposte alternative, tra cui è poi possibile cercare esternamente quella corretta. Nella terza e ultima parte fornisco un prototipo che mostra come si può impiegare tale tecnica per arricchire i sistemi tipo ProofWriter con approcci simbolici basati su nozioni linguistiche, conoscenze specifiche sul dominio applicativo o semplice buonsenso. Ciò che ne risulta è un significativo miglioramento dell'accuratezza rispetto al ProofWriter originale, ma soprattutto la dimostrazione che è possibile sfruttare tale capacità dei modelli T5 per migliorarne le prestazioni.

Sviluppo di Retrieval-based Chatbot per l'Italiano con Transformer

L’Intelligenza Artificiale negli ultimi anni sta plasmando il futuro dell’umanità in quasi tutti i settori. È già il motore principale di diverse tecnologie emergenti come i big data, la robotica e l’IoT e continuerà ad agire come innovatore tecnologico nel futuro prossimo. Le recenti scoperte e migliorie sia nel campo dell’hardware che in quello matematico hanno migliorato l’efficienza e ridotto i tempi di esecuzione dei software. È in questo contesto che sta evolvendo anche il Natural Language Processing (NLP), un ramo dell’Intelligenza Artificiale che studia il modo in cui fornire ai computer l'abilità di comprendere un testo scritto o parlato allo stesso modo in cui lo farebbe un essere umano. Le ambiguità che distinguono la lingua naturale dalle altre rendono ardui gli studi in questo settore. Molti dei recenti sviluppi algoritmici su NLP si basano su tecnologie inventate decenni fa. La ricerca in questo settore è quindi in continua evoluzione. Questa tesi si pone l'obiettivo di sviluppare la logica di una chatbot help-desk per un'azienda privata. Lo scopo è, sottoposta una domanda da parte di un utente, restituire la risposta associata presente in una collezione domande-risposte. Il problema che questa tesi affronta è sviluppare un modello di NLP in grado di comprendere il significato semantico delle domande in input, poiché esse possono essere formulate in molteplici modi, preservando il contenuto semantico a discapito della sintassi. A causa delle ridotte dimensioni del dataset italiano proprietario su cui testare il modello chatbot, sono state eseguite molteplici sperimentazioni su un ulteriore dataset italiano con task affine. Attraverso diversi approcci di addestramento, tra cui apprendimento metrico, sono state raggiunte alte accuratezze sulle più comuni metriche di valutazione, confermando le capacità del modello proposto e sviluppato.

Continual learning on nano drones

Miniaturized flying robotic platforms, called nano-drones, have the potential to revolutionize the autonomous robots industry sector thanks to their very small form factor. The nano-drones’ limited payload only allows for a sub-100mW microcontroller unit for the on-board computations. Therefore, traditional computer vision and control algorithms are too computationally expensive to be executed on board these palm-sized robots, and we are forced to rely on artificial intelligence to trade off accuracy in favor of lightweight pipelines for autonomous tasks. However, relying on deep learning exposes us to the problem of generalization since the deployment scenario of a convolutional neural network (CNN) is often composed by different visual cues and different features from those learned during training, leading to poor inference performances. Our objective is to develop and deploy and adaptation algorithm, based on the concept of latent replays, that would allow us to fine-tune a CNN to work in new and diverse deployment scenarios. To do so we start from an existing model for visual human pose estimation, called PULPFrontnet, which is used to identify the pose of a human subject in space through its 4 output variables, and we present the design of our novel adaptation algorithm, which features automatic data gathering and labeling and on-device deployment. We therefore showcase the ability of our algorithm to adapt PULP-Frontnet to new deployment scenarios, improving the R2 scores of the four network outputs, with respect to an unknown environment, from approximately [−0.2, 0.4, 0.0,−0.7] to [0.25, 0.45, 0.2, 0.1]. Finally we demonstrate how it is possible to fine-tune our neural network in real time (i.e., under 76 seconds), using the target parallel ultra-low power GAP 8 System-on-Chip on board the nano-drone, and we show how all adaptation operations can take place using less than 2mWh of energy, a small fraction of the available battery power.

Development of a data acquisition system to characterise sustainable energy technologies and to support Web-Based learning (DAQ-WBL)

Driven by concerns about rising energy costs, security of supply and climate change a new wave of Sustainable Energy Technologies (SET’s) have been embraced by the Irish consumer. Such systems as solar collectors, heat pumps and biomass boilers have become common due to government backed financial incentives and revisions of the building regulations. However, there is a deficit of knowledge and understanding of how these technologies operate and perform under Ireland’s maritime climate. This AQ-WBL project was designed to address both these needs by developing a Data Acquisition (DAQ) system to monitor the performance of such technologies and a web-based learning environment to disseminate performance characteristics and supplementary information about these systems. A DAQ system consisting of 108 sensors was developed as part of Galway-Mayo Institute of Technology’s (GMIT’s) Centre for the Integration of Sustainable EnergyTechnologies (CiSET) in an effort to benchmark the performance of solar thermal collectors and Ground Source Heat Pumps (GSHP’s) under Irish maritime climate, research new methods of integrating these systems within the built environment and raise awareness of SET’s. It has operated reliably for over 2 years and has acquired over 25 million data points. Raising awareness of these SET’s is carried out through the dissemination of the performance data through an online learning environment. A learning environment was created to provide different user groups with a basic understanding of a SET’s with the support of performance data, through a novel 5 step learning process and two examples were developed for the solar thermal collectors and the weather station which can be viewed at http://www.kdp 1 .aquaculture.ie/index.aspx. This online learning environment has been demonstrated to and well received by different groups of GMIT’s undergraduate students and plans have been made to develop it further to support education, awareness, research and regional development.

The Value of Imperfection : the Wabi-Sabi Principle in Aesthetics and Learning

OER-based learning has the potential to overcome many shortcomings and problems of traditional education. It is not hampered by IP restrictions; can depend on collaborative, cumulative, iterative refinement of resources; and the digital form provides unprecedented flexibility with respect to configuration and delivery. The OER community is a progressive group of educators and learners with decades of learning research to draw from, who know that we must prepare learners for an evolving and diverse reality. Despite this OER tends to replicate the unsuccessful characteristics of traditional education. To remedy this we may need to remember the importance of imperfection, mistakes, problems, disagreement, and the incomplete for engaged learning, and relinquish our notions of perfection, acknowledging that learners learn differently and we need diverse learners. We must stretch our perceptions of quality and provide mechanisms for engaging the incredible pool of educators globally to fulfill the promise of inclusive education.

Reinforcement learning utilizes proxemics: an avatar learns to manipulate the position of people in immersive virtual reality

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.

Quality of Learners' Time and Learning Performance Beyond Quantitative Time-on-Task

Peer-reviewed

Reinforcement learning utilizes proxemics: an avatar learns to manipulate the position of people in immersive virtual reality

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.

The development and evaluation of a distance learning system in ophthalmology

INTRODUCTION: Web-based e-learning is a teaching tool increasingly used in many medical schools and specialist fields, including ophthalmology. AIMS: this pilot study aimed to develop internet-based course-based clinical cases and to evaluate the effectiveness of this method within a graduate medical education group. METHODS: this was an interventional randomized study. First, a website was built using a distance learning platform. Sixteen first-year ophthalmology residents were then divided into two randomized groups: one experimental group, which was submitted to the intervention (use of the e-learning site) and another control group, which was not submitted to the intervention. The students answered a printed clinical case and their scores were compared. RESULTS: there was no statistically significant difference between the groups. CONCLUSION: We were able to successfully develop the e-learning site and the respective clinical cases. Despite the fact that there was no statistically significant difference between the access and the non access group, the study was a pioneer in our department, since a clinical case online program had never previously been developed.

Deep generative models for facial keypoints detection

A new area of machine learning research called deep learning, has moved machine learning closer to one of its original goals: artificial intelligence and general learning algorithm. The key idea is to pretrain models in completely unsupervised way and finally they can be fine-tuned for the task at hand using supervised learning. In this thesis, a general introduction to deep learning models and algorithms are given and these methods are applied to facial keypoints detection. The task is to predict the positions of 15 keypoints on grayscale face images. Each predicted keypoint is specified by an (x,y) real-valued pair in the space of pixel indices. In experiments, we pretrained deep belief networks (DBN) and finally performed a discriminative fine-tuning. We varied the depth and size of an architecture. We tested both deterministic and sampled hidden activations and the effect of additional unlabeled data on pretraining. The experimental results show that our model provides better results than publicly available benchmarks for the dataset.