Visual Relationship Detection

L'elaborato consiste in uno studio dello stato dell'arte della visual relationship detection. Partendo da una breve introduzione riguardante l'object detection e i privi lavori in cui sono state utilizzate le relazioni verranno affrontati i principali metodi utilizzati per risolvere il problema.

Sviluppo di un algoritmo di Object Detection per la privacy in ambito industriale

In questa tesi è descritto il lavoro svolto presso un'azienda informatica locale, allo scopo di ricerca ed implementazione di un algoritmo per individuare ed offuscare i volti presenti all'interno di video di e-learning in ambito industriale, al fine di garantire la privacy degli operai presenti. Tale algoritmo sarebbe stato poi da includere in un modulo software da inserire all'interno di un applicazione web già esistente per la gestione di questi video. Si è ricercata una soluzione ad hoc considerando le caratteristiche particolare del problema in questione, studiando le principali tecniche della Computer Vision per comprendere meglio quale strada percorrere. Si è deciso quindi di implementare un algoritmo di Blob Tracking basato sul colore.

Texting and Driving: rilevazione di utilizzo dello smartphone alla guida tramite Object Detection e Machine Learning

Questa tesi si ispira a lavori precedentemente portati avanti da altri studenti e si pone il problema della possibilit\`a di riconoscere se uno smartphone \`e utilizzato da un utente mentre esso si trova alla guida di un'autovettura. In essa verranno presentati vari metodi per risolvere questo problema di Machine Learning, ovvero realizzazione di dataset per l'allenamento di modelli e creazione e allenamento di modelli stessi, dediti al riconoscimento di un problema di classificazione binaria e riconoscimento di oggetti tramite Object Detection. Il cercare di riconoscere se l'utente \`e alla guida o meno, avverr\`a tramite l'output della fotocamera frontale dello smartphone, quindi lavoreremo su immagini, video e frame. Arriveremo a riconoscere la posizione della persona rappresentata da questi fotogrammi tramite un modello di Object Detection, che riconosce cintura e finestrino e determina se sono appartenenti al sedile e alla posizione del conducente o del passeggero. Vedremo alla fine, attraverso un'attenta analisi dei risultati ottenuti su ben 8 video diversi che saranno divisi in molti frame, che si ottengono risultati molto interessanti, dai quali si pu\`o prendere spunto per la creazione di un importante sistema di sicurezza alla guida.

Dataset Fusion application on deep learning Object Detection task

Application of dataset fusion techniques to an object detection task, involving the use of deep learning as convolutional neural networks, to manage to create a single RCNN architecture able to inference with good performances on two distinct datasets with different domains.

Embedding Video Object Detection Capabilities on Low-Power Nano-Drones

Our objective for this thesis work was the deployment of a Neural Network based approach for video object detection on board a nano-drone. Furthermore, we have studied some possible extensions to exploit the temporal nature of videos to improve the detection capabilities of our algorithm. For our project, we have utilized the Mobilenetv2/v3SSDLite due to their limited computational and memory requirements. We have trained our networks on the IMAGENET VID 2015 dataset and to deploy it onto the nano-drone we have used the NNtool and Autotiler tools by GreenWaves. To exploit the temporal nature of video data we have tried different approaches: the introduction of an LSTM based convolutional layer in our architecture, the introduction of a Kalman filter based tracker as a postprocessing step to augment the results of our base architecture. We have obtain a total improvement in our performances of about 2.5 mAP with the Kalman filter based method(BYTE). Our detector run on a microcontroller class processor on board the nano-drone at 1.63 fps.

Production Lot Detection through Synthetic Generated Data

Correctness of information gathered in production environments is an essential part of quality assurance processes in many industries, this task is often performed by human resources who visually take annotations in various steps of the production flow. Depending on the performed task the correlation between where exactly the information is gathered and what it represents is more than often lost in the process. The lack of labeled data places a great boundary on the application of deep neural networks aimed at object detection tasks, moreover supervised training of deep models requires a great amount of data to be available. Reaching an adequate large collection of labeled images through classic techniques of data annotations is an exhausting and costly task to perform, not always suitable for every scenario. A possible solution is to generate synthetic data that replicates the real one and use it to fine-tune a deep neural network trained on one or more source domains to a different target domain. The purpose of this thesis is to show a real case scenario where the provided data were both in great scarcity and missing the required annotations. Sequentially a possible approach is presented where synthetic data has been generated to address those issues while standing as a training base of deep neural networks for object detection, capable of working on images taken in production-like environments. Lastly, it compares performance on different types of synthetic data and convolutional neural networks used as backbones for the model.

Image Analysis Methods for Sugar Beet Phenotyping

In un mondo che richiede sempre maggiormente un'automazione delle attività della catena produttiva industriale, la computer vision rappresenta uno strumento fondamentale perciò che viene già riconosciuta internazionalmente come la Quarta Rivoluzione Industriale o Industry 4.0. Avvalendomi di questo strumento ho intrapreso presso l'azienda Syngenta lo studio della problematica della conta automatica del numero di foglie di una pianta. Il problema è stato affrontato utilizzando due differenti approcci, ispirandosi alla letteratura. All'interno dell'elaborato è presente anche la descrizione progettuale di un ulteriore metodo, ad oggi non presente in letteratura. Le metodologie saranno spiegate in dettaglio ed i risultati ottenuti saranno confrontati utilizzando i primi due approcci. Nel capitolo finale si trarranno le conclusioni sulle basi dei risultati ottenuti e dall'analisi degli stessi.

Deep learning models for the automatic analysis of volleyball plays

Collecting and analysing data is an important element in any field of human activity and research. Even in sports, collecting and analyzing statistical data is attracting a growing interest. Some exemplar use cases are: improvement of technical/tactical aspects for team coaches, definition of game strategies based on the opposite team play or evaluation of the performance of players. Other advantages are related to taking more precise and impartial judgment in referee decisions: a wrong decision can change the outcomes of important matches. Finally, it can be useful to provide better representations and graphic effects that make the game more engaging for the audience during the match. Nowadays it is possible to delegate this type of task to automatic software systems that can use cameras or even hardware sensors to collect images or data and process them. One of the most efficient methods to collect data is to process the video images of the sporting event through mixed techniques concerning machine learning applied to computer vision. As in other domains in which computer vision can be applied, the main tasks in sports are related to object detection, player tracking, and to the pose estimation of athletes. The goal of the present thesis is to apply different models of CNNs to analyze volleyball matches. Starting from video frames of a volleyball match, we reproduce a bird's eye view of the playing court where all the players are projected, reporting also for each player the type of action she/he is performing.

Tecniche di deep learning per il monitoraggio del processo nell’industria dello pneumatico

Il seguente elaborato affronta l'implementazione di un algoritmo che affronta un problema di controllo di processo in ambito industriale utilizzando algoritmi di object detection. Infatti, il progetto concordato con il professore Di Stefano si è svolto in collaborazione con l’azienda Pirelli, nell’ambito della produzione di pneumatici. Lo scopo dell'algoritmo implementato è di verificare il preciso orientamento di elementi grafici della copertura, utilizzati dalle case automobilistiche per equipaggiare correttamente le vetture. In particolare, si devono individuare delle scritte sul battistrada della copertura e identificarne la posizione rispetto ad altri elementi fissati su di essa. La tesi affronta questo task in due parti distinte: la prima consiste nel training di algoritmi di deep learning per il riconoscimento degli elementi grafici e del battistrada, la seconda è un decisore che opera a valle del primo sistema utilizzando gli output delle reti allenate.

Collection and real-time analysis of images from aircraft to support civil protection operational activities

This report describes the realization of a system, in which an object detection model will be implemented, whose aim is to detect the presence of people in images. This system could be used for several applications: for example, it could be carried on board an aircraft or a drone. In this case, the system is designed in such a way that it can be mounted on light/medium weight helicopters, helping the operator to find people in emergency situations. In the first chapter the use of helicopters for civil protection is analysed and applications similar to this case study are listed. The second chapter describes the choice of the hardware devices that have been used to implement a prototype of a system to collect, analyse and display images. At first, the PC necessary to process the images was chosen, based on the characteristics of the algorithms that are necessary to run the analysis. In the further, a camera that could be compatible with the PC was selected. Finally, the battery pack was chosen taking into account the electrical consumption of the devices. The third chapter illustrates the algorithms used for image analysis. In the fourth, some of the requirements listed in the regulations that must be taken into account for carrying on board all the devices have been briefly analysed. In the fifth chapter the activity of design and modelling, with the CAD Solidworks, the devices and a prototype of a case that will house them is described. The sixth chapter discusses the additive manufacturing, since the case was printed exploiting this technology. In the seventh chapter, part of the tests that must be carried out on the equipment to certificate it have been analysed, and some simulations have been carried out. In the eighth chapter the results obtained once loaded the object detection model on a hardware for image analyses were showed. In the ninth chapter, conclusions and future applications were discussed.

Analisi di video con visione binoculare

Nel TCR - Termina container Ravenna, è importante che nel momento di scarico del container sul camion non siano presenti persone nell’area. In questo elaborato si descrive la realizzazione e il funzionamento di un sistema di allarme automatico, in grado di rilevare persone ed eventualmente interrompere la procedura di scarico del container. Tale sistema si basa sulla tecnica della object segmentation tramite rimozione dello sfondo, a cui viene affiancata una classificazione e rimozione delle eventuali ombre con un metodo cromatico. Inoltre viene identificata la possibile testa di una persona e avendo a disposizione due telecamere, si mette in atto una visione binoculare per calcolarne l’altezza. Infine, viene presa in considerazione anche la dinamica del sistema, per cui la classificazione di una persona si può basare sulla grandezza, altezza e velocità dell’oggetto individuato.

Robotic grasping: State of the art and modular real-world implementation

Robotic Grasping is an important research topic in robotics since for robots to attain more general-purpose utility, grasping is a necessary skill, but very challenging to master. In general the robots may use their perception abilities like an image from a camera to identify grasps for a given object usually unknown. A grasp describes how a robotic end-effector need to be positioned to securely grab an object and successfully lift it without lost it, at the moment state of the arts solutions are still far behind humans. In the last 5–10 years, deep learning methods take the scene to overcome classical problem like the arduous and time-consuming approach to form a task-specific algorithm analytically. In this thesis are present the progress and the approaches in the robotic grasping field and the potential of the deep learning methods in robotic grasping. Based on that, an implementation of a Convolutional Neural Network (CNN) as a starting point for generation of a grasp pose from camera view has been implemented inside a ROS environment. The developed technologies have been integrated into a pick-and-place application for a Panda robot from Franka Emika. The application includes various features related to object detection and selection. Additionally, the features have been kept as generic as possible to allow for easy replacement or removal if needed, without losing time for improvement or new testing.

Analysis of human posterior EEG alpha frequency as a function of implicit learning of probabilistic lateralized stimuli in a visual detection task

Alpha oscillatory activity has long been associated with perceptual and cognitive processes related to attention control. The aim of this study is to explore the task-dependent role of alpha frequency in a lateralized visuo-spatial detection task. Specifically, the thesis focuses on consolidating the scientific literature's knowledge about the role of alpha frequency in perceptual accuracy, and deepening the understanding of what determines trial-by-trial fluctuations of alpha parameters and how these fluctuations influence overall task performance. The hypotheses, confirmed empirically, were that different implicit strategies are put in place based on the task context, in order to maximize performance with optimal resource distribution (namely alpha frequency, associated positively with performance): “Lateralization” of the attentive resources towards one hemifield should be associated with higher alpha frequency difference between contralateral and ipsilateral hemisphere; “Distribution” of the attentive resources across hemifields should be associated with lower alpha frequency difference between hemispheres; These strategies, used by the participants according to their brain capabilities, have proven themselves adaptive or maladaptive depending on the different tasks to which they have been set: "Distribution" of the attentive resources seemed to be the best strategy when the distribution probability between hemifields was balanced: i.e. the neutral condition task. "Lateralization" of the attentive resources seemed to be more effective when the distribution probability between hemifields was biased towards one hemifield: i.e., the biased condition task.

Classificazione di oggetti in immagini attraverso il modello Bag of Visual Words

Generic object recognition is an important function of the human visual system and everybody finds it highly useful in their everyday life. For an artificial vision system it is a really hard, complex and challenging task because instances of the same object category can generate very different images, depending of different variables such as illumination conditions, the pose of an object, the viewpoint of the camera, partial occlusions, and unrelated background clutter. The purpose of this thesis is to develop a system that is able to classify objects in 2D images based on the context, and identify to which category the object belongs to. Given an image, the system can classify it and decide the correct categorie of the object. Furthermore the objective of this thesis is also to test the performance and the precision of different supervised Machine Learning algorithms in this specific task of object image categorization. Through different experiments the implemented application reveals good categorization performances despite the difficulty of the problem. However this project is open to future improvement; it is possible to implement new algorithms that has not been invented yet or using other techniques to extract features to make the system more reliable. This application can be installed inside an embedded system and after trained (performed outside the system), so it can become able to classify objects in a real-time. The information given from a 3D stereocamera, developed inside the department of Computer Engineering of the University of Bologna, can be used to improve the accuracy of the classification task. The idea is to segment a single object in a scene using the depth given from a stereocamera and in this way make the classification more accurate.

Surface-enhanced vibrational techniques for the detection of synthetic organic colorants

The research project object of this thesis is focused on the development of an advanced analytical system based on the combination of an improved thin layer chromatography (TLC) plate coupled with infrared (FTIR) and Raman microscopies for the detection of synthetic dyes. Indeed, the characterization of organic colorants, which are commonly present in mixtures with other components and in a very limited amount, still represents a challenging task in scientific analyses of cultural heritage materials. The approach provides selective spectral fingerprints for each compound, foreseeing the complementary information obtained by micro ATR-RAIRS-FTIR and SERS-Raman analyses, which can be performed on the same separated spot. In particular, silver iodide (AgI) applied on a gold coated slide is proposed as an efficient stationary phase for the discrimination of complex analyte mixtures, such as dyes present in samples of art-historical interest. The gold-AgI-TLC plate shows high performances related both to the chromatographic separation of analytes and to the spectroscopic detection of components. The use of a mid-IR transparent inorganic salt as the stationary phase avoids interferences of the background absorption in FTIR investigations. Moreover, by ATR microscopy measurements performed on the gold-AgI surface, a considerable enhancement in the intensity of spectra is observed. Complementary information can be obtained by Raman analyses, foreseeing a SERS activity of the AgI substrate. The method has been tested for the characterization of a mixture of three synthetic organic colorants widely used in dyeing processes: Brilliant Green (BG1), Rhodamine B (BV10) and Methylene Blue (BB9).