Computer vision in aseptic pharmaceutical manufacturing: a deep learning-based approach to vial inspection and pack formation during a lyophilizer loading cycle

Vision systems are powerful tools playing an increasingly important role in modern industry, to detect errors and maintain product standards. With the enlarged availability of affordable industrial cameras, computer vision algorithms have been increasingly applied in industrial manufacturing processes monitoring. Until a few years ago, industrial computer vision applications relied only on ad-hoc algorithms designed for the specific object and acquisition setup being monitored, with a strong focus on co-designing the acquisition and processing pipeline. Deep learning has overcome these limits providing greater flexibility and faster re-configuration. In this work, the process to be inspected consists in vials’ pack formation entering a freeze-dryer, which is a common scenario in pharmaceutical active ingredient packaging lines. To ensure that the machine produces proper packs, a vision system is installed at the entrance of the freeze-dryer to detect eventual anomalies with execution times compatible with the production specifications. Other constraints come from sterility and safety standards required in pharmaceutical manufacturing. This work presents an overview about the production line, with particular focus on the vision system designed, and about all trials conducted to obtain the final performance. Transfer learning, alleviating the requirement for a large number of training data, combined with data augmentation methods, consisting in the generation of synthetic images, were used to effectively increase the performances while reducing the cost of data acquisition and annotation. The proposed vision algorithm is composed by two main subtasks, designed respectively to vials counting and discrepancy detection. The first one was trained on more than 23k vials (about 300 images) and tested on 5k more (about 75 images), whereas 60 training images and 52 testing images were used for the second one.

Classificazione di oggetti in immagini attraverso il modello Bag of Visual Words

Generic object recognition is an important function of the human visual system and everybody finds it highly useful in their everyday life. For an artificial vision system it is a really hard, complex and challenging task because instances of the same object category can generate very different images, depending of different variables such as illumination conditions, the pose of an object, the viewpoint of the camera, partial occlusions, and unrelated background clutter. The purpose of this thesis is to develop a system that is able to classify objects in 2D images based on the context, and identify to which category the object belongs to. Given an image, the system can classify it and decide the correct categorie of the object. Furthermore the objective of this thesis is also to test the performance and the precision of different supervised Machine Learning algorithms in this specific task of object image categorization. Through different experiments the implemented application reveals good categorization performances despite the difficulty of the problem. However this project is open to future improvement; it is possible to implement new algorithms that has not been invented yet or using other techniques to extract features to make the system more reliable. This application can be installed inside an embedded system and after trained (performed outside the system), so it can become able to classify objects in a real-time. The information given from a 3D stereocamera, developed inside the department of Computer Engineering of the University of Bologna, can be used to improve the accuracy of the classification task. The idea is to segment a single object in a scene using the depth given from a stereocamera and in this way make the classification more accurate.

Operazioni di assemblaggio automatizzato tramite robot antropomorfo e visione artificiale

Obiettivo dello studio condotto è l’implementazione di cicli di operazioni per l’assemblaggio automatizzato di componenti che costituiscono un sistema di trasporto a catena presente in alcune macchine automatiche. L’automazione del processo, fino ad oggi svolto manualmente, prevede l’utilizzo di un robot e, per il controllo di quest’ultimo, di un sistema di visione artificiale. L’attività di tirocinio associata alla tesi di laurea, che ha incluso una parte sperimentale oltre alla scrittura degli algoritmi di controllo del robot, è stata svolta all’interno del laboratorio TAILOR (Technology and Automation for Industry LabORatory) presso Siropack Italia S.r.l dove è presente una cella dotata di robot antropomorfo (Mitsubishi Electric) e di sistema di visione artificiale con camere 2D (Omron). La presenza di quest’ultimo è risultata strategica in termini di possibilità di adattare il montaggio anche a diversi posizionamenti degli oggetti all’interno dello spazio di lavoro, fermo restando che gli stessi risultassero appoggiati su una superficie piana. In primo luogo, affinché fosse garantita la ripetibilità del processo, sono state testate le prestazioni del sistema di visione tramite opportuna calibrazione della camera e del sistema di illuminazione ad esso collegata, al fine di ottenere un’acquisizione delle immagini che fosse sufficientemente robusta e risoluta mediante lo sfruttamento del software di elaborazione Omron FH Vision System. Un’opportuna programmazione della traiettoria del robot in ambiente di simulazione RT Toolbox 3, software integrato nel sistema di controllo del robot Mitsubishi Electric, ha infine consentito le regolari operazioni di assemblaggio, garantendo un processo affidabile ed, allo stesso tempo, adattabile ad ambienti eventualmente non strutturati in cui esso si trova ad operare.

Night Vision Imaging System (NVIS) certification requirements analysis of an Airbus Helicopters H135

The safe operation of nighttime flight missions would be enhanced using Night Vision Imaging Systems (NVIS) equipment. This has been clear to the military since 1970s and to the civil helicopters since 1990s. In these last months, even Italian Emergency Medical Service (EMS) operators require Night Vision Goggles (NVG) devices that therefore amplify the ambient light. In order to fly with this technology, helicopters have to be NVIS-approved. The author have supported a company, to quantify the potentiality of undertaking the certification activity, through a feasibility study. Even before, NVG description and working principles have been done, then specifications analysis about the processes to make a helicopter NVIS-approved has been addressed. The noteworthy difference between military specifications and the civilian ones highlights non-irrevelant lacks in the latter. The activity of NVIS certification could be a good investment because the following targets have been achieved: Reductions of the certification cost, of the operating time and of the number of non-compliance.

An Embedded In-Cabin Lightweight High-Performance 3D Gaze Estimation System

Gaze estimation has gained interest in recent years for being an important cue to obtain information about the internal cognitive state of humans. Regardless of whether it is the 3D gaze vector or the point of gaze (PoG), gaze estimation has been applied in various fields, such as: human robot interaction, augmented reality, medicine, aviation and automotive. In the latter field, as part of Advanced Driver-Assistance Systems (ADAS), it allows the development of cutting-edge systems capable of mitigating road accidents by monitoring driver distraction. Gaze estimation can be also used to enhance the driving experience, for instance, autonomous driving. It also can improve comfort with augmented reality components capable of being commanded by the driver's eyes. Although, several high-performance real-time inference works already exist, just a few are capable of working with only a RGB camera on computationally constrained devices, such as a microcontroller. This work aims to develop a low-cost, efficient and high-performance embedded system capable of estimating the driver's gaze using deep learning and a RGB camera. The proposed system has achieved near-SOTA performances with about 90% less memory footprint. The capabilities to generalize in unseen environments have been evaluated through a live demonstration, where high performance and near real-time inference were obtained using a webcam and a Raspberry Pi4.