Integrazione di algoritmi di visione basati su flusso ottico per il controllo di un velivolo senza pilota

Il lavoro svolto in questa tesi si colloca nell’area della robotica aerea e della visione artificiale attraverso l’integrazione di algoritmi di visione per il controllo di un velivolo senza pilota. Questo lavoro intende dare un contributo al progetto europeo SHERPA (Smart collaboration between Humans and ground-aErial Robots for imProving rescuing activities in Alpine environments), coordinato dall’università di Bologna e con la compartecipazione delle università di Brema, Zurigo, Twente, Leuven, Linkopings, del CREATE (Consorzio di Ricerca per l’Energia e le Applicazioni Tecnologiche dell’Elettromagnetismo), di alcune piccole e medie imprese e del club alpino italiano, che consiste nel realizzare un team di robots eterogenei in grado di collaborare con l’uomo per soccorrere i dispersi nell’ambiente alpino. L’obiettivo di SHERPA consiste nel progettare e integrare l’autopilota all’interno del team. In tale contesto andranno gestiti problemi di grande complessità, come il controllo della stabilità del velivolo a fronte di incertezze dovute alla presenza di vento, l’individuazione di ostacoli presenti nella traiettoria di volo, la gestione del volo in prossimità di ostacoli, ecc. Inoltre tutte queste operazioni devono essere svolte in tempo reale. La tesi è stata svolta presso il CASY (Center for Research on Complex Automated Systems) dell’università di Bologna, utilizzando per le prove sperimentali una PX4FLOW Smart Camera. Inizialmente è stato studiato un autopilota, il PIXHAWK, sul quale è possibile interfacciare la PX4FLOW, in seguito sono stati studiati e simulati in MATLAB alcuni algoritmi di visione basati su flusso ottico. Infine è stata studiata la PX4FLOW Smart Camera, con la quale sono state svolte le prove sperimentali. La PX4FLOW viene utilizzata come interfaccia alla PIXHAWK, in modo da eseguire il controllo del velivolo con la massima efficienza. E’ composta da una telecamera per la ripresa della scena, un giroscopio per la misura della velocità angolare, e da un sonar per le misure di distanza. E’ in grado di fornire la velocità di traslazione del velivolo, e quest’ultima, integrata, consente di ricostruire la traiettoria percorsa dal velivolo.

Sistema Multi Smart-Camera per la localizzazione e la guida di un drone

Con un sistema di multivisione correttamente installato e calibrato, si è tracciato il moto di un drone a seguito di operazioni di triangolazione e ne si è controllata automaticamente la traiettoria.

A socio-economic approach to online vision graph generation and handover in distributed smart camera networks

In this paper we propose an approach based on self-interested autonomous cameras, which exchange responsibility for tracking objects in a market mechanism, in order to maximise their own utility. A novel ant-colony inspired mechanism is used to grow the vision graph during runtime, which may then be used to optimise communication between cameras. The key benefits of our completely decentralised approach are on the one hand generating the vision graph online which permits the addition and removal cameras to the network during runtime and on the other hand relying only on local information, increasing the robustness of the system. Since our market-based approach does not rely on a priori topology information, the need for any multi-camera calibration can be avoided. © 2011 IEEE.

Socio-economic vision graph generation and handover in distributed smart camera networks

In this article we present an approach to object tracking handover in a network of smart cameras, based on self-interested autonomous agents, which exchange responsibility for tracking objects in a market mechanism, in order to maximise their own utility. A novel ant-colony inspired mechanism is used to learn the vision graph, that is, the camera neighbourhood relations, during runtime, which may then be used to optimise communication between cameras. The key benefits of our completely decentralised approach are on the one hand generating the vision graph online, enabling efficient deployment in unknown scenarios and camera network topologies, and on the other hand relying only on local information, increasing the robustness of the system. Since our market-based approach does not rely on a priori topology information, the need for any multicamera calibration can be avoided. We have evaluated our approach both in a simulation study and in network of real distributed smart cameras.

CamSim:a distributed smart camera network simulator

Smart cameras allow pre-processing of video data on the camera instead of sending it to a remote server for further analysis. Having a network of smart cameras allows various vision tasks to be processed in a distributed fashion. While cameras may have different tasks, we concentrate on distributed tracking in smart camera networks. This application introduces various highly interesting problems. Firstly, how can conflicting goals be satisfied such as cameras in the network try to track objects while also trying to keep communication overhead low? Secondly, how can cameras in the network self adapt in response to the behavior of objects and changes in scenarios, to ensure continued efficient performance? Thirdly, how can cameras organise themselves to improve the overall network's performance and efficiency? This paper presents a simulation environment, called CamSim, allowing distributed self-adaptation and self-organisation algorithms to be tested, without setting up a physical smart camera network. The simulation tool is written in Java and hence allows high portability between different operating systems. Relaxing various problems of computer vision and network communication enables a focus on implementing and testing new self-adaptation and self-organisation algorithms for cameras to use.

Learning to be different:heterogeneity and efficiency in distributed smart camera networks

In this paper we study the self-organising behaviour of smart camera networks which use market-based handover of object tracking responsibilities to achieve an efficient allocation of objects to cameras. Specifically, we compare previously known homogeneous configurations, when all cameras use the same marketing strategy, with heterogeneous configurations, when each camera makes use of its own, possibly different marketing strategy. Our first contribution is to establish that such heterogeneity of marketing strategies can lead to system wide outcomes which are Pareto superior when compared to those possible in homogeneous configurations. However, since the particular configuration required to lead to Pareto efficiency in a given scenario will not be known in advance, our second contribution is to show how online learning of marketing strategies at the individual camera level can lead to high performing heterogeneous configurations from the system point of view, extending the Pareto front when compared to the homogeneous case. Our third contribution is to show that in many cases, the dynamic behaviour resulting from online learning leads to global outcomes which extend the Pareto front even when compared to static heterogeneous configurations. Our evaluation considers results obtained from an open source simulation package as well as data from a network of real cameras. © 2013 IEEE.

Resource-aware configuration in smart camera networks

A recent trend in smart camera networks is that they are able to modify the functionality during runtime to better reflect changes in the observed scenes and in the specified monitoring tasks. In this paper we focus on different configuration methods for such networks. A configuration is given by three components: (i) a description of the camera nodes, (ii) a specification of the area of interest by means of observation points and the associated monitoring activities, and (iii) a description of the analysis tasks. We introduce centralized, distributed and proprioceptive configuration methods and compare their properties and performance. © 2012 IEEE.

Static, dynamic, and adaptive heterogeneity in distributed smart camera networks

We study heterogeneity among nodes in self-organizing smart camera networks, which use strategies based on social and economic knowledge to target communication activity efficiently. We compare homogeneous configurations, when cameras use the same strategy, with heterogeneous configurations, when cameras use different strategies. Our first contribution is to establish that static heterogeneity leads to new outcomes that are more efficient than those possible with homogeneity. Next, two forms of dynamic heterogeneity are investigated: nonadaptive mixed strategies and adaptive strategies, which learn online. Our second contribution is to show that mixed strategies offer Pareto efficiency consistently comparable with the most efficient static heterogeneous configurations. Since the particular configuration required for high Pareto efficiency in a scenario will not be known in advance, our third contribution is to show how decentralized online learning can lead to more efficient outcomes than the homogeneous case. In some cases, outcomes from online learning were more efficient than all other evaluated configuration types. Our fourth contribution is to show that online learning typically leads to outcomes more evenly spread over the objective space. Our results provide insight into the relationship between static, dynamic, and adaptive heterogeneity, suggesting that all have a key role in achieving efficient self-organization.

Distributed object tracking with robot and disjoint camera networks

We describe a novel two stage approach to object localization and tracking using a network of wireless cameras and a mobile robot. In the first stage, a robot travels through the camera network while updating its position in a global coordinate frame which it broadcasts to the cameras. The cameras use this information, along with image plane location of the robot, to compute a mapping from their image planes to the global coordinate frame. This is combined with an occupancy map generated by the robot during the mapping process to track the objects. We present results with a nine node indoor camera network to demonstrate that this approach is feasible and offers acceptable level of accuracy in terms of object locations.

Improved adaptivity and robustness in decentralised multi-camera networks

In this paper we present increased adaptivity and robustness in distributed object tracking by multi-camera networks using a socio-economic mechanism for learning the vision graph. To build-up the vision graph autonomously within a distributed smart-camera network, we use an ant-colony inspired mechanism, which exchanges responsibility for tracking objects using Vickrey auctions. Employing the learnt vision graph allows the system to optimise its communication continuously. Since distributed smart camera networks are prone to uncertainties in individual cameras, such as failures or changes in extrinsic parameters, the vision graph should be sufficiently robust and adaptable during runtime to enable seamless tracking and optimised communication. To better reflect real smart-camera platforms and networks, we consider that communication and handover are not instantaneous, and that cameras may be added, removed or their properties changed during runtime. Using our dynamic socio-economic approach, the network is able to continue tracking objects well, despite all these uncertainties, and in some cases even with improved performance. This demonstrates the adaptivity and robustness of our approach.

Self-aware and self-expressive camera networks

Smart cameras perform on-board image analysis, adapt their algorithms to changes in their environment, and collaborate with other networked cameras to analyze the dynamic behavior of objects. A proposed computational framework adopts the concepts of self-awareness and self-expression to more efficiently manage the complex tradeoffs among performance, flexibility, resources, and reliability. The Web extra at http://youtu.be/NKe31-OKLz4 is a video demonstrating CamSim, a smart camera simulation tool, enables users to test self-adaptive and self-organizing smart-camera techniques without deploying a smart-camera network.

Strumenti di guida ottica per un sistema di marcatura laser

Il percorso di tesi che ho intrapreso è stato svolto presso l'azienda Datalogic, con l'intento di integrare un sistema di visione ad un sistema di marcatura laser. L'utilizzo di questo potente strumento è però vincolato dalla particolare posizione fisica occupata, di volta in volta, dall'oggetto; per questo motivo viene fissato nella posizione desiderata, attraverso dime meccaniche. Fin ad ora si riteneva assolutamente necessaria la presenza di un operatore per il controllo del corretto posizionamento, tramite una simulazione della marcatura. Per ovviare a questo limite strutturale, Datalogic ha pensato di introdurre uno strumento di aiuto e di visione del processo: la camera. L'idea di base è stata quella di impiegare le moderne smart camera per individuare l'oggetto da marcare e rendere quindi il processo più automatico possibile. Per giungere a questo risultato è stato necessario effettuare una calibrazione del sistema totale: Camera più Laser. Il mio studio si è focalizzato quindi nel creare un eseguibile che aiutasse il cliente ad effettuare questa operazione nella maniera più semplice possibile. E' stato creato un eseguibile in C# che mettesse in comunicazione i due dispositivi ed eseguisse la calibrazione dei parametri intrinseci ed estrinseci. Il risultato finale ha permesso di avere il sistema di riferimento mondo della camera coincidente con quello del piano di marcatura del laser. Ne segue che al termine del processo di calibrazione se un oggetto verrà rilevato dalla camera, avente il baricentro nella posizione (10,10), il laser, utilizzando le medesime coordinate, marcherà proprio nel baricentro dell'oggetto desiderato. La maggiore difficoltà riscontrata è stata la differenza dei software che permettono la comunicazione con i due dispositivi e la creazione di una comunicazione con il laser, non esistente prima in C#.

Distributed low-power image processing in wireless sensor networks for intelligent video surveillance applications

Distributed Wireless Smart Camera (DWSC) network is a special type of Wireless Sensor Network (WSN) that processes captured images in a distributed manner. While image processing on DWSCs sees a great potential for growth, with its applications possessing a vast practical application domain such as security surveillance and health care, it suffers from tremendous constraints. In addition to the limitations of conventional WSNs, image processing on DWSCs requires more computational power, bandwidth and energy that presents significant challenges for large scale deployments. This dissertation has developed a number of algorithms that are highly scalable, portable, energy efficient and performance efficient, with considerations of practical constraints imposed by the hardware and the nature of WSN. More specifically, these algorithms tackle the problems of multi-object tracking and localisation in distributed wireless smart camera net- works and optimal camera configuration determination. Addressing the first problem of multi-object tracking and localisation requires solving a large array of sub-problems. The sub-problems that are discussed in this dissertation are calibration of internal parameters, multi-camera calibration for localisation and object handover for tracking. These topics have been covered extensively in computer vision literatures, however new algorithms must be invented to accommodate the various constraints introduced and required by the DWSC platform. A technique has been developed for the automatic calibration of low-cost cameras which are assumed to be restricted in their freedom of movement to either pan or tilt movements. Camera internal parameters, including focal length, principal point, lens distortion parameter and the angle and axis of rotation, can be recovered from a minimum set of two images of the camera, provided that the axis of rotation between the two images goes through the camera's optical centre and is parallel to either the vertical (panning) or horizontal (tilting) axis of the image. For object localisation, a novel approach has been developed for the calibration of a network of non-overlapping DWSCs in terms of their ground plane homographies, which can then be used for localising objects. In the proposed approach, a robot travels through the camera network while updating its position in a global coordinate frame, which it broadcasts to the cameras. The cameras use this, along with the image plane location of the robot, to compute a mapping from their image planes to the global coordinate frame. This is combined with an occupancy map generated by the robot during the mapping process to localised objects moving within the network. In addition, to deal with the problem of object handover between DWSCs of non-overlapping fields of view, a highly-scalable, distributed protocol has been designed. Cameras that follow the proposed protocol transmit object descriptions to a selected set of neighbours that are determined using a predictive forwarding strategy. The received descriptions are then matched at the subsequent camera on the object's path using a probability maximisation process with locally generated descriptions. The second problem of camera placement emerges naturally when these pervasive devices are put into real use. The locations, orientations, lens types etc. of the cameras must be chosen in a way that the utility of the network is maximised (e.g. maximum coverage) while user requirements are met. To deal with this, a statistical formulation of the problem of determining optimal camera configurations has been introduced and a Trans-Dimensional Simulated Annealing (TDSA) algorithm has been proposed to effectively solve the problem.

智能相机在激光焊缝跟踪检测系统中的应用研究

本文以中国科学院知识创新工程重要方向项目“全自动激光拼焊成套装备关键技术研究与示范应用”及沈阳市科技攻关项目“激光视觉焊缝自动跟踪与质量检测系统”为依托，针对激光焊接这个难点问题，在广泛调研国内外研究现状的基础上，研究开发了一套激光视觉焊缝跟踪检测原理样机。本文主要包括以下四方面的工作：1焊缝跟踪系统的系统结构搭建；2图像处理方法研究；3图像处理方法在FPGA中的实现；4基于工业机器人的激光焊接实验 及结果分析。具体工作如下： 本文首先论述了应用于焊缝跟踪的线结构光视觉传感器检测原理，建立了激光焊缝跟踪检测系统实验平台。该平台由图像采集与处理模块、上位机系统、DSP控制器、伺服电机驱动器、伺服电机等五部分组成。 激光拼焊焊缝跟踪图像的处理方法是关键技术之一，直接影响系统的实时性，根据激光拼焊焊缝跟踪图像的特点设计了相应的图像处理算法，分析研究了基于数学形态学的焊缝跟踪结构光条纹图像增强算法，并根据本课题的特点提出了一种基于模板的边缘提取方法，能简洁快速地提取出单像素边缘，然后研究了结构光中心线提取算法以及焊缝特征点识别算法，最后通过仿真实验验证了该图像处理流程的有效性。 论文的重点在于图像处理方法在智能相机中的实时实现。跟踪系统对图像处理的实时性要求很高，传统的处理方法主要是在DSP中以软件编程的方式实现，速度难以进一步提高，本课题中通过在智能相机中的FPGA中构建一个SOPC系统，将基于硬件描述语言VHDL完成的图像预处理模块和基于Xilinx公司的microblaze软核的特征点提取模块集成在单片芯片上，实现了激光条纹特征点的实时提取，系统具有高度的灵活性与出色的功能。 最后对搭建的跟踪系统平台进行了实验研究，用实验验证了焊缝跟踪系统的性能，保证了该套系统能够满足实时跟踪的要求，可以达到预期的设计目标。