Análise de regiões em cenários agrícolas utilizando um sistema de visão catadióptrico

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Vision-based vehicle detection and tracking with a mobile camera using a statistical framework

En esta tesis se aborda la detección y el seguimiento automático de vehículos mediante técnicas de visión artificial con una cámara monocular embarcada. Este problema ha suscitado un gran interés por parte de la industria automovilística y de la comunidad científica ya que supone el primer paso en aras de la ayuda a la conducción, la prevención de accidentes y, en última instancia, la conducción automática. A pesar de que se le ha dedicado mucho esfuerzo en los últimos años, de momento no se ha encontrado ninguna solución completamente satisfactoria y por lo tanto continúa siendo un tema de investigación abierto. Los principales problemas que plantean la detección y seguimiento mediante visión artificial son la gran variabilidad entre vehículos, un fondo que cambia dinámicamente debido al movimiento de la cámara, y la necesidad de operar en tiempo real. En este contexto, esta tesis propone un marco unificado para la detección y seguimiento de vehículos que afronta los problemas descritos mediante un enfoque estadístico. El marco se compone de tres grandes bloques, i.e., generación de hipótesis, verificación de hipótesis, y seguimiento de vehículos, que se llevan a cabo de manera secuencial. No obstante, se potencia el intercambio de información entre los diferentes bloques con objeto de obtener el máximo grado posible de adaptación a cambios en el entorno y de reducir el coste computacional. Para abordar la primera tarea de generación de hipótesis, se proponen dos métodos complementarios basados respectivamente en el análisis de la apariencia y la geometría de la escena. Para ello resulta especialmente interesante el uso de un dominio transformado en el que se elimina la perspectiva de la imagen original, puesto que este dominio permite una búsqueda rápida dentro de la imagen y por tanto una generación eficiente de hipótesis de localización de los vehículos. Los candidatos finales se obtienen por medio de un marco colaborativo entre el dominio original y el dominio transformado. Para la verificación de hipótesis se adopta un método de aprendizaje supervisado. Así, se evalúan algunos de los métodos de extracción de características más populares y se proponen nuevos descriptores con arreglo al conocimiento de la apariencia de los vehículos. Para evaluar la efectividad en la tarea de clasificación de estos descriptores, y dado que no existen bases de datos públicas que se adapten al problema descrito, se ha generado una nueva base de datos sobre la que se han realizado pruebas masivas. Finalmente, se presenta una metodología para la fusión de los diferentes clasificadores y se plantea una discusión sobre las combinaciones que ofrecen los mejores resultados. El núcleo del marco propuesto está constituido por un método Bayesiano de seguimiento basado en filtros de partículas. Se plantean contribuciones en los tres elementos fundamentales de estos filtros: el algoritmo de inferencia, el modelo dinámico y el modelo de observación. En concreto, se propone el uso de un método de muestreo basado en MCMC que evita el elevado coste computacional de los filtros de partículas tradicionales y por consiguiente permite que el modelado conjunto de múltiples vehículos sea computacionalmente viable. Por otra parte, el dominio transformado mencionado anteriormente permite la definición de un modelo dinámico de velocidad constante ya que se preserva el movimiento suave de los vehículos en autopistas. Por último, se propone un modelo de observación que integra diferentes características. En particular, además de la apariencia de los vehículos, el modelo tiene en cuenta también toda la información recibida de los bloques de procesamiento previos. El método propuesto se ejecuta en tiempo real en un ordenador de propósito general y da unos resultados sobresalientes en comparación con los métodos tradicionales. ABSTRACT This thesis addresses on-road vehicle detection and tracking with a monocular vision system. This problem has attracted the attention of the automotive industry and the research community as it is the first step for driver assistance and collision avoidance systems and for eventual autonomous driving. Although many effort has been devoted to address it in recent years, no satisfactory solution has yet been devised and thus it is an active research issue. The main challenges for vision-based vehicle detection and tracking are the high variability among vehicles, the dynamically changing background due to camera motion and the real-time processing requirement. In this thesis, a unified approach using statistical methods is presented for vehicle detection and tracking that tackles these issues. The approach is divided into three primary tasks, i.e., vehicle hypothesis generation, hypothesis verification, and vehicle tracking, which are performed sequentially. Nevertheless, the exchange of information between processing blocks is fostered so that the maximum degree of adaptation to changes in the environment can be achieved and the computational cost is alleviated. Two complementary strategies are proposed to address the first task, i.e., hypothesis generation, based respectively on appearance and geometry analysis. To this end, the use of a rectified domain in which the perspective is removed from the original image is especially interesting, as it allows for fast image scanning and coarse hypothesis generation. The final vehicle candidates are produced using a collaborative framework between the original and the rectified domains. A supervised classification strategy is adopted for the verification of the hypothesized vehicle locations. In particular, state-of-the-art methods for feature extraction are evaluated and new descriptors are proposed by exploiting the knowledge on vehicle appearance. Due to the lack of appropriate public databases, a new database is generated and the classification performance of the descriptors is extensively tested on it. Finally, a methodology for the fusion of the different classifiers is presented and the best combinations are discussed. The core of the proposed approach is a Bayesian tracking framework using particle filters. Contributions are made on its three key elements: the inference algorithm, the dynamic model and the observation model. In particular, the use of a Markov chain Monte Carlo method is proposed for sampling, which circumvents the exponential complexity increase of traditional particle filters thus making joint multiple vehicle tracking affordable. On the other hand, the aforementioned rectified domain allows for the definition of a constant-velocity dynamic model since it preserves the smooth motion of vehicles in highways. Finally, a multiple-cue observation model is proposed that not only accounts for vehicle appearance but also integrates the available information from the analysis in the previous blocks. The proposed approach is proven to run near real-time in a general purpose PC and to deliver outstanding results compared to traditional methods.

Catadioptric panoramic stereovision for humanoid robots

This paper proposes a novel design of a reconfigurable humanoid robot head, based on biological likeness of human being so that the humanoid robot could agreeably interact with people in various everyday tasks. The proposed humanoid head has a modular and adaptive structural design and is equipped with three main components: frame, neck motion system and omnidirectional stereovision system modules. The omnidirectional stereovision system module being the last module, a motivating contribution with regard to other computer vision systems implemented in former humanoids, it opens new research possibilities for achieving human-like behaviour. A proposal for a real-time catadioptric stereovision system is presented, including stereo geometry for rectifying the system configuration and depth estimation. The methodology for an initial approach for visual servoing tasks is divided into two phases, first related to the robust detection of moving objects, their depth estimation and position calculation, and second the development of attention-based control strategies. Perception capabilities provided allow the extraction of 3D information from a wide range of visions from uncontrolled dynamic environments, and work results are illustrated through a number of experiments.

Vibration reduction for vision systems on board unmanned aerial vehicles using a neuro-fuzzy controller

In this paper, an intelligent control approach based on neuro-fuzzy systems performance is presented, with the objective of counteracting the vibrations that affect the low-cost vision platform onboard an unmanned aerial system of rotating nature. A scaled dynamical model of a helicopter is used to simulate vibrations on its fuselage. The impact of these vibrations on the low-cost vision system will be assessed and an intelligent control approach will be derived in order to reduce its detrimental influence. Different trials that consider a neuro-fuzzy approach as a fundamental part of an intelligent semi-active control strategy have been carried out. Satisfactory results have been achieved compared to those obtained by means of vibration reduction passive techniques.

Automatic expert system based on images for accuracy crop row detection in maize fields

This paper proposes an automatic expert system for accuracy crop row detection in maize fields based on images acquired from a vision system. Different applications in maize, particularly those based on site specific treatments, require the identification of the crop rows. The vision system is designed with a defined geometry and installed onboard a mobile agricultural vehicle, i.e. submitted to vibrations, gyros or uncontrolled movements. Crop rows can be estimated by applying geometrical parameters under image perspective projection. Because of the above undesired effects, most often, the estimation results inaccurate as compared to the real crop rows. The proposed expert system exploits the human knowledge which is mapped into two modules based on image processing techniques. The first one is intended for separating green plants (crops and weeds) from the rest (soil, stones and others). The second one is based on the system geometry where the expected crop lines are mapped onto the image and then a correction is applied through the well-tested and robust Theil–Sen estimator in order to adjust them to the real ones. Its performance is favorably compared against the classical Pearson product–moment correlation coefficient.

Experiments in outdoor operation of RatSLAM

This paper shows initial results in deploying the biologically inspired Simultaneous Localisation and Mapping system, RatSLAM, in an outdoor environment. RatSLAM has been widely tested in indoor environments on the task of producing topologically coherent maps based on a fusion of odometric and visual information. This paper details the changes required to deploy RatSLAM on a small tractor equipped with odometry and an omnidirectional camera. The principal changes relate to the vision system, with others required for RatSLAM to use omnidirectional visual data. The initial results from mapping around a 500 m loop are promising, with many improvements still to be made.

Helicopter automation using a low-cost sensing system

This paper details the design of an autonomous helicopter control system using a low cost sensor suite. Control is maintained using simple nested PID loops. Aircraft attitude, velocity, and height is estimated using an in-house designed IMU and vision system. Information is combined using complimentary filtering. The aircraft is shown to be stabilised and responding to high level demands on all axes, including heading, height, lateral velocity and longitudinal velocity.

Real-time tracking system using C80 DSPs and a binocular robotic head

[EN]An active vision system to perform tracking of moving objects in real time is described. The main goal is to obtain a system integrating off-the-self components. These components includes a stereoscopic robotic-head, as active perception hardware; a DSP based board SDB C80, as massive data processor and image acquisition board; and finally, a Pentium PC running Windows NT that interconnects and manages the whole system. Real-time is achieved taking advantage of the special architecture of DSP. An evaluation of the performance is included.

Design and Pilot Study for an Efficient High-Throughput Automated Computer-Vision Guided Intelligent De-Calyxing Machine for Post-Harvest Strawberry Processing

Strawberries harvested for processing as frozen fruits are currently de-calyxed manually in the field. This process requires the removal of the stem cap with green leaves (i.e. the calyx) and incurs many disadvantages when performed by hand. Not only does it necessitate the need to maintain cutting tool sanitation, but it also increases labor time and exposure of the de-capped strawberries before in-plant processing. This leads to labor inefficiency and decreased harvest yield. By moving the calyx removal process from the fields to the processing plants, this new practice would reduce field labor and improve management and logistics, while increasing annual yield. As labor prices continue to increase, the strawberry industry has shown great interest in the development and implementation of an automated calyx removal system. In response, this dissertation describes the design, operation, and performance of a full-scale automatic vision-guided intelligent de-calyxing (AVID) prototype machine. The AVID machine utilizes commercially available equipment to produce a relatively low cost automated de-calyxing system that can be retrofitted into existing food processing facilities. This dissertation is broken up into five sections. The first two sections include a machine overview and a 12-week processing plant pilot study. Results of the pilot study indicate the AVID machine is able to de-calyx grade-1-with-cap conical strawberries at roughly 66 percent output weight yield at a throughput of 10,000 pounds per hour. The remaining three sections describe in detail the three main components of the machine: a strawberry loading and orientation conveyor, a machine vision system for calyx identification, and a synchronized multi-waterjet knife calyx removal system. In short, the loading system utilizes rotational energy to orient conical strawberries. The machine vision system determines cut locations through RGB real-time feature extraction. The high-speed multi-waterjet knife system uses direct drive actuation to locate 30,000 psi cutting streams to precise coordinates for calyx removal. Based on the observations and studies performed within this dissertation, the AVID machine is seen to be a viable option for automated high-throughput strawberry calyx removal. A summary of future tasks and further improvements is discussed at the end.

Computer vision in aseptic pharmaceutical manufacturing: a deep learning-based approach to vial inspection and pack formation during a lyophilizer loading cycle

Vision systems are powerful tools playing an increasingly important role in modern industry, to detect errors and maintain product standards. With the enlarged availability of affordable industrial cameras, computer vision algorithms have been increasingly applied in industrial manufacturing processes monitoring. Until a few years ago, industrial computer vision applications relied only on ad-hoc algorithms designed for the specific object and acquisition setup being monitored, with a strong focus on co-designing the acquisition and processing pipeline. Deep learning has overcome these limits providing greater flexibility and faster re-configuration. In this work, the process to be inspected consists in vials’ pack formation entering a freeze-dryer, which is a common scenario in pharmaceutical active ingredient packaging lines. To ensure that the machine produces proper packs, a vision system is installed at the entrance of the freeze-dryer to detect eventual anomalies with execution times compatible with the production specifications. Other constraints come from sterility and safety standards required in pharmaceutical manufacturing. This work presents an overview about the production line, with particular focus on the vision system designed, and about all trials conducted to obtain the final performance. Transfer learning, alleviating the requirement for a large number of training data, combined with data augmentation methods, consisting in the generation of synthetic images, were used to effectively increase the performances while reducing the cost of data acquisition and annotation. The proposed vision algorithm is composed by two main subtasks, designed respectively to vials counting and discrepancy detection. The first one was trained on more than 23k vials (about 300 images) and tested on 5k more (about 75 images), whereas 60 training images and 52 testing images were used for the second one.

Communication and camouflage with the same 'bright' colours in reef fishes

Reef fishes present the observer with the most diverse and stunning assemblage of animal colours anywhere on earth. The functions of some of these colours and their combinations are examined using new non-subjective spectrophotometer ic measurements of the colours of fishes and their habitat. Conclusions reached are as follows: (i) the spectra of colours in high spatial frequency patterns are often well designed to be very conspicuous to a colour vision system at close range but well camouflaged at a distance; (ii) blue and yellow the most frequently used colours in reef fishes, may be good for camouflage or communication depending on the background they are viewed against; and (iii) reef fishes use a combination of colour and behaviour to regulate their conspicuousness and crypsis.

Aprendizagem por imitação através de mapeamento Visuomotor baseado em imagens omnidirecionais

Este trabalho propõe uma metodologia de aprendizagem que permite a um robô aprender uma tarefa adaptando-a e representando-a de acordo com a sua capacidade motora e sensorial. Primeiramente, um mapeamento sensoriomotor é criado e converte informação sensorial em informação motora. Depois, através de imitação, o robô aprende um conjunto de ações elementares formando um vocabulário motor. A imitação é baseada nas representações motoras obtidas com o mapeamento sensoriomotor. O vocabulário motor criado é então utilizado para aprender e realizar tarefas mais sofisticadas, compostas por seqüências ou combinações de ações elementares. Esta metodologia é ilustrada através de uma aplicação de mapeamento e navegação topológica com um robô móvel. O automovimento é utilizado como mapeamento visuomotor, convertendo o fluxo óptico em imagens omnidirecionais em informação motora (translação e rotação), a qual é usada para a criação de um vocabulário motor. A seguir, o vocabulário é utilizado para mapeamento e navegação topológica. Os resultados obtidos são interessantes e a abordagem proposta pode ser estendida a diferentes robôs e aplicações.

Sistema de visão para percepção e navegação de um UAV Quadrotor

Mestrado em Engenharia Electrotécnica e de Computadores

Sistema de Visão Stereo Ativo aplicado aos robôs do ISePorto

A presente dissertação endereça o desenvolvimento de um sistema de visão stereo ativo para os robôs de futebol robótico da equipa ISePorto do ISEP, de modo a que estes tirem o máximo partido das câmaras rotativas neles existentes. Este trabalho surge da necessidade de melhorar a capacidade de perceção do ambiente por parte dos robôs, principalmente da perceção da bola quando não está no plano do campo e dos robôs adversários. Esta necessidade surge devido ao aumento da dinâmica que se tem vindo a veri car ultimamente nas competições. Para tal, foram estudados algumas trabalhos relacionados no que diz respeito a sistemas de visão stereo com baselines variáveis e eixos de rotação em ambas as câmaras, bem como fundamentos de visão stereo. Foi proposta uma arquitetura para o sistema de visão ativo de modo a ser aplicado em qualquer robô da equipa MSL (Middle Size League). Para tornar possível a implementação desta arquitetura foi desenvolvido um procedimento para a calibração e determinação em tempo real dos parâmetros extrínsecos do par stereo em função da posição angular dos eixos rotativos do robô. O sistema de visão foi também dotado de capacidade de sincronismo e foram implementadas funcionalidades ao nível de software que possibilitam a deteção de objetos na imagem, a correspondência de objetos presentes nas imagens de ambas as câmaras e consequentemente a determinação das posições tridimensionais desses objetos relativamente ao robô. O sistema desenvolvido foi testado e validado em cenário MSL ao nível de perceção da bola, robôs adversários e linhas do campo. Os resultados obtidos apresentam uma melhoria signi cativa, face à implementação atual dos robôs, na perceção tridimensional da bola quando não está no plano do campo, e dos robôs adversários.

Sistema de visão para aterragem automática de UAV

Dissertação para obtenção do grau de Mestre em Engenharia Electrotécnica Ramo de Automação e Electrónica Industrial