Minimum Mean Square Error Approach in Images Processing for Full-field Displacements and Deformation Measurements

A vision system is applied to full-field displacements and deformation measurements in solid mechanics. A speckle like pattern is preliminary formed on the surface under investigation. To determine displacements field of one speckle image with respect to a reference speckle image, sub-images, referred to Zones Of Interest (ZOI) are considered. The field is obtained by matching a ZOI in the reference image with the respective ZOI in the moved image. Two image processing techniques are used for implementing the matching procedure: – cross correlation function and minimum mean square error (MMSE) of the ZOI intensity distribution. The two algorithms are compared and the influence of the ZOI size on the accuracy of measurements is studied.

Field Testing and Development of Novel SHM systems at Queen’s University Belfast

Ageing and deterioration of infrastructure is a challenge facing transport authorities. In
particular, there is a need for increased bridge monitoring in order to provide adequate
maintenance and to guarantee acceptable levels of transport safety. The Intelligent
Infrastructure group at Queens University Belfast (QUB) are working on a number of aspects
of infrastructure monitoring and this paper presents summarised results from three distinct
monitoring projects carried out by this group. Firstly the findings from a project on next
generation Bridge Weight in Motion (B-WIM) are reported, this includes full scale field testing
using fibre optic strain sensors. Secondly, results from early phase testing of a computer
vision system for bridge deflection monitoring are reported on. This research seeks to exploit
recent advances in image processing technology with a view to developing contactless
bridge monitoring approaches. Considering the logistical difficulty of installing sensors on a
‘live’ bridge, contactless monitoring has some inherent advantages over conventional
contact based sensing systems. Finally the last section of the paper presents some recent
findings on drive by bridge monitoring. In practice a drive-by monitoring system will likely
require GPS to allow the response of a given bridge to be identified; this study looks at the
feasibility of using low-cost GPS sensors for this purpose, via field trials. The three topics
outlined above cover a spectrum of SHM approaches namely, wired monitoring, contactless
monitoring and drive by monitoring

Before characterizing faces

[EN]This paper describes the approach for face detection and selection of frontal views, for further processing. This approach based on color detection and symmetry operator application, is integrated in an Active Vision System o ering promising results just making use of some opportunistic skills.

Uma arquitetura distribuída de hardware e software para controle de um robô móvel autônomo

BRITTO, Ricardo S.; MEDEIROS, Adelardo A. D.; ALSINA, Pablo J. Uma arquitetura distribuída de hardware e software para controle de um robô móvel autônomo. In: SIMPÓSIO BRASILEIRO DE AUTOMAÇÃO INTELIGENTE,8., 2007, Florianópolis. Anais... Florianópolis: SBAI, 2007.

Detección y evitación de objetos flotantes en el contexto de un vehículo autónomo de superficie

[ES]El Instituto Universitario de Sistemas Inteligentes y Aplicaciones Numéricas en Ingeniería y en especial la División de Robótica y Oceanografía Computacional está desarrollando un velero autónomo de superficie que requiere de un sistema para la detección y evasión de obstáculos. Dicho sistema se ha desarrollado sobre una Raspberry Pi con un servicio para la captura de imágenes, así como un servidor web que permita la modificación de la configuración de la cámara. Una vez completada dicha infraestructura se tomaron las fotografías que conformarán el conjunto de entrenamiento para el sistema de visión por computador y se desarrollará este último. Los resultados se han integrado con el sistema del control modificando el rumbo cuando se detecte un obstáculo.

Uma arquitetura distribuída de hardware e software para controle de um robô móvel autônomo

BRITTO, Ricardo S.; MEDEIROS, Adelardo A. D.; ALSINA, Pablo J. Uma arquitetura distribuída de hardware e software para controle de um robô móvel autônomo. In: SIMPÓSIO BRASILEIRO DE AUTOMAÇÃO INTELIGENTE,8., 2007, Florianópolis. Anais... Florianópolis: SBAI, 2007.

Comparação da medida de cor de pescado entre sistema de visão computacional e colorímetro convencional no frescor do pescado

A cor da superfície dos alimentos é o primeiro parâmetro de qualidade avaliado pelos consumidores, e é critico para a aceitação do produto, então a medição adequada da cor é uma importante ferramenta. Nesta pesquisa avaliou-se a variação da cor em corvina (Micropogonias furnieri) armazenada em gelo durante 16 dias; os parâmetros de luminosidade (L*), valor cromático a*, valor cromático b*, variação total da cor (ΔE) e croma (C*) foram obtidos por sistema de visão computacional, e por colorímetro Konica Minolta CR-400. O frescor da corvina baseada nas mudanças da cor das brânquias foi avaliado utilizando um sistema de visão computacional. Também se modelou a oxidação da mioglobina em files de burriquete (Pogonias cromis), utilizando os parâmetros de vermelho (valor a* e R). Para registrar as mudanças da cor durante 57,6 h utilizou-se um sistema de visão computacional, a análise química realizou-se determinando a concentração de metamioglobina (%). Na avaliação da cor de corvina armazenada em gelo, o sistema de visão computacional mostrou diferenças significativas para L*, a*, ΔE e C*, enquanto que o colorímetro mostrou diferenças significativas para L* e ΔE, o único parâmetro que não apresentou diferenças entre instrumentos foi ΔE durante a avaliação da corvina armazenada em gelo. O coeficiente de correlação entre os parâmetros da cor (L*, a* e b*) das brânquias da corvina armazenada em gelo pelo tempo de armazenamento foi de 0,9747. O sistema de visão computacional registrou as mudanças da cor em filés de burriquete e se modelaram as mudanças utilizando um modelo exponencial. O sistema de visão computacional mostrou ser mais sensível às mudanças da cor durante a avaliação da cor na corvina armazenada em gelo. É possível prognosticar o tempo de armazenamento da corvina em gelo em função da mudança da cor das brânquias. Assim, foi possível modelar a variação da mioglobina em filés de burriquete utilizando sistemas de visão computacional para registrar ditas mudanças. Os sistemas de visão computacional têm grande capacidade para registrar as mudanças da cor e é possível utiliza-los para avaliar os alimentos em função da cor.

Integrated visual perception architecture for robotic clothes perception and manipulation

This thesis proposes a generic visual perception architecture for robotic clothes perception and manipulation. This proposed architecture is fully integrated with a stereo vision system and a dual-arm robot and is able to perform a number of autonomous laundering tasks. Clothes perception and manipulation is a novel research topic in robotics and has experienced rapid development in recent years. Compared to the task of perceiving and manipulating rigid objects, clothes perception and manipulation poses a greater challenge. This can be attributed to two reasons: firstly, deformable clothing requires precise (high-acuity) visual perception and dexterous manipulation; secondly, as clothing approximates a non-rigid 2-manifold in 3-space, that can adopt a quasi-infinite configuration space, the potential variability in the appearance of clothing items makes them difficult to understand, identify uniquely, and interact with by machine. From an applications perspective, and as part of EU CloPeMa project, the integrated visual perception architecture refines a pre-existing clothing manipulation pipeline by completing pre-wash clothes (category) sorting (using single-shot or interactive perception for garment categorisation and manipulation) and post-wash dual-arm flattening. To the best of the author’s knowledge, as investigated in this thesis, the autonomous clothing perception and manipulation solutions presented here were first proposed and reported by the author. All of the reported robot demonstrations in this work follow a perception-manipulation method- ology where visual and tactile feedback (in the form of surface wrinkledness captured by the high accuracy depth sensor i.e. CloPeMa stereo head or the predictive confidence modelled by Gaussian Processing) serve as the halting criteria in the flattening and sorting tasks, respectively. From scientific perspective, the proposed visual perception architecture addresses the above challenges by parsing and grouping 3D clothing configurations hierarchically from low-level curvatures, through mid-level surface shape representations (providing topological descriptions and 3D texture representations), to high-level semantic structures and statistical descriptions. A range of visual features such as Shape Index, Surface Topologies Analysis and Local Binary Patterns have been adapted within this work to parse clothing surfaces and textures and several novel features have been devised, including B-Spline Patches with Locality-Constrained Linear coding, and Topology Spatial Distance to describe and quantify generic landmarks (wrinkles and folds). The essence of this proposed architecture comprises 3D generic surface parsing and interpretation, which is critical to underpinning a number of laundering tasks and has the potential to be extended to other rigid and non-rigid object perception and manipulation tasks. The experimental results presented in this thesis demonstrate that: firstly, the proposed grasp- ing approach achieves on-average 84.7% accuracy; secondly, the proposed flattening approach is able to flatten towels, t-shirts and pants (shorts) within 9 iterations on-average; thirdly, the proposed clothes recognition pipeline can recognise clothes categories from highly wrinkled configurations and advances the state-of-the-art by 36% in terms of classification accuracy, achieving an 83.2% true-positive classification rate when discriminating between five categories of clothes; finally the Gaussian Process based interactive perception approach exhibits a substantial improvement over single-shot perception. Accordingly, this thesis has advanced the state-of-the-art of robot clothes perception and manipulation.

Context Models for Understanding Images and Videos

A computer vision system that has to interact in natural language needs to understand the visual appearance of interactions between objects along with the appearance of objects themselves. Relationships between objects are frequently mentioned in queries of tasks like semantic image retrieval, image captioning, visual question answering and natural language object detection. Hence, it is essential to model context between objects for solving these tasks. In the first part of this thesis, we present a technique for detecting an object mentioned in a natural language query. Specifically, we work with referring expressions which are sentences that identify a particular object instance in an image. In many referring expressions, an object is described in relation to another object using prepositions, comparative adjectives, action verbs etc. Our proposed technique can identify both the referred object and the context object mentioned in such expressions. Context is also useful for incrementally understanding scenes and videos. In the second part of this thesis, we propose techniques for searching for objects in an image and events in a video. Our proposed incremental algorithms use the context from previously explored regions to prioritize the regions to explore next. The advantage of incremental understanding is restricting the amount of computation time and/or resources spent for various detection tasks. Our first proposed technique shows how to learn context in indoor scenes in an implicit manner and use it for searching for objects. The second technique shows how explicitly written context rules of one-on-one basketball can be used to sequentially detect events in a game.

Object detection and recognition in complex scenes

Dissertação de Mestrado, Engenharia Informática, Faculdade de Ciências e Tecnologia, Universidade do Algarve, 2014

Guided Bus Rapid Transit

Electrical Bus Rapid Transit (eBRT) is a charging electrical public transport which brings a clean, high performance, and affordable cost alternative from the conventional traffic vehicles which work with combustion and hybrid technology. These buses charge the battery in every bus stop to arrive at the next station. But, this charging system needs an appropriate infrastructure called pantograph, and it requires a high precision bus location to maintain battery lifetime, energy saving and charging time. To overcome this issue Vicomtech and Datik has planned a project based on computer vision to help to the driver to locate the vehicle in the correct place. In this document, we present a mono camera bus driver guided fast algorithm because these vehicles embedded computers do not support high computation and precision operations. In addition to the frequent lane sign, there are more accurate geometric beacons painted on the road to bring metric information to the vision system. This method uses segmentation to binarize the image discriminating the background space. Besides it detects, tracks and counts different lane mark contours in addition to classify each special painted mark. Besides it does not need any calibration task to calculate longitudinal and cross distances because we know the lane mark sizes.

Mars delivery service - development of the electro-mechanical systems of the Sample Fetch Rover for the Mars Sample Return Campaign

This thesis describes the development of the Sample Fetch Rover (SFR), studied for Mars Sample Return (MSR), an international campaign carried out in cooperation between the National Aeronautics and Space Administration (NASA) and the European Space Agency (ESA). The focus of this document is the design of the electro-mechanical systems of the rover. After placing this work into the general context of robotic planetary exploration and summarising the state of the art for what concerns Mars rovers, the architecture of the Mars Sample Return Campaign is presented. A complete overview of the current SFR architecture is provided, touching upon all the main subsystems of the spacecraft. For each area, it is discussed what are the design drivers, the chosen solutions and whether they use heritage technology (in particular from the ExoMars Rover) or new developments. This research focuses on two topics of particular interest, due to their relevance for the mission and the novelty of their design: locomotion and sample acquisition, which are discussed in depth. The early SFR locomotion concepts are summarised, covering the initial trade-offs and discarded designs for higher traverse performance. Once a consolidated architecture was reached, the locomotion subsystem was developed further, defining the details of the suspension, actuators, deployment mechanisms and wheels. This technology is presented here in detail, including some key analysis and test results that support the design and demonstrate how it responds to the mission requirements. Another major electro-mechanical system developed as part of this work is the one dedicated to sample tube acquisition. The concept of operations of this machinery was defined to be robust against the unknown conditions that characterise the mission. The design process led to a highly automated robotic system which is described here in its main components: vision system, robotic arm and tube storage.

A ROS-based software architecture for a versatile collaborative dual-armed autonomous mobile robot for the manufacturing industry

The industrial context is changing rapidly due to advancements in technology fueled by the Internet and Information Technology. The fourth industrial revolution counts integration, flexibility, and optimization as its fundamental pillars, and, in this context, Human-Robot Collaboration has become a crucial factor for manufacturing sustainability in Europe. Collaborative robots are appealing to many companies due to their low installation and running costs and high degree of flexibility, making them ideal for reshoring production facilities with a short return on investment. The ROSSINI European project aims to implement a true Human-Robot Collaboration by designing, developing, and demonstrating a modular and scalable platform for integrating human-centred robotic technologies in industrial production environments. The project focuses on safety concerns related to introducing a cobot in a shared working area and aims to lay the groundwork for a new working paradigm at the industrial level. The need for a software architecture suitable to the robotic platform employed in one of three use cases selected to deploy and test the new technology was the main trigger of this Thesis. The chosen application consists of the automatic loading and unloading of raw-material reels to an automatic packaging machine through an Autonomous Mobile Robot composed of an Autonomous Guided Vehicle, two collaborative manipulators, and an eye-on-hand vision system for performing tasks in a partially unstructured environment. The results obtained during the ROSSINI use case development were later used in the SENECA project, which addresses the need for robot-driven automatic cleaning of pharmaceutical bins in a very specific industrial context. The inherent versatility of mobile collaborative robots is evident from their deployment in the two projects with few hardware and software adjustments. The positive impact of Human-Robot Collaboration on diverse production lines is a motivation for future investments in research on this increasingly popular field by the industry.

Operazioni di assemblaggio automatizzato tramite robot antropomorfo e visione artificiale

Obiettivo dello studio condotto è l’implementazione di cicli di operazioni per l’assemblaggio automatizzato di componenti che costituiscono un sistema di trasporto a catena presente in alcune macchine automatiche. L’automazione del processo, fino ad oggi svolto manualmente, prevede l’utilizzo di un robot e, per il controllo di quest’ultimo, di un sistema di visione artificiale. L’attività di tirocinio associata alla tesi di laurea, che ha incluso una parte sperimentale oltre alla scrittura degli algoritmi di controllo del robot, è stata svolta all’interno del laboratorio TAILOR (Technology and Automation for Industry LabORatory) presso Siropack Italia S.r.l dove è presente una cella dotata di robot antropomorfo (Mitsubishi Electric) e di sistema di visione artificiale con camere 2D (Omron). La presenza di quest’ultimo è risultata strategica in termini di possibilità di adattare il montaggio anche a diversi posizionamenti degli oggetti all’interno dello spazio di lavoro, fermo restando che gli stessi risultassero appoggiati su una superficie piana. In primo luogo, affinché fosse garantita la ripetibilità del processo, sono state testate le prestazioni del sistema di visione tramite opportuna calibrazione della camera e del sistema di illuminazione ad esso collegata, al fine di ottenere un’acquisizione delle immagini che fosse sufficientemente robusta e risoluta mediante lo sfruttamento del software di elaborazione Omron FH Vision System. Un’opportuna programmazione della traiettoria del robot in ambiente di simulazione RT Toolbox 3, software integrato nel sistema di controllo del robot Mitsubishi Electric, ha infine consentito le regolari operazioni di assemblaggio, garantendo un processo affidabile ed, allo stesso tempo, adattabile ad ambienti eventualmente non strutturati in cui esso si trova ad operare.

Hand gesture recognition system based in computer vision and machine learning : applications on human-machine interaction

Sendo uma forma natural de interação homem-máquina, o reconhecimento de gestos implica uma forte componente de investigação em áreas como a visão por computador e a aprendizagem computacional. O reconhecimento gestual é uma área com aplicações muito diversas, fornecendo aos utilizadores uma forma mais natural e mais simples de comunicar com sistemas baseados em computador, sem a necessidade de utilização de dispositivos extras. Assim, o objectivo principal da investigação na área de reconhecimento de gestos aplicada à interacção homemmáquina é o da criação de sistemas, que possam identificar gestos específicos e usálos para transmitir informações ou para controlar dispositivos. Para isso as interfaces baseados em visão para o reconhecimento de gestos, necessitam de detectar a mão de forma rápida e robusta e de serem capazes de efetuar o reconhecimento de gestos em tempo real. Hoje em dia, os sistemas de reconhecimento de gestos baseados em visão são capazes de trabalhar com soluções específicas, construídos para resolver um determinado problema e configurados para trabalhar de uma forma particular. Este projeto de investigação estudou e implementou soluções, suficientemente genéricas, com o recurso a algoritmos de aprendizagem computacional, permitindo a sua aplicação num conjunto alargado de sistemas de interface homem-máquina, para reconhecimento de gestos em tempo real. A solução proposta, Gesture Learning Module Architecture (GeLMA), permite de forma simples definir um conjunto de comandos que pode ser baseado em gestos estáticos e dinâmicos e que pode ser facilmente integrado e configurado para ser utilizado numa série de aplicações. É um sistema de baixo custo e fácil de treinar e usar, e uma vez que é construído unicamente com bibliotecas de código. As experiências realizadas permitiram mostrar que o sistema atingiu uma precisão de 99,2% em termos de reconhecimento de gestos estáticos e uma precisão média de 93,7% em termos de reconhecimento de gestos dinâmicos. Para validar a solução proposta, foram implementados dois sistemas completos. O primeiro é um sistema em tempo real capaz de ajudar um árbitro a arbitrar um jogo de futebol robótico. A solução proposta combina um sistema de reconhecimento de gestos baseada em visão com a definição de uma linguagem formal, o CommLang Referee, à qual demos a designação de Referee Command Language Interface System (ReCLIS). O sistema identifica os comandos baseados num conjunto de gestos estáticos e dinâmicos executados pelo árbitro, sendo este posteriormente enviado para um interface de computador que transmite a respectiva informação para os robôs. O segundo é um sistema em tempo real capaz de interpretar um subconjunto da Linguagem Gestual Portuguesa. As experiências demonstraram que o sistema foi capaz de reconhecer as vogais em tempo real de forma fiável. Embora a solução implementada apenas tenha sido treinada para reconhecer as cinco vogais, o sistema é facilmente extensível para reconhecer o resto do alfabeto. As experiências também permitiram mostrar que a base dos sistemas de interação baseados em visão pode ser a mesma para todas as aplicações e, deste modo facilitar a sua implementação. A solução proposta tem ainda a vantagem de ser suficientemente genérica e uma base sólida para o desenvolvimento de sistemas baseados em reconhecimento gestual que podem ser facilmente integrados com qualquer aplicação de interface homem-máquina. A linguagem formal de definição da interface pode ser redefinida e o sistema pode ser facilmente configurado e treinado com um conjunto de gestos diferentes de forma a serem integrados na solução final.