Grounding human vocabulary in robot perception through interaction

This thesis addresses the problem of word learning in computational agents. The motivation behind this work lies in the need to support language-based communication between service robots and their human users, as well as grounded reasoning using symbols relevant for the assigned tasks. The research focuses on the problem of grounding human vocabulary in robotic agent’s sensori-motor perception. Words have to be grounded in bodily experiences, which emphasizes the role of appropriate embodiments. On the other hand, language is a cultural product created and acquired through social interactions. This emphasizes the role of society as a source of linguistic input. Taking these aspects into account, an experimental scenario is set up where a human instructor teaches a robotic agent the names of the objects present in a visually shared environment. The agent grounds the names of these objects in visual perception. Word learning is an open-ended problem. Therefore, the learning architecture of the agent will have to be able to acquire words and categories in an openended manner. In this work, four learning architectures were designed that can be used by robotic agents for long-term and open-ended word and category acquisition. The learning methods used in these architectures are designed for incrementally scaling-up to larger sets of words and categories. A novel experimental evaluation methodology, that takes into account the openended nature of word learning, is proposed and applied. This methodology is based on the realization that a robot’s vocabulary will be limited by its discriminatory capacity which, in turn, depends on its sensors and perceptual capabilities. An extensive set of systematic experiments, in multiple experimental settings, was carried out to thoroughly evaluate the described learning approaches. The results indicate that all approaches were able to incrementally acquire new words and categories. Although some of the approaches could not scale-up to larger vocabularies, one approach was shown to learn up to 293 categories, with potential for learning many more.

Robot

Klimowski’s graphic novel, Robot, was commissioned by the Adam Mickiewicz Institute in Warsaw to mark Poland’s Presidency of the European Union’s Cultural Programme in 2011. Self Made Hero and Timof Comiks published the book simultaneously in the UK and Poland. Klimowski adapted and translated Stanisław Lem’s short fiction ‘The sanatorium of Dr Vliperdius’ (1977), aiming to develop a new position for illustration and the graphic novel aside from mainstream graphic novels and literature, and a new approach to visual bookmaking. The project proved to be an artistic challenge: Lem often proclaimed his disapproval of adaptations of his work, dismissing even Andrei Tarkovsky’s film adaptation (1972) of his novel Solaris (1961). Produced in collaboration with Danusia Schejbal, Robot features a diptych form, counter-pointing (both formally and conceptually) two contrasting stories. The first is a colourful parable describing a totalitarian and autocratic regime that must be vanquished, the second a monochromatic dialectic on philosophy, humanism and mechanisation. Klimowski and Schejbal’s publication is intended to challenge stereotypes and established styles and formulas associated with the production of graphic novels. Much emphasis was laid upon the depiction of space and location, artificiality and realism. Silence and the suspension of linear time were also strong features of the artists’ investigations. These qualities were recognised and discussed by the media, in particular by a panel of critics on Polish Television’s Cultural Channel, in the most respected comics blog, Zeszyty Komiksowe (http://zeszytykomiksowe.org/recenzja_robot, 2012), and by Monika Malkowskain in the national newspaper Rzeczpospolita (2011). The artists gave a special talk at the Science Museum, London, during the Robot Festival ‘Robotville’ (December 2011). Lem, one of the world’s leading writers of science fiction, was featured throughout the year in the UK on stage, cinema and in literary events (Barbican Centre London, British Library, Science Museum London).

Observação e análise da utilização da mini-linguagem Karel the robot num modelo de ensino para aprendizagem de programação no 10º ano do Curso Tecnológico de Informática

O grande objectivo deste estudo foi contribuir para a melhoria do ensino da programação de computadores para aluno principiantes, através de um processo de observação da relação pedagógica estabelecida em sala de aula.

Biologically Inspired Vision for Indoor Robot Navigation

Ultrasonic, infrared, laser and other sensors are being applied in robotics. Although combinations of these have allowed robots to navigate, they are only suited for specific scenarios, depending on their limitations. Recent advances in computer vision are turning cameras into useful low-cost sensors that can operate in most types of environments. Cameras enable robots to detect obstacles, recognize objects, obtain visual odometry, detect and recognize people and gestures, among other possibilities. In this paper we present a completely biologically inspired vision system for robot navigation. It comprises stereo vision for obstacle detection, and object recognition for landmark-based navigation. We employ a novel keypoint descriptor which codes responses of cortical complex cells. We also present a biologically inspired saliency component, based on disparity and colour.

O robot NXT Mindstorms e a área de projecto

Tese de mestrado, Educação (Tecnologias de Informação e Comunicação e Educação), Universidade de Lisboa, Instituto de Educação, 2010

A mobile robot platform for open learning based on serious games and remote laboratories

Within the pedagogical community, Serious Games have arisen as a viable alternative to traditional course-based learning materials. Until now, they have been based strictly on software solutions. Meanwhile, research into Remote Laboratories has shown that they are a viable, low-cost solution for experimentation in an engineering context, providing uninterrupted access, low-maintenance requirements, and a heightened sense of reality when compared to simulations. This paper will propose a solution where both approaches are combined to deliver a Remote Laboratory-based Serious Game for use in engineering and school education. The platform for this system is the WebLab-Deusto Framework, already well-tested within the remote laboratory context, and based on open standards. The laboratory allows users to control a mobile robot in a labyrinth environment and take part in an interactive game where they must locate and correctly answer several questions, the subject of which can be adapted to educators' needs. It also integrates the Google Blockly graphical programming language, allowing students to learn basic programming and logic principles without needing to understand complex syntax.

Decentralized Target Tracking based on Multi-Robot Cooperative Triangulation

Target tracking with bearing-only sensors is a challenging problem when the target moves dynamically in complex scenarios. Besides the partial observability of such sensors, they have limited field of views, occlusions can occur, etc. In those cases, cooperative approaches with multiple tracking robots are interesting, but the different sources of uncertain information need to be considered appropriately in order to achieve better estimates. Even though there exist probabilistic filters that can estimate the position of a target dealing with incertainties, bearing-only measurements bring usually additional problems with initialization and data association. In this paper, we propose a multi-robot triangulation method with a dynamic baseline that can triangulate bearing-only measurements in a probabilistic manner to produce 3D observations. This method is combined with a decentralized stochastic filter and used to tackle those initialization and data association issues. The approach is validated with simulations and field experiments where a team of aerial and ground robots with cameras track a dynamic target.

TIGRE - An autonomous ground robot for outdoor exploration

13th International Conference on Autonomous Robot Systems (Robotica), 2013

Neural control of an autonomous robot

This article aims to apply the concepts associated with artificial neural networks (ANN) in the control of an autonomous robot system that is intended to be used in competitions of robots. The robot was tested in several arbitrary paths in order to verify its effectiveness. The results show that the robot performed the tasks with success. Moreover, in the case of arbitrary paths the ANN control outperforms other methodologies, such as fuzzy logic control (FLC).

Development of advanced human robot interaction systems

O trabalho aqui apresentado é a Dissertação da minha Tese do curso de Mestrado em Engenharia Eletrotécnica e de Computadores do ISEP, realizada em parceria com o INESC TEC. O trabalho consiste no desenvolvimento de um sistema avançado de interação entre homem-robô, usando ferramentas de software livres e de domínio público e hardware pouco dispendioso e facilmente acessível. Pretende-se que o sistema desenvolvido possa ser adotado por pequenas ou micro empresas, daí a restrição monetária. Este tipo de empresas tem, por norma, uma capacidade de investimento pequena, e ficam impossibilitadas de aceder a este tipo de sistemas automatizados se estes forem caros. No entanto, o robô continua a ser um componente fundamental, sendo dispendioso. Os trabalhos realizados pelos sistemas robóticos podem por um lado, ser repetitivos sem necessidade de grandes ajustes; por outro lado, o trabalho a realizar pode ser bastante diverso, sendo necessários bastantes ajustes com (possivelmente) programação do robô. As empresas podem não ter disponível mão-de-obra qualificada para realização da programação do robô. Pretende-se então um sistema de “ensino” que seja simples e rápido. Este trabalho pretende satisfazer as necessidades de um sistema de interação homem-robô intuitivo mesmo para operadores que não estejam familiarizados com a robótica. Para simplificar a transferência de informação da tarefa a desempenhar pelo sistema robótico é usado um sistema de infravermelhos para delinear a operação a desempenhar, neste caso concreto uma operação de soldadura. O operador usa um apontador com marcadores, a posição destes marcadores é detetada usando duas câmaras para permitir o posicionamento tridimensional no espaço. As câmaras possuem filtros infravermelhos para separar o espectro de luz. Para o controlo do sistema e interface com o robô é usado um computador de baixos recursos computacionais e energéticos, e também de baixo custo. O sistema desenvolvido é portanto computacionalmente leve para poder ser executado neste computador.

Residence of William W. Kitchen & Residence of Charles Kitchen - Vignette

A vignette two homes in Grimsby. There are two homes in the image, one the residence of William W. Kitchen, the second the residence of Charles Kitchen.

Diseño de un observador Backstepping para un robot con articulación rotaria flexible

Tesis (Maestro en Ciencias de la Ingeniería Eléctrica con Especialidad en Control) UANL, 2001.

Estrategia didáctica para el desarrollo de competencias en solución de problemas mediante la programación digital del robot Karel.

Tesis (Maestría en Docencia con Orientación en Educación Media Superior) UANL, 2012.

Optimization and validation of a new 3D-US imaging robot to detect, localize and quantify lower limb arterial stenoses

L’athérosclérose est une maladie qui cause, par l’accumulation de plaques lipidiques, le durcissement de la paroi des artères et le rétrécissement de la lumière. Ces lésions sont généralement localisées sur les segments artériels coronariens, carotidiens, aortiques, rénaux, digestifs et périphériques. En ce qui concerne l’atteinte périphérique, celle des membres inférieurs est particulièrement fréquente. En effet, la sévérité de ces lésions artérielles est souvent évaluée par le degré d’une sténose (réduction >50 % du diamètre de la lumière) en angiographie, imagerie par résonnance magnétique (IRM), tomodensitométrie ou échographie. Cependant, pour planifier une intervention chirurgicale, une représentation géométrique artérielle 3D est notamment préférable. Les méthodes d’imagerie par coupe (IRM et tomodensitométrie) sont très performantes pour générer une imagerie tridimensionnelle de bonne qualité mais leurs utilisations sont dispendieuses et invasives pour les patients. L’échographie 3D peut constituer une avenue très prometteuse en imagerie pour la localisation et la quantification des sténoses. Cette modalité d’imagerie offre des avantages distincts tels la commodité, des coûts peu élevés pour un diagnostic non invasif (sans irradiation ni agent de contraste néphrotoxique) et aussi l’option d’analyse en Doppler pour quantifier le flux sanguin. Étant donné que les robots médicaux ont déjà été utilisés avec succès en chirurgie et en orthopédie, notre équipe a conçu un nouveau système robotique d’échographie 3D pour détecter et quantifier les sténoses des membres inférieurs. Avec cette nouvelle technologie, un radiologue fait l’apprentissage manuel au robot d’un balayage échographique du vaisseau concerné. Par la suite, le robot répète à très haute précision la trajectoire apprise, contrôle simultanément le processus d’acquisition d’images échographiques à un pas d’échantillonnage constant et conserve de façon sécuritaire la force appliquée par la sonde sur la peau du patient. Par conséquent, la reconstruction d’une géométrie artérielle 3D des membres inférieurs à partir de ce système pourrait permettre une localisation et une quantification des sténoses à très grande fiabilité. L’objectif de ce projet de recherche consistait donc à valider et optimiser ce système robotisé d’imagerie échographique 3D. La fiabilité d’une géométrie reconstruite en 3D à partir d’un système référentiel robotique dépend beaucoup de la précision du positionnement et de la procédure de calibration. De ce fait, la précision pour le positionnement du bras robotique fut évaluée à travers son espace de travail avec un fantôme spécialement conçu pour simuler la configuration des artères des membres inférieurs (article 1 - chapitre 3). De plus, un fantôme de fils croisés en forme de Z a été conçu pour assurer une calibration précise du système robotique (article 2 - chapitre 4). Ces méthodes optimales ont été utilisées pour valider le système pour l’application clinique et trouver la transformation qui convertit les coordonnées de l’image échographique 2D dans le référentiel cartésien du bras robotisé. À partir de ces résultats, tout objet balayé par le système robotique peut être caractérisé pour une reconstruction 3D adéquate. Des fantômes vasculaires compatibles avec plusieurs modalités d’imagerie ont été utilisés pour simuler différentes représentations artérielles des membres inférieurs (article 2 - chapitre 4, article 3 - chapitre 5). La validation des géométries reconstruites a été effectuée à l`aide d`analyses comparatives. La précision pour localiser et quantifier les sténoses avec ce système robotisé d’imagerie échographique 3D a aussi été déterminée. Ces évaluations ont été réalisées in vivo pour percevoir le potentiel de l’utilisation d’un tel système en clinique (article 3- chapitre 5).