Grounding human vocabulary in robot perception through interaction

This thesis addresses the problem of word learning in computational agents. The motivation behind this work lies in the need to support language-based communication between service robots and their human users, as well as grounded reasoning using symbols relevant for the assigned tasks. The research focuses on the problem of grounding human vocabulary in robotic agent’s sensori-motor perception. Words have to be grounded in bodily experiences, which emphasizes the role of appropriate embodiments. On the other hand, language is a cultural product created and acquired through social interactions. This emphasizes the role of society as a source of linguistic input. Taking these aspects into account, an experimental scenario is set up where a human instructor teaches a robotic agent the names of the objects present in a visually shared environment. The agent grounds the names of these objects in visual perception. Word learning is an open-ended problem. Therefore, the learning architecture of the agent will have to be able to acquire words and categories in an openended manner. In this work, four learning architectures were designed that can be used by robotic agents for long-term and open-ended word and category acquisition. The learning methods used in these architectures are designed for incrementally scaling-up to larger sets of words and categories. A novel experimental evaluation methodology, that takes into account the openended nature of word learning, is proposed and applied. This methodology is based on the realization that a robot’s vocabulary will be limited by its discriminatory capacity which, in turn, depends on its sensors and perceptual capabilities. An extensive set of systematic experiments, in multiple experimental settings, was carried out to thoroughly evaluate the described learning approaches. The results indicate that all approaches were able to incrementally acquire new words and categories. Although some of the approaches could not scale-up to larger vocabularies, one approach was shown to learn up to 293 categories, with potential for learning many more.

Robot

Klimowski’s graphic novel, Robot, was commissioned by the Adam Mickiewicz Institute in Warsaw to mark Poland’s Presidency of the European Union’s Cultural Programme in 2011. Self Made Hero and Timof Comiks published the book simultaneously in the UK and Poland. Klimowski adapted and translated Stanisław Lem’s short fiction ‘The sanatorium of Dr Vliperdius’ (1977), aiming to develop a new position for illustration and the graphic novel aside from mainstream graphic novels and literature, and a new approach to visual bookmaking. The project proved to be an artistic challenge: Lem often proclaimed his disapproval of adaptations of his work, dismissing even Andrei Tarkovsky’s film adaptation (1972) of his novel Solaris (1961). Produced in collaboration with Danusia Schejbal, Robot features a diptych form, counter-pointing (both formally and conceptually) two contrasting stories. The first is a colourful parable describing a totalitarian and autocratic regime that must be vanquished, the second a monochromatic dialectic on philosophy, humanism and mechanisation. Klimowski and Schejbal’s publication is intended to challenge stereotypes and established styles and formulas associated with the production of graphic novels. Much emphasis was laid upon the depiction of space and location, artificiality and realism. Silence and the suspension of linear time were also strong features of the artists’ investigations. These qualities were recognised and discussed by the media, in particular by a panel of critics on Polish Television’s Cultural Channel, in the most respected comics blog, Zeszyty Komiksowe (http://zeszytykomiksowe.org/recenzja_robot, 2012), and by Monika Malkowskain in the national newspaper Rzeczpospolita (2011). The artists gave a special talk at the Science Museum, London, during the Robot Festival ‘Robotville’ (December 2011). Lem, one of the world’s leading writers of science fiction, was featured throughout the year in the UK on stage, cinema and in literary events (Barbican Centre London, British Library, Science Museum London).

Genetic programming and bacterial algorithm for neural networks and fuzzy systems design

In the field of control systems it is common to use techniques based on model adaptation to carry out control for plants for which mathematical analysis may be intricate. Increasing interest in biologically inspired learning algorithms for control techniques such as Artificial Neural Networks and Fuzzy Systems is in progress. In this line, this paper gives a perspective on the quality of results given by two different biologically connected learning algorithms for the design of B-spline neural networks (BNN) and fuzzy systems (FS). One approach used is the Genetic Programming (GP) for BNN design and the other is the Bacterial Evolutionary Algorithm (BEA) applied for fuzzy rule extraction. Also, the facility to incorporate a multi-objective approach to the GP algorithm is outlined, enabling the designer to obtain models more adequate for their intended use.

Genetic and bacterial programming for B-Spline neural networks design

The design phase of B-spline neural networks is a highly computationally complex task. Existent heuristics have been found to be highly dependent on the initial conditions employed. Increasing interest in biologically inspired learning algorithms for control techniques such as Artificial Neural Networks and Fuzzy Systems is in progress. In this paper, the Bacterial Programming approach is presented, which is based on the replication of the microbial evolution phenomenon. This technique produces an efficient topology search, obtaining additionally more consistent solutions.

Design of B-spline neural networks using a bacterial programming approach

The design phase of B-spline neural networks represents a very high computational task. For this purpose, heuristics have been developed, but have been shown to be dependent on the initial conditions employed. In this paper a new technique, Bacterial Programming, is proposed, whose principles are based on the replication of the microbial evolution phenomenon. The performance of this approach is illustrated and compared with existing alternatives.

A computational study of a parallel Branch and Bound algorithm for the quadratic 0-1 programming problem on transputers

Discrete optimization problems are very difficult to solve, even if the dimantion is small. For most of them the problem of finding an ε-approximate solution is already NP-hard.

Observação e análise da utilização da mini-linguagem Karel the robot num modelo de ensino para aprendizagem de programação no 10º ano do Curso Tecnológico de Informática

O grande objectivo deste estudo foi contribuir para a melhoria do ensino da programação de computadores para aluno principiantes, através de um processo de observação da relação pedagógica estabelecida em sala de aula.

Biologically Inspired Vision for Indoor Robot Navigation

Ultrasonic, infrared, laser and other sensors are being applied in robotics. Although combinations of these have allowed robots to navigate, they are only suited for specific scenarios, depending on their limitations. Recent advances in computer vision are turning cameras into useful low-cost sensors that can operate in most types of environments. Cameras enable robots to detect obstacles, recognize objects, obtain visual odometry, detect and recognize people and gestures, among other possibilities. In this paper we present a completely biologically inspired vision system for robot navigation. It comprises stereo vision for obstacle detection, and object recognition for landmark-based navigation. We employ a novel keypoint descriptor which codes responses of cortical complex cells. We also present a biologically inspired saliency component, based on disparity and colour.

Methodological aspects of a mathematical programming model to evaluate soil tillage technologies in a risky environment

In this work we develop a methodology for the economic evaluation of soil tillage technologies, in a risky environment, and to capture the influence of farmer behaviour on his technology choice. The model has short-term activities, that change with the type of year, and long-term activities, in which sets of traction investment activities are included. Although these activities do not change with the type of year, they lead to different availability of resources for each type of year, since the same tractor has different available fieldwork days under different weather conditions. We prove that the model is sensitive to the greater income variability resulting from the use of alternative technologies and to the balance between income and risk, accounting for the probability of occurrence of each state of nature and giving an investment solution that considers the best production plan for each type of year. (c) 2005 Elsevier B.V. All rights reserved.

Early nutritional programming in fish: tailoring the metabolic use of dietary carbohydrates

Tese de doutoramento, Ciências da Vida, do Mar, da Terra e do Ambiente (Nutrição), Faculdade de Ciências e Tecnologia, Universidade do Algarve, 2015

O robot NXT Mindstorms e a área de projecto

Tese de mestrado, Educação (Tecnologias de Informação e Comunicação e Educação), Universidade de Lisboa, Instituto de Educação, 2010

A robótica educativa no ensino e aprendizagem de conceitos de programação e algoritmos

Relatório da Prática de Ensino Supervisionada, Ensino de Informática, Universidade de Lisboa, 2013

From embryonic stem cells to sensory hair cells : a programming approach

Tese de doutoramento, Ciências Biomédicas (Biologia do Desenvolvimento), Universidade de Lisboa, Faculdade de Medicina, 2014

Introdução à inteligência artificial com recurso à programação visual

Relatório da Prática de Ensino Supervisionada, Mestrado em Ensino da Matemática, Universidade de Lisboa, Instituto de Educação, 2014

Programming multicores safely : handling barrier deadlocks

Tese de doutoramento, Informática (Ciências da Computação), Universidade de Lisboa, Faculdade de Ciências, 2015