Grounding human vocabulary in robot perception through interaction

This thesis addresses the problem of word learning in computational agents. The motivation behind this work lies in the need to support language-based communication between service robots and their human users, as well as grounded reasoning using symbols relevant for the assigned tasks. The research focuses on the problem of grounding human vocabulary in robotic agent’s sensori-motor perception. Words have to be grounded in bodily experiences, which emphasizes the role of appropriate embodiments. On the other hand, language is a cultural product created and acquired through social interactions. This emphasizes the role of society as a source of linguistic input. Taking these aspects into account, an experimental scenario is set up where a human instructor teaches a robotic agent the names of the objects present in a visually shared environment. The agent grounds the names of these objects in visual perception. Word learning is an open-ended problem. Therefore, the learning architecture of the agent will have to be able to acquire words and categories in an openended manner. In this work, four learning architectures were designed that can be used by robotic agents for long-term and open-ended word and category acquisition. The learning methods used in these architectures are designed for incrementally scaling-up to larger sets of words and categories. A novel experimental evaluation methodology, that takes into account the openended nature of word learning, is proposed and applied. This methodology is based on the realization that a robot’s vocabulary will be limited by its discriminatory capacity which, in turn, depends on its sensors and perceptual capabilities. An extensive set of systematic experiments, in multiple experimental settings, was carried out to thoroughly evaluate the described learning approaches. The results indicate that all approaches were able to incrementally acquire new words and categories. Although some of the approaches could not scale-up to larger vocabularies, one approach was shown to learn up to 293 categories, with potential for learning many more.

Beyond the artefact and techno-centricity: towards process-centric understanding of architecture - alternative facilitation strategies and proposals

The artefact and techno-centricity of the research into the architecture process needs to be counterbalanced by other approaches. An increasing amount of information is collected and used in the process, resulting in challenges related to information and knowledge management, as this research evidences through interviews with practicing architects. However, emerging technologies are expected to resolve many of the traditional challenges, opening up new avenues for research. This research suggests that among them novel techniques addressing how architects interact with project information, especially that indirectly related to the artefacts, and tools which better address the social nature of work, notably communication between participants, become a higher priority. In the fields associated with the Human Computer Interaction generic solutions still frequently prevail, whereas it appears that specific alternative approaches would be particularly in demand for the dynamic and context dependent design process. This research identifies an opportunity for a process-centric and integrative approach for architectural practice and proposes an information management and communication software application, developed for the needs discovered in close collaboration with architects. Departing from the architects’ challenges, an information management software application, Mneme, was designed and developed until a working prototype. It proposes the use of visualizations as an interface to provide an overview of the process, facilitate project information retrieval and access, and visualize relationships between the pieces of information. Challenges with communication about visual content, such as images and 3D files, led to a development of a communication feature allowing discussions attached to any file format and searchable from a database. Based on the architects testing the prototype and literature recognizing the subjective side of usability, this thesis argues that visualizations, even 3D visualizations, present potential as an interface for information management in the architecture process. The architects confirmed that Mneme allowed them to have a better project overview, to easier locate heterogeneous content, and provided context for the project information. Communication feature in Mneme was seen to offer a lot of potential in design projects where diverse file formats are typically used. Through empirical understanding of the challenges in the architecture process, and through testing the resulting software proposal, this thesis suggests promising directions for future research into the architecture and design process.