ROSSI: Emergence of communication in Robots through Sensorimotor and Social Interaction

ROSSI: Emergence of communication in Robots through Sensorimotor and Social Interaction, T. Ziemke, A. Borghi, F. Anelli, C. Gianelli, F. Binkovski, G. Buccino, V. Gallese, M. Huelse, M. Lee, R. Nicoletti, D. Parisi, L. Riggio, A. Tessari, E. Sahin, International Conference on Cognitive Systems (CogSys 2008), University of Karlsruhe, Karlsruhe, Germany, 2008 Sponsorship: EU-FP7

Starting from the assumption that cognition is embodied, the ROSSI project addresses the question of how the possibility of communication between agents (e.g. humans and robots) is affected by differences in sensorimotor capacities. This is an important issue, given that robots are expected to become more common in nonstructured environments, such as homes or hospitals. While there is a trend towards humanoid robots, it is clear that for the foreseeable future there will still be substantial differences in physical embodiment between robots and humans. To explore to what extent concepts must be shared to facilitate communication, we will build robots with sensorimotor systems structurally roughly similar to human beings. Furthermore, the control mechanisms for these robots will be based on insights into the neural mechanisms underlying human concepts and language. In particular, two types of neurons in premotor cortex will be modelled: (1) canonical neurons, which are active during both the execution of specific object-directed actions and the mere visual observation of the same objects, and (2) mirror neurons, which are involved in both an agent?s own actions and the visual observation of such actions performed by others. In this framework, the project?s aims are twofold: - (a) to provide new neuroscientific/psychological insights into the sensorimotor grounding of human conceptualization and language use, in particular the role of canonical and mirror neurons as underlying the use of nouns and verbs, - (b) to develop novel approaches to sensorimotor grounding of robotic conceptualization and language use (more precisely, verbal labeling of objects and actions), based on the insights gained under (a) and richer computational/robotic models of the underlying neural mechanisms. More specifically, the objectives are: - (1) to provide robots with mechanisms for the detection and use of object affordances (with a focus on object manipulation and grasping) that allow to respond with appropriate motor responses to different object characteristics (e.g. size, orientation, weight). For example, robots should be able to manipulate objects that differ in size, thus eliciting a different kind of grip, that differ in weight, that are differently oriented in relation to the agent. - (2) to develop mechanisms that allow robots to learn through simple forms of social interaction, in particular the imitation of human behavior (cf. Tessari & Rumiati, 2004; this will also address the distinction between trying to produce the same results of the movements of a human being imitating the movements themselves) - (3) to build mechanisms that allow robots to develop embodied concepts and a very basic form of language, more specifically, the verbal labeling of (a) objects (use of nouns; presumably involving models of canonical neurons and affordances), (b) actions (use of verbs; presumably involving the use of mirror neuron system models) - (4) to enable robots to learn to use such verbal labels in simple forms of social/linguistic interaction with humans, such as, for example, (a) responding with the production of a verbal label 'ball' when observing a ball, (b) responding with appropriate actions to verbal commands such as 'grasp', (c) the combination of elementary verbal labels, e.g. commands such as 'grasp ball' - (5) to demonstrate capacities (1)-(4) mentioned above in a real-world robotic platform - (6) to make significant contributions to the development of the theory and engineering principles for sensorimotor grounding of robotic language use, based on both the neuroscientific/psychological work and the computational and robotic models developed.

Non peer reviewed