Three-dimensional mapping of real environments for cognitive robots navigation

Abstract The development of cognitive robots needs a strong “sensorial” support which should allow it to perceive the real world for interacting with it properly. Therefore the development of efficient visual-processing software to be equipped in effective artificial agents is a must. In this project we study and develop a visual-processing software that will work as the “eyes” of a cognitive robot. This software performs a three-dimensional mapping of the robot’s environment, providing it with the essential information required to make proper decisions during its navigation. Due to the complexity of this objective we have adopted the Scrum methodology in order to achieve an agile development process, which has allowed us to correct and improve in a fast way the successive versions of the product. The present project is structured in Sprints, which cover the different stages of the software development based on the requirements imposed by the robot and its real necessities. We have initially explored different commercial devices oriented to the acquisition of the required visual information, adopting the Kinect Sensor camera (Microsoft) as the most suitable option. Later on, we have studied the available software to manage the obtained visual information as well as its integration with the robot’s software, choosing the high-level platform Matlab as the common nexus to join the management of the camera, the management of the robot and the implementation of the behavioral algorithms. During the last stages the software has been developed to include the fundamental functionalities required to process the real environment, such as depth representation, segmentation, and clustering. Finally the software has been optimized to exhibit real-time processing and a suitable performance to fulfill the robot’s requirements during its operation in real situations.

Luz industrial e imagen tecnificada: de Moholy Nagy al C.A.V.S. (Center for Advanced Visual Studies)

El desarrollo de la tecnología de la luz implicará la transformación de la vida social, cultural y económica. Tanto las consideraciones espaciales del Movimiento Moderno, como los efectos producidos por la segunda Guerra Mundial, tendrán efectos visibles en las nuevas configuraciones espaciales y en la relación simbiótica y recíproca que se dará entre ideología y tecnología. La transformación en la comprensión de la articulación espacial, asociada al desarrollo tecnológico, afectará al modo en que este espacio es experimentado y percibido. El espacio expositivo y el espacio escénico se convertirán en laboratorio práctico donde desarrollar y hacer comprensible todo el potencial ilusorio de la luz, la proyección y la imagen, como parámetros modificadores y dinamizadores del espacio arquitectónico. Esta experimentación espacial estará precedida por la investigación y creación conceptual en el mundo plástico, donde los nuevos medios mecánicos serán responsables de la construcción de una nueva mirada moderna mediatizada por los elementos técnicos. La experimentación óptica, a través de la fotografía, el cine, o el movimiento de la luz y su percepción, vinculada a nuevos modos de representación y comunicación, se convertirá en elemento fundamental en la configuración espacial. Este ámbito de experimentación se hará patente en la Escuela de la Bauhaus, de la mano de Gropius, Schlemmer o Moholy Nagy entre otros; tanto en reflexiones teóricas como en el desarrollo de proyectos expositivos, arquitectónicos o teatrales, que evolucionarán en base a la tecnología y la modificación de la relación con el espectador. El espacio expositivo y el espacio escénico se tomarán como oportunidad de investigación espacial y de análisis de los modos de percepción, convirtiéndose en lugares de experimentación básicos para el aprendizaje. El teatro se postula como punto de encuentro entre el arte y la técnica, cobrando especial importancia la intersección con otras disciplinas en la definición espacial. Las múltiples innovaciones técnicas ligadas a los nuevos fundamentos teatrales en la modificación de la relación con la escena, que se producen a principios del siglo XX, tendrán como consecuencia la transformación del espacio en un espacio dinámico, tanto física como perceptivamente, que dará lugar a nuevas concepciones espaciales, muchas de ellas utópicas. La luz, la proyección y la creación de ilusión en base a estímulos visuales y sonoros, aparecen como elementos proyectuales efímeros e inmateriales, que tendrán una gran incidencia en el espacio y su modo de ser experimentado. La implicación de la tecnología en el arte conllevará modificaciones en la visualización, así como en la configuración espacial de los espacios destinados a esta. Destacaremos como propuesta el Teatro Total de Walter Gropius, en cuyo desarrollo se recogen de algún modo las experiencias espaciales y las investigaciones desarrolladas sobre la estructura formal de la percepción realizadas por Moholy Nagy, además de los conceptos acerca del espacio escénico desarrollados en el taller de Teatro de la Bauhaus por Oskar Schlemmer. En el Teatro Total, Gropius incorporará su propia visión de cuestiones que pertenecen a la tradición de la arquitectura teatral y las innovaciones conceptuales que estaban teniendo lugar desde finales del s.XIX, tales como la participación activa del público o la superación entre escena y auditorio, estableciendo en el proyecto una nueva relación perceptual entre sala, espectáculo y espectador; aumentando la sensación de inmersión, a través del uso de la física, la óptica, y la acústica, creando una energía concéntrica capaz de extenderse en todas direcciones. El Teatro Total será uno de los primeros ejemplos en los que desde el punto de partida del proyecto, se conjuga la imagen como elemento comunicativo con la configuración espacial. Las nuevas configuraciones escénicas tendrán como premisa de desarrollo la capacidad de transformación tanto perceptiva, como física. En la segunda mitad del s.XX, la creación de centros de investigación como el CAVS (The Center for Advanced Visual Studies,1967), o el EAT (Experiments in Art and Technology, 1966), favorecerán la colaboración interdisciplinar entre arte y ciencia, implicando a empresas de carácter tecnológico, como Siemens, HP, IBM o Philips, facilitando soporte técnico y económico para el desarrollo de nuevos sistemas. Esta colaboración interdisciplinar dará lugar a una serie de intervenciones espaciales que tendrán su mayor visibilidad en algunas Exposiciones Universales. El resultado será, en la mayoría de los casos, la creación de espacios de carácter inmersivo, donde se establecerá una relación simbiótica entre espacio, imagen, sonido, y espectador. La colocación del espectador en el centro de la escena y la disposición dinámica de imagen y sonido, crearán una particular narrativa espacial no lineal, concebida para la experiencia. Desde las primeras proyecciones de cine a la pantalla múltiple de los Eames, las técnicas espaciales de difusión del sonido en Stockhausen, o los experimentos con el movimiento físico interactivo, la imagen, la luz en movimiento y el sonido, quedan inevitablemente convertidos en material arquitectónico. ABSTRACT. Light technology development would lead to a social, cultural and economic transformation. Both spatial consideration of “Modern Movement” and Second World War effects on technology, would have a visible aftereffect on spatial configuration and on the symbiotic and mutual relationship between ideology & technology. Comprehension adjustment on the articulation of space together with technology development, would impact on how space is perceived and felt. Exhibition space and scenic space would turn into a laboratory where developing and making comprehensive all illusory potential of light, projection and image. These new parameters would modify and revitalize the architectonic space. as modifying and revitalizing parameters of architectonic space. Spatial experimentation would be preceded by conceptual creation and investigation on the sculptural field, where new mechanic media would be responsible for a fresh and modern look influenced by technical elements. Optical experimentation, through photography, cinema or light movement and its perception, would turn into essential components for spatial arrangement linked to new ways of performance and communication. This experimentation sphere would be clear at The Bauhaus School, by the hand of Gropius, Schlemmer or Moholy Nag among others; in theoretical, theatrical or architectural performance’s projects, that would evolve based on technology and also based on the transformation of the relationship with the observer. Exhibition and perfor-mance areas would be taken as opportunities of spatial investigation and for the analysis of the different ways of perception, thus becoming key places for learning. Theater is postulated as a meeting point between art and technique, taking on a new significance at its intersection with other disciplines working with spatial definition too. The multiple innovation techniques linked to the new foundations for the theater regarding stage relation, would have as a consequence the regeneration of the space. Space would turn dynamic, both physically and perceptibly, bringing innovative spatial conceptions, many of them unrealistic. Light, projection and illusory creation based on sound and visual stimulus would appear as intangible and momentary design components, which would have a great impact on the space and on the way it is experienced. Implication of technology in art would bring changes on the observer as well as on the spatial configuration of the art spaces2. It would stand out as a proposal Walter Groupis Total Theater, whose development would include somehow the spatial experiments and studies about formal structure of perception accomplished by Moholy Nagy besides the concepts regarding stage space enhanced at the Bauhaus Theater Studio by Oskar Schlemmer. Within Total Theater, Groupis would incorporate his own view about traditional theatric architecture and conceptual innovations that were taking place since the end of the nineteenth century, such as active audience participation or the diffusing limits between scene and audience, establishing a new perception relationship between auditorium, performance and audience, improving the feeling of immersion through the use of physics, optics and acoustics, creating a concentric energy capable of spreading in all directions. Total Theater would be one of the first example in which, from the beginning of the Project, image is combined as a communicating element with the spatial configuration. As a premise of development, new stage arrangement would have the capacity of transformation, both perceptive and physically. During the second half or the twentieth century, the creation of investigation centers such as CAVS (Center for Advanced Visual Studies, 1967) or EAT (Experiments in Art and Technology, 1966), would help to the interdisciplinary collaboration between art and science, involving technology companies like Siemens, HP, IBM or Philips, providing technical and economic support to the development of new systems. This interdisciplinary collaboration would give room to a series of spatial interventions which would have visibility in some Universal Exhibitions. The result would be, in most cases, the creation of immersive character spaces, where a symbiotic relationship would be stablished between space, image, sound and audience. The new location of the audience in the middle of the display, together with the dynamic arrangement of sound and image would create a particular, no lineal narrative conceived to be experienced. Since the first cinema projections, the multiple screen of Eames, the spatial techniques for sound dissemination at Stockhausen or the interactive physical movement experimentation, image, motion light and sound would turn inevitably into architectural material.

Visual perception for the 3D recognition of geometric pieces in robotic manipulation

During grasping and intelligent robotic manipulation tasks, the camera position relative to the scene changes dramatically because the robot is moving to adapt its path and correctly grasp objects. This is because the camera is mounted at the robot effector. For this reason, in this type of environment, a visual recognition system must be implemented to recognize and “automatically and autonomously” obtain the positions of objects in the scene. Furthermore, in industrial environments, all objects that are manipulated by robots are made of the same material and cannot be differentiated by features such as texture or color. In this work, first, a study and analysis of 3D recognition descriptors has been completed for application in these environments. Second, a visual recognition system designed from specific distributed client-server architecture has been proposed to be applied in the recognition process of industrial objects without these appearance features. Our system has been implemented to overcome problems of recognition when the objects can only be recognized by geometric shape and the simplicity of shapes could create ambiguity. Finally, some real tests are performed and illustrated to verify the satisfactory performance of the proposed system.

3D Visual Data-Driven Spatiotemporal Deformations for Non-Rigid Object Grasping Using Robot Hands

Sensing techniques are important for solving problems of uncertainty inherent to intelligent grasping tasks. The main goal here is to present a visual sensing system based on range imaging technology for robot manipulation of non-rigid objects. Our proposal provides a suitable visual perception system of complex grasping tasks to support a robot controller when other sensor systems, such as tactile and force, are not able to obtain useful data relevant to the grasping manipulation task. In particular, a new visual approach based on RGBD data was implemented to help a robot controller carry out intelligent manipulation tasks with flexible objects. The proposed method supervises the interaction between the grasped object and the robot hand in order to avoid poor contact between the fingertips and an object when there is neither force nor pressure data. This new approach is also used to measure changes to the shape of an object’s surfaces and so allows us to find deformations caused by inappropriate pressure being applied by the hand’s fingers. Test was carried out for grasping tasks involving several flexible household objects with a multi-fingered robot hand working in real time. Our approach generates pulses from the deformation detection method and sends an event message to the robot controller when surface deformation is detected. In comparison with other methods, the obtained results reveal that our visual pipeline does not use deformations models of objects and materials, as well as the approach works well both planar and 3D household objects in real time. In addition, our method does not depend on the pose of the robot hand because the location of the reference system is computed from a recognition process of a pattern located place at the robot forearm. The presented experiments demonstrate that the proposed method accomplishes a good monitoring of grasping task with several objects and different grasping configurations in indoor environments.

Time-series Luminance Distribution Maps: implementation of annual daylight simulation methods for occupant visual comfort analysis

Thesis (Master's)--University of Washington, 2016-06

The effect of auditory and visual distracters on the useful field of view: Implications for the driving task

PURPOSE. The driving environment is becoming increasingly complex, including both visual and auditory distractions within the in- vehicle and external driving environments. This study was designed to investigate the effect of visual and auditory distractions on a performance measure that has been shown to be related to driving safety, the useful field of view. METHODS. A laboratory study recorded the useful field of view in 28 young visually normal adults (mean 22.6 +/- 2.2 years). The useful field of view was measured in the presence and absence of visual distracters (of the same angular subtense as the target) and with three levels of auditory distraction (none, listening only, listening and responding). RESULTS. Central errors increased significantly (P < 0.05) in the presence of auditory but not visual distracters, while peripheral errors increased in the presence of both visual and auditory distracters. Peripheral errors increased with eccentricity and were greatest in the inferior region in the presence of distracters. CONCLUSIONS. Visual and auditory distracters reduce the extent of the useful field of view, and these effects are exacerbated in inferior and peripheral locations. This result has significant ramifications for road safety in an increasingly complex in-vehicle and driving environment.

Sensory Processing in Aquatic Environments

Research on sensory processing or the way animals see, hear, smell, taste, feel and electrically and magnetically sense their environment has advanced a great deal over the last fifteen years. This book discusses the most important themes that have emerged from recent research and provides a summary of likely future directions. The book starts with two sections on the detection of sensory signals over long and short ranges by aquatic animals, covering the topics of navigation, communication, and finding food and other localized sources. The next section, the co-evolution of signal and sense, deals with how animals decide whether the source is prey, predator or mate by utilizing receptors that have evolved to take full advantage of the acoustical properties of the signal. Organisms living in the deep-sea environment have also received a lot of recent attention, so the next section deals with visual adaptations to limited light environments where sunlight is replaced by bioluminescence and the visual system has undergone changes to optimize light capture and sensitivity. The last section on central co-ordination of sensory systems covers how signals are processed and filtered for use by the animal. This book will be essential reading for all researchers and graduate students interested in sensory systems.

Meaningful Interaction in Virtual Learning Environments

There is still a great deal of opportunity for research on contextual interactive immersion in virtual heritage environments. The general failure of virtual environment technology to create engaging and educational experiences may be attributable not just to deficiencies in technology or in visual fidelity, but also to a lack of contextual and performative-based interaction, such as that found in games. However, there is little written so far on exactly how game-style interaction can help improve virtual learning environments.

Assessment of the oculomotor response in human factor environments

The need to measure the response of the oculomotor system, such as ocular accommodation, accurately and in real-world environments is essential. New instruments have been developed over the past 50 years to measure eye focus including the extensively utilised and well validated Canon R-1, but in general these have had limitations such as a closed field-of-view, a poor temporal resolution and the need for extensive instrumentation bulk preventing naturalistic performance of environmental tasks. The use of photoretinoscopy and more specifically the PowerRefractor was examined in this regard due to its remote nature, binocular measurement of accommodation, eye movement and pupil size and its open field-of-view. The accuracy of the PowerRefractor to measure refractive error was on averaging similar, but more variable than subjective refraction and previously validated instrumentation. The PowerRefractor was found to be tolerant to eye movements away from the visual axis, but could not function with small pupil sizes in brighter illumination. The PowerRefractor underestimated the lead of accommodation and overestimated the slope of the accommodation stimulus response curve. The PowerRefractor and the SRW-5000 were used to measure the oculomotor responses in a variety of real-world environment: spectacles compared to single vision contract lenses; the use of multifocal contact lenses by pre-presbyopes (relevant to studies on myopia retardation); and ‘accommodating’ intraocular lenses. Due to the accuracy concerns with the PowerRefractor, a purpose-built photoretinoscope was designed to measure the oculomotor response to a monocular head-mounted display. In conclusion, this thesis has shown the ability of photoretinoscopy to quantify changes in the oculomotor system. However there are some major limitations to the PowerRefractor, such as the need for individual calibration for accurate measures of accommodation and vergence, and the relatively large pupil size necessary for measurement.

A penny for your thoughts! Patterns of fMRI activity reveal the content and the spatial topography of visual mental images

Visual mental imagery is a complex process that may be influenced by the content of mental images. Neuropsychological evidence from patients with hemineglect suggests that in the imagery domain environments and objects may be represented separately and may be selectively affected by brain lesions. In the present study, we used functional magnetic resonance imaging (fMRI) to assess the possibility of neural segregation among mental images depicting parts of an object, of an environment (imagined from a first-person perspective), and of a geographical map, using both a mass univariate and a multivariate approach. Data show that different brain areas are involved in different types of mental images. Imagining an environment relies mainly on regions known to be involved in navigational skills, such as the retrosplenial complex and parahippocampal gyrus, whereas imagining a geographical map mainly requires activation of the left angular gyrus, known to be involved in the representation of categorical relations. Imagining a familiar object mainly requires activation of parietal areas involved in visual space analysis in both the imagery and the perceptual domain. We also found that the pattern of activity in most of these areas specifically codes for the spatial arrangement of the parts of the mental image. Our results clearly demonstrate a functional neural segregation for different contents of mental images and suggest that visuospatial information is coded by different patterns of activity in brain areas involved in visual mental imagery. Hum Brain Mapp 36:945-958, 2015.

Virtual Environments as an Education Tool for Digital Natives

Report published in the Proceedings of the National Conference on "Education in the Information Society", Plovdiv, May, 2013

Redefinición de las TIC en el museo: del discurso invasivo al inclusivo

Las TIC son inseparables de la museografía in situ e imprescindibles en la museografía en red fija y móvil. En demasiados casos se han instalado prótesis tecnológicas para barnizar de modernidad el espacio cultural, olvidando que la tecnología debe estar al servicio de los contenidos de manera que resulte invisible y perfectamente imbricada con la museografía tradicional. Las interfaces móviles pueden fusionar museo in situ y en red y acompañar a las personas más allá del espacio físico. Esa fusión debe partir de una base de datos narrativa y abierta a obras materiales e inmateriales de otros museos de manera que no se trasladen las limitaciones del museo físico al virtual. En el museo in situ tienen sentido las instalaciones hipermedia inmersivas que faciliten experiencias culturales innovadoras. La interactividad (relaciones virtuales) debe convivir con la interacción (relaciones físicas y personales) y estar al servicio de todas las personas, partiendo de que todas, todos tenemos limitaciones. Trabajar interdisciplinarmente ayuda a comprender mejor el museo para ponerlo al servicio de las personas.

Una aproximación a la realidad de las plataformas virtuales de las universidades españolas: el primer reto para una educación personalizada en personas con discapacidad motórica o visual

El punto de partida para ofrecer una educación universitaria personalizada basada en entornos de educación onlinea personas con diversidad funcional debe centrarse en garantizar la óptima accesibilidad a su contenido web y a los recursos de aprendizaje. En la actualidad, la mayoría de las universidades disponen de plataformas virtuales de educación, con un importante peso para el correcto desarrollo de sus estudios y para la optimización del aprendizaje de su alumnado. Por otro lado, los entornos virtuales suponen un importante punto de acceso a la posible matriculación en dichas universidades del alumnado con discapacidad. Este trabajo tiene dos objetivos. El primero de ellos consiste en definir cuáles son las condiciones óptimas de accesibilidad web de las plataformas virtuales de educación y de los sitios web de las universidades para personas que presentan discapacidad visual o discapacidad motórica, de acuerdo a los estándares internacionales. El segundo objetivo del trabajo persigue ofrecer una breve aproximación al grado de accesibilidad de los sitios web de las universidades españolas para el alumnado con esas dos discapacidades. El trabajo sigue un diseño descriptivo basado en dos elementos de análisis. Por un lado, la revisión de estudios de accesibilidad y diversidad funcional y, por otro, el análisis de la accesibilidad web de los sitios web de las 78 universidades españolas. Se concluye que existen dificultades de accesibilidad web y navegación en los entornos virtuales de las universidades españolas, situación que, por lo tanto, redunda en limitaciones en el aprendizaje de los colectivos de estudiantes con limitaciones.

The Undersea World of the Sound Department: the Construction of Sonic Conventions in Sub-aqua Screen Environments

The availability of new underwater cameras and sub-aqua diving gear in the immediate post-war era opened up exciting possibilities for both narrative and documentary filmmakers. While the visual elements of this new world could now be more easily captured on film, the sound elements of the sub-aqua environment remained more elusive. What did, or should, this undersea world sound like? This article examines the use of sound in the sub-aqua scenes of both fictional and documentary films in the 1950s and asks questions about the methods used in the sonification of these worlds. Comparing the operation of underwater sound and human hearing with the production and post-production strategies used by filmmakers, I seek to identify the emergence of a sound convention and its implications for issues of cinematic realism. Central to this convention is the manipulation of sonic frequencies. The sound strategies adopted also raise questions about the malleability of viewer perspective and sound-image relationship in terms of a realist mode of address. Linked to this is the use of sound to enhance audience experience on an affective level. As well as underpinning cinematic realism, these new sound environments offered fresh experiences to audiences seeking new reasons to visit the cinema in an era of widening forms of entertainment.