Autonomous Acquisition of Natural Situated Communication

An important part of human intelligence, both historically and operationally, is our ability to communicate. We learn how to communicate, and maintain our communicative skills, in a society of communicators – a highly effective way to reach and maintain proficiency in this complex skill. Principles that might allow artificial agents to learn language this way are in completely known at present – the multi-dimensional nature of socio-communicative skills are beyond every machine learning framework so far proposed. Our work begins to address the challenge of proposing a way for observation-based machine learning of natural language and communication. Our framework can learn complex communicative skills with minimal up-front knowledge. The system learns by incrementally producing predictive models of causal relationships in observed data, guided by goal-inference and reasoning using forward-inverse models. We present results from two experiments where our S1 agent learns human communication by observing two humans interacting in a realtime TV-style interview, using multimodal communicative gesture and situated language to talk about recycling of various materials and objects. S1 can learn multimodal complex language and multimodal communicative acts, a vocabulary of 100 words forming natural sentences with relatively complex sentence structure, including manual deictic reference and anaphora. S1 is seeded only with high-level information about goals of the interviewer and interviewee, and a small ontology; no grammar or other information is provided to S1 a priori. The agent learns the pragmatics, semantics, and syntax of complex utterances spoken and gestures from scratch, by observing the humans compare and contrast the cost and pollution related to recycling aluminum cans, glass bottles, newspaper, plastic, and wood. After 20 hours of observation S1 can perform an unscripted TV interview with a human, in the same style, without making mistakes.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Toward Visual Autonomous Ship Board Landing of a VTOL UAV

In this paper we tackle the problem of landing a helicopter autonomously on a ship deck, using as the main sensor, an on-board colour camera. To create a test-bed, we first adequately simulate the movement of a ship landing platform on the Sea, for different Sea States, for different ships, randomly and realistically enough. We use a commercial parallel robot to get this movement. Once we had this, we developed an accurate and robust computer vision system to measure the pose of the helipad with respect to the on-board camera. To deal with the noise and the possible fails of the computer vision, a state estimator was created. With all of this, we are now able to develop and test a controller that closes the loop and finish the autonomous landing task.

A Ground-Truth Video Dataset for the Development and Evaluation of Vision-based Sense-and-Avoid systems

The importance of vision-based systems for Sense-and-Avoid is increasing nowadays as remotely piloted and autonomous UAVs become part of the non-segregated airspace. The development and evaluation of these systems demand flight scenario images which are expensive and risky to obtain. Currently Augmented Reality techniques allow the compositing of real flight scenario images with 3D aircraft models to produce useful realistic images for system development and benchmarking purposes at a much lower cost and risk. With the techniques presented in this paper, 3D aircraft models are positioned firstly in a simulated 3D scene with controlled illumination and rendering parameters. Realistic simulated images are then obtained using an image processing algorithm which fuses the images obtained from the 3D scene with images from real UAV flights taking into account on board camera vibrations. Since the intruder and camera poses are user-defined, ground truth data is available. These ground truth annotations allow to develop and quantitatively evaluate aircraft detection and tracking algorithms. This paper presents the software developed to create a public dataset of 24 videos together with their annotations and some tracking application results.

Online Learning-based Robust Visual Tracking for Autonomous Landing of Unmanned Aerial Vehicles

Autonomous landing is a challenging and important technology for both military and civilian applications of Unmanned Aerial Vehicles (UAVs). In this paper, we present a novel online adaptive visual tracking algorithm for UAVs to land on an arbitrary field (that can be used as the helipad) autonomously at real-time frame rates of more than twenty frames per second. The integration of low-dimensional subspace representation method, online incremental learning approach and hierarchical tracking strategy allows the autolanding task to overcome the problems generated by the challenging situations such as significant appearance change, variant surrounding illumination, partial helipad occlusion, rapid pose variation, onboard mechanical vibration (no video stabilization), low computational capacity and delayed information communication between UAV and Ground Control Station (GCS). The tracking performance of this presented algorithm is evaluated with aerial images from real autolanding flights using manually- labelled ground truth database. The evaluation results show that this new algorithm is highly robust to track the helipad and accurate enough for closing the vision-based control loop.

Autonomous classification models in ubiquitous environments

Stream-mining approach is defined as a set of cutting-edge techniques designed to process streams of data in real time, in order to extract knowledge. In the particular case of classification, stream-mining has to adapt its behaviour to the volatile underlying data distributions, what has been called concept drift. Moreover, it is important to note that concept drift may lead to situations where predictive models become invalid and have therefore to be updated to represent the actual concepts that data poses. In this context, there is a specific type of concept drift, known as recurrent concept drift, where the concepts represented by data have already appeared in the past. In those cases the learning process could be saved or at least minimized by applying a previously trained model. This could be extremely useful in ubiquitous environments that are characterized by the existence of resource constrained devices. To deal with the aforementioned scenario, meta-models can be used in the process of enhancing the drift detection mechanisms used by data stream algorithms, by representing and predicting when the change will occur. There are some real-world situations where a concept reappears, as in the case of intrusion detection systems (IDS), where the same incidents or an adaptation of them usually reappear over time. In these environments the early prediction of drift by means of a better knowledge of past models can help to anticipate to the change, thus improving efficiency of the model regarding the training instances needed. By means of using meta-models as a recurrent drift detection mechanism, the ability to share concepts representations among different data mining processes is open. That kind of exchanges could improve the accuracy of the resultant local model as such model may benefit from patterns similar to the local concept that were observed in other scenarios, but not yet locally. This would also improve the efficiency of training instances used during the classification process, as long as the exchange of models would aid in the application of already trained recurrent models, that have been previously seen by any of the collaborative devices. Which it is to say that the scope of recurrence detection and representation is broaden. In fact the detection, representation and exchange of concept drift patterns would be extremely useful for the law enforcement activities fighting against cyber crime. Being the information exchange one of the main pillars of cooperation, national units would benefit from the experience and knowledge gained by third parties. Moreover, in the specific scope of critical infrastructures protection it is crucial to count with information exchange mechanisms, both from a strategical and technical scope. The exchange of concept drift detection schemes in cyber security environments would aid in the process of preventing, detecting and effectively responding to threads in cyber space. Furthermore, as a complement of meta-models, a mechanism to assess the similarity between classification models is also needed when dealing with recurrent concepts. In this context, when reusing a previously trained model a rough comparison between concepts is usually made, applying boolean logic. The introduction of fuzzy logic comparisons between models could lead to a better efficient reuse of previously seen concepts, by applying not just equal models, but also similar ones. This work faces the aforementioned open issues by means of: the MMPRec system, that integrates a meta-model mechanism and a fuzzy similarity function; a collaborative environment to share meta-models between different devices; a recurrent drift generator that allows to test the usefulness of recurrent drift systems, as it is the case of MMPRec. Moreover, this thesis presents an experimental validation of the proposed contributions using synthetic and real datasets.

Estratos arquitectónicos : conceptos, lógicas, sinopsis

La presente tesis doctoral se centra en la investigación del "estrato arquitectónico", manejando conceptos, lógicas y sinopsis como puntos organizativos, enlazados y superpuestos, que den paso a un marco propositivo que pueda ser simultáneamente articulado e independiente. Una situación que, ante nuevos desafíos, pretende ir más allá de una estricta definición de estrato o de su noción preconcebida descrita desde la superposición de diversos materiales. Todo ello a fin de revelar los diferentes perfiles del estrato y de la estratificación. Para ello, se abre y amplía el campo de estudio proyectual/procesual desdibujando los límites entre disciplinas –geología, filosofía, tecnología, arquitectura–. Y al mismo tiempo, se construye una cartografía del estrato arquitectónico asociada: a una taxonomía descriptora, a nuevas formulaciones, a pensamientos de personajes reconocibles y/o rescatados –como Rowe/Slutzky, Smithson, Parent/Virilio, Koolhaas, MVRDV, etc.–, y, por último, a nuevas lógicas proyectuales y operativas referidas al estrato, que fusionan conocimientos complejos. Unos contenidos que habitualmente no se presentan en investigaciones doctorales tradicionales –lo que permite afrontar nuevos cambios metodológicos, e imprimir un punto de partida para el desarrollo de futuras investigaciones o de otros posibles caminos emergentes–. Así, el discurso se impregna de nuevos referentes teóricos dentro de una misma mitología común (la del estrato). Un tema abordado con claves híbridas, es decir, con estudios selectivos y escritos coleccionados –analizados a partir de antecedentes encontrados y consolidados–, combinados con interpretaciones subyacentes en la arquitectura contemporánea –que pueden todavía encontrarse en fase experimental–. Una investigación que muestra la adaptación, la evolución y el cambio de la definición de "estrato arquitectónico", así como de su apariencia, utilización y aplicación a lo largo del tiempo, consecuencia de las nuevas necesidades y/o de las nuevas bases ideológicas ocasionadas por nuevos descubrimientos. Estratos arquitectónicos que dotan al espacio de profundidad –desde la bidimensionalidad– mediante la superposición de planos paralelos hacia la construcción de una nueva experiencia de estratificación fenomenológica. Estratos que generan estímulos y cambian su materialidad –de opacos a (en)tramados– creando apariencias y produciendo diversos efectos, para posteriormente exfoliarse y convertirse en estratos habitables. Estratos capturadores del espacio-tiempo, o de las dinámicas internas. Llegando incluso, con un último giro, a la era digital, donde el estrato informacional –que continua siendo independiente, autónomo y diverso, y que puede ser modificado, manipulado, suprimido o activable sin alterar el conjunto–, propicia obtener sistemas flexibles, relacionales, capaces de mutar o adaptarse, asociados a distintos niveles de organización dispositiva, que impulsan acciones y hacen emerger lógicas proyectuales. Estratos que surgen de forma elaborada o espontánea en el territorio urbano o natural (ciudad sobre ciudad, suelo sobre suelo), que se aplican extendiendo e hibridando el territorio (paisajes operativos), que son replanteados artificialmente como mecanismos edificatorios según las exigencias existentes (estratos programáticos, capas especializadas), y que se implantan como capas/layers en los programas informáticos utilizados como herramientas de diseño (multicapas, sistemas de intercambio informacional). Un trabajo que se estructura de manera multicapa lo que permite recorrer la tesis como un mapa de secuencias, saltar entre estratos de información, o seleccionar lecturas según intereses de pensamiento o acción arquitectónica. En consecuencia, una investigación sobre el "estrato arquitectónico" que conecta diversas disciplinas, que recopila distintos posicionamientos teórico/prácticos –lo que implica la introducción de parámetros y datos vivos–, y afronta un posicionamiento estratégico que vuelve a poner en valor el estrato arquitectónico –partiendo de estrategias formales que se han transformado en estrategias experimentales/operativas (a medida que el entorno se ha ido complejizando), siguiendo una estrategia conceptual e intelectual contemporánea, propia y específica–. ABSTRACT This PhD thesis is based on the research of "architectural stratum" using concepts, logical and synopsis as organizational, linked and overlapping points, which give way to a proactive framework that can be articulated and independent simultaneously. A situation that, faced with new challenges, aims to go beyond a strict definition of stratum or their preconceived notion described from the superposition of different materials. All this in order to reveal the different profiles of stratum and stratification. To do this, it opens and widens the field of project/process study, blurring the boundaries between disciplines –geology, philosophy, technology, architecture–. And, at the same time, a cartography of the architectural stratum is constructed, that associates with: descriptor taxonomy, new formulations, thoughts of recognizable and/or rescued characters, –as Rowe/Slutzky, Smithson, Parent/Virilio, Koolhaas, MVRDV, etc.–, and finally, a new project and operational logics referred to stratum, which merge complex knowledge. A content not usually presented in traditional doctoral research –which can face new methodological changes, and creates a starting point for future research or other possible emerging paths–. Thus, the work is imbued with new theoretical framework within a common mythology (stratum’s myth). An issue addressed with hybrid keys, that is, with selective studies and collected writings –analyzed starting from found and consolidated backgrounds–, combined with underlying performances on contemporary architecture –which can still be in experimental phase–. A research that shows adaptation, evolution and change of the definition of "architectural stratum", as well as their appearance, use and application throughout time, result of the new requirements and/or the new ideological foundations caused by new discoveries. Architectural strata that give depth to space –from two-dimensionality– by overlapping parallel planes towards the construction of a new experience of phenomenological stratification. Strata that generate stimuli and change their materiality –from opaque to lattice– creating appearances and producing various effects, for later exfoliating and becoming habitable strata. Strata that capture space-time or internal dynamics. Even reaching, with a final twist, the digital age, where the informational stratum –which remains independent, autonomous and diverse, and can be modified, manipulated, deleted or activated without altering the whole–, contribute to obtaining flexible, relational systems, able to mutate or adapt, associated with different levels of regulatory organization, that drive actions and make emerge project logics. Strata that arise elaborately or spontaneously in urban or natural territory (city upon city, floor upon floor), which apply expanding and hybridizing the territory (operational landscapes), which are artificially restated as building mechanisms according to existing requirements (strata program, specialized layers), which are implemented as layers in the software used as design tools (multilayer, systems of informational interchange). A work that is structured in a multi-layered way, which allows explore the thesis as a sequence map, jump between layers of information, or select readings according to an interest in thought or architectural action. Therefore, an investigation into the "architectural stratum" that connects different disciplines, which collects different theoretical/practical positions –which implies the introduction of live parameters and data– and faces a strategic positioning returning to value the architectural stratum –based on formal strategies that have become experimental/operational strategies (as the environment has become more complex), following a conceptual, intellectual, contemporary, own specific strategy–.

Developing Autonomous Microgrids: A Theoretical Case Study

Microgrids are autonomously operated, geographically clustered electricity generation and distribution systems that supply power in closed system settings; they are highly compatible with renewable energy sources and distributed generation technologies. Mocrogrids are currently a serially underutilized and underappreciated commodity in the energy infrastructure portfolio worldwide. To demonstrate feasibility under poor conditions (little renewable energy potential and high demand) this capstone project develops a theoretical case study in which a renewable microgrid is employed to power rural communities of southern Montgomery County, Arkansas. Utilizing commercially manufactured 1.5-megawatt wind turbines and a 1-megawatt solar panel generation system, 4-megawatts of lithium ion battery storage, and demand response technology, a microgrid is designed that supplies 235 households with reliable electricity supply.

Integrated architecture for vision-based indoor localization and mapping of a quadrotor micro-air vehicle

Atualmente os sistemas de pilotagem autónoma de quadricópteros estão a ser desenvolvidos de forma a efetuarem navegação em espaços exteriores, onde o sinal de GPS pode ser utilizado para definir waypoints de navegação, modos de position e altitude hold, returning home, entre outros. Contudo, o problema de navegação autónoma em espaços fechados sem que se utilize um sistema de posicionamento global dentro de uma sala, subsiste como um problema desafiante e sem solução fechada. Grande parte das soluções são baseadas em sensores dispendiosos, como o LIDAR ou como sistemas de posicionamento externos (p.ex. Vicon, Optitrack). Algumas destas soluções reservam a capacidade de processamento de dados dos sensores e dos algoritmos mais exigentes para sistemas de computação exteriores ao veículo, o que também retira a componente de autonomia total que se pretende num veículo com estas características. O objetivo desta tese pretende, assim, a preparação de um sistema aéreo não-tripulado de pequeno porte, nomeadamente um quadricóptero, que integre diferentes módulos que lhe permitam simultânea localização e mapeamento em espaços interiores onde o sinal GPS ´e negado, utilizando, para tal, uma câmara RGB-D, em conjunto com outros sensores internos e externos do quadricóptero, integrados num sistema que processa o posicionamento baseado em visão e com o qual se pretende que efectue, num futuro próximo, planeamento de movimento para navegação. O resultado deste trabalho foi uma arquitetura integrada para análise de módulos de localização, mapeamento e navegação, baseada em hardware aberto e barato e frameworks state-of-the-art disponíveis em código aberto. Foi também possível testar parcialmente alguns módulos de localização, sob certas condições de ensaio e certos parâmetros dos algoritmos. A capacidade de mapeamento da framework também foi testada e aprovada. A framework obtida encontra-se pronta para navegação, necessitando apenas de alguns ajustes e testes.

Hardware/software compatibility for traffic management/IVHS systems.

Texas Department of Transportation, Austin

Hippocampal models for simultaneous localisation and mapping on an autonomous robot

To navigate successfully in a novel environment a robot needs to be able to Simultaneously Localize And Map (SLAM) its surroundings. The most successful solutions to this problem so far have involved probabilistic algorithms, but there has been much promising work involving systems based on the workings of part of the rodent brain known as the hippocampus. In this paper we present a biologically plausible system called RatSLAM that uses competitive attractor networks to carry out SLAM in a probabilistic manner. The system can effectively perform parameter self-calibration and SLAM in onedimension. Tests in two dimensional environments revealed the inability of the RatSLAM system to maintain multiple pose hypotheses in the face of ambiguous visual input. These results support recent rat experimentation that suggest current competitive attractor models are not a complete solution to the hippocampal modelling problem.