Cooperative off-policy prediction of markov decision processes in adaptive networks

We apply diffusion strategies to propose a cooperative reinforcement learning algorithm, in which agents in a network communicate with their neighbors to improve predictions about their environment. The algorithm is suitable to learn off-policy even in large state spaces. We provide a mean-square-error performance analysis under constant step-sizes. The gain of cooperation in the form of more stability and less bias and variance in the prediction error, is illustrated in the context of a classical model. We show that the improvement in performance is especially significant when the behavior policy of the agents is different from the target policy under evaluation.

Bifurcation diagrams for polymer blends with diffuse interfaces in confined and adaptive geometries

Dynamics of binary mixtures such as polymer blends, and fluids near the critical point, is described by the model-H, which couples momentum transport and diffusion of the components [1]. We present an extended version of the model-H that allows to study the combined effect of phase separation in a polymer blend and surface structuring of the film itself [2]. We apply it to analyze the stability of vertically stratified base states on extended films of polymer blends and show that convective transport leads to new mechanisms of instability as compared to the simpler diffusive case described by the Cahn- Hilliard model [3, 4]. We carry out this analysis for realistic parameters of polymer blends used in experimental setups such as PS/PVME. However, geometrically more complicated states involving lateral structuring, strong deflections of the free surface, oblique diffuse interfaces, checkerboard modes, or droplets of a component above of the other are possible at critical composition solving the Cahn Hilliard equation in the static limit for rectangular domains [5, 6] or with deformable free surfaces [6]. We extend these results for off-critical compositions, since balanced overall composition in experiments are unusual. In particular, we study steady nonlinear solutions of the Cahn-Hilliard equation for bidimensional layers with fixed geometry and deformable free surface. Furthermore we distinguished the cases with and without energetic bias at the free surface. We present bifurcation diagrams for off-critical films of polymer blends with free surfaces, showing their free energy, and the L2-norms of surface deflection and the concentration field, as a function of lateral domain size and mean composition. Simultaneously, we look at spatial dependent profiles of the height and concentration. To treat the problem of films with arbitrary surface deflections our calculations are based on minimizing the free energy functional at given composition and geometric constraints using a variational approach based on the Cahn-Hilliard equation. The problem is solved numerically using the finite element method (FEM).

Adaptive protection scheme for MVC-encoded stereoscopic video streaming in IP-based networks

We present an adaptive unequal error protection (UEP) strategy built on the 1-D interleaved parity Application Layer Forward Error Correction (AL-FEC) code for protecting the transmission of stereoscopic 3D video content encoded with Multiview Video Coding (MVC) through IP-based networks. Our scheme targets the minimization of quality degradation produced by packet losses during video transmission in time-sensitive application scenarios. To that end, based on a novel packet-level distortion model, it selects in real time the most suitable packets within each Group of Pictures (GOP) to be protected and the most convenient FEC technique parameters, i.e., the size of the FEC generator matrix. In order to make these decisions, it considers the relevance of the packet, the behavior of the channel, and the available bitrate for protection purposes. Simulation results validate both the distortion model introduced to estimate the importance of packets and the optimization of the FEC technique parameter values.

High Fidelity Models for Near-Earth Object Dynamics

Motivado por los últimos hallazgos realizados gracias a los recientes avances tecnológicos y misiones espaciales, el estudio de los asteroides ha despertado el interés de la comunidad científica. Tal es así que las misiones a asteroides han proliferado en los últimos años (Hayabusa, Dawn, OSIRIX-REx, ARM, AIMS-DART, ...) incentivadas por su enorme interés científico. Los asteroides son constituyentes fundamentales en la evolución del Sistema Solar, son además grandes concentraciones de valiosos recursos naturales, y también pueden considerarse como objectivos estratégicos para la futura exploración espacial. Desde hace tiempo se viene especulando con la posibilidad de capturar objetos próximos a la Tierra (NEOs en su acrónimo anglosajón) y acercarlos a nuestro planeta, permitiendo así un acceso asequible a los mismos para estudiarlos in-situ, explotar sus recursos u otras finalidades. Por otro lado, las asteroides se consideran con frecuencia como posibles peligros de magnitud planetaria, ya que impactos de estos objetos con la Tierra suceden constantemente, y un asteroide suficientemente grande podría desencadenar eventos catastróficos. Pese a la gravedad de tales acontecimientos, lo cierto es que son ciertamente difíciles de predecir. De hecho, los ricos aspectos dinámicos de los asteroides, su modelado complejo y las incertidumbres observaciones hacen que predecir su posición futura con la precisión necesaria sea todo un reto. Este hecho se hace más relevante cuando los asteroides sufren encuentros próximos con la Tierra, y más aún cuando estos son recurrentes. En tales situaciones en las cuales fuera necesario tomar medidas para mitigar este tipo de riesgos, saber estimar con precisión sus trayectorias y probabilidades de colisión es de una importancia vital. Por ello, se necesitan herramientas avanzadas para modelar su dinámica y predecir sus órbitas con precisión, y son también necesarios nuevos conceptos tecnológicos para manipular sus órbitas llegado el caso. El objetivo de esta Tesis es proporcionar nuevos métodos, técnicas y soluciones para abordar estos retos. Las contribuciones de esta Tesis se engloban en dos áreas: una dedicada a la propagación numérica de asteroides, y otra a conceptos de deflexión y captura de asteroides. Por lo tanto, la primera parte de este documento presenta novedosos avances de apliación a la propagación dinámica de alta precisión de NEOs empleando métodos de regularización y perturbaciones, con especial énfasis en el método DROMO, mientras que la segunda parte expone ideas innovadoras para la captura de asteroides y comenta el uso del “ion beam shepherd” (IBS) como tecnología para deflectarlos. Abstract Driven by the latest discoveries enabled by recent technological advances and space missions, the study of asteroids has awakened the interest of the scientific community. In fact, asteroid missions have become very popular in the recent years (Hayabusa, Dawn, OSIRIX-REx, ARM, AIMS-DART, ...) motivated by their outstanding scientific interest. Asteroids are fundamental constituents in the evolution of the Solar System, can be seen as vast concentrations of valuable natural resources, and are also considered as strategic targets for the future of space exploration. For long it has been hypothesized with the possibility of capturing small near-Earth asteroids and delivering them to the vicinity of the Earth in order to allow an affordable access to them for in-situ science, resource utilization and other purposes. On the other side of the balance, asteroids are often seen as potential planetary hazards, since impacts with the Earth happen all the time, and eventually an asteroid large enough could trigger catastrophic events. In spite of the severity of such occurrences, they are also utterly hard to predict. In fact, the rich dynamical aspects of asteroids, their complex modeling and observational uncertainties make exceptionally challenging to predict their future position accurately enough. This becomes particularly relevant when asteroids exhibit close encounters with the Earth, and more so when these happen recurrently. In such situations, where mitigation measures may need to be taken, it is of paramount importance to be able to accurately estimate their trajectories and collision probabilities. As a consequence, advanced tools are needed to model their dynamics and accurately predict their orbits, as well as new technological concepts to manipulate their orbits if necessary. The goal of this Thesis is to provide new methods, techniques and solutions to address these challenges. The contributions of this Thesis fall into two areas: one devoted to the numerical propagation of asteroids, and another to asteroid deflection and capture concepts. Hence, the first part of the dissertation presents novel advances applicable to the high accuracy dynamical propagation of near-Earth asteroids using regularization and perturbations techniques, with a special emphasis in the DROMO method, whereas the second part exposes pioneering ideas for asteroid retrieval missions and discusses the use of an “ion beam shepherd” (IBS) for asteroid deflection purposes.

A computational model of the anterior intraparietal area

The monkey anterior intraparietal area (AIP) encodes visual information about three-dimensional object shape that is used to shape the hand for grasping. In robotics a similar role has been played by modules that fit point cloud data to the superquadric family of shapes and its various extensions. We developed a model of shape tuning in AIP based on cosine tuning to superquadric parameters. However, the model did not fit the data well, and we also found that it was difficult to accurately reproduce these parameters using neural networks with the appropriate inputs (modelled on the caudal intraparietal area, CIP). The latter difficulty was related to the fact that there are large discontinuities in the superquadric parameters between very similar shapes. To address these limitations we adopted an alternative shape parameterization based on an Isomap nonlinear dimension reduction. The Isomap was built using gradients and curvatures of object surface depth. This alternative parameterization was low-dimensional (like superquadrics), but data-driven (similar to an alternative clustering approach that is also sometimes used in robotics) and lacked large discontinuities. Isomaps with 16 or more dimensions reproduced the AIP data fairly well. Moreover, we found that the Isomap parameters could be approximated from CIP-like input much more accurately than the superquadric parameters. We conclude that Isomaps, or perhaps alternative dimension reductions of CIP signals, provide a promising model of AIP tuning. We have now started to integrate our model with a robot hand, to explore the efficacy of Isomap shape reductions in grasp planning. Future work will consider dynamics of spike responses and integration with related visual and motor area models.

A flexible open source web platform to facilitate Learning Object evaluation

Systematic evaluation of Learning Objects is essential to make high quality Web-based education possible. For this reason, several educational repositories and e-Learning systems have developed their own evaluation models and tools. However, the differences of the context in which Learning Objects are produced and consumed suggest that no single evaluation model is sufficient for all scenarios. Besides, no much effort has been put in developing open tools to facilitate Learning Object evaluation and use the quality information for the benefit of end users. This paper presents LOEP, an open source web platform that aims to facilitate Learning Object evaluation in different scenarios and educational settings by supporting and integrating several evaluation models and quality metrics. The work exposed in this paper shows that LOEP is capable of providing Learning Object evaluation to e-Learning systems in an open, low cost, reliable and effective way. Possible scenarios where LOEP could be used to implement quality control policies and to enhance search engines are also described. Finally, we report the results of a survey conducted among reviewers that used LOEP, showing that they perceived LOEP as a powerful and easy to use tool for evaluating Learning Objects.

Cooperative Learning Model based on Multi-Agent Architecture for Embedded Intelligent Systems

Cooperative systems are suitable for many types of applications and nowadays these system are vastly used to improve a previously defined system or to coordinate multiple devices working together. This paper provides an alternative to improve the reliability of a previous intelligent identification system. The proposed approach implements a cooperative model based on multi-agent architecture. This new system is composed of several radar-based systems which identify a detected object and transmit its own partial result by implementing several agents and by using a wireless network to transfer data. The proposed topology is a centralized architecture where the coordinator device is in charge of providing the final identification result depending on the group behavior. In order to find the final outcome, three different mechanisms are introduced. The simplest one is based on majority voting whereas the others use two different weighting voting procedures, both providing the system with learning capabilities. Using an appropriate network configuration, the success rate can be improved from the initial 80% up to more than 90%.

RESSIST: a research object-based recommender system

The scientific method is a methodological approach to the process of inquiry { in which empirically grounded theory of nature is constructed and verified [14]. It is a hard, exhaustive and dedicated multi-stage procedure that a researcher must perform to achieve valuable knowledge. Trying to help researchers during this process, a recommender system, intended as a researcher assistant, is designed to provide them useful tools and information for each stage of the procedure. A new similarity measure between research objects and a representational model, based on domain spaces, to handle them in dif ferent levels are created as well as a system to build them from OAI-PMH (and RSS) resources. It tries to represents a sound balance between scientific insight into individual scientific creative processes and technical implementation using innovative technologies in information extraction, document summarization and semantic analysis at a large scale.

Self-adaptive strategy based on fuzzy control systems for improving performance in wireless sensors networks

The solutions to cope with new challenges that societies have to face nowadays involve providing smarter daily systems. To achieve this, technology has to evolve and leverage physical systems automatic interactions, with less human intervention. Technological paradigms like Internet of Things (IoT) and Cyber-Physical Systems (CPS) are providing reference models, architectures, approaches and tools that are to support cross-domain solutions. Thus, CPS based solutions will be applied in different application domains like e-Health, Smart Grid, Smart Transportation and so on, to assure the expected response from a complex system that relies on the smooth interaction and cooperation of diverse networked physical systems. The Wireless Sensors Networks (WSN) are a well-known wireless technology that are part of large CPS. The WSN aims at monitoring a physical system, object, (e.g., the environmental condition of a cargo container), and relaying data to the targeted processing element. The WSN communication reliability, as well as a restrained energy consumption, are expected features in a WSN. This paper shows the results obtained in a real WSN deployment, based on SunSPOT nodes, which carries out a fuzzy based control strategy to improve energy consumption while keeping communication reliability and computational resources usage among boundaries.

Advanced Control Strategies for a 6 DoF Hydraulic Parallel Robot Based on the Dynamic Model

Nowadays robots have made their way into real applications that were prohibitive and unthinkable thirty years ago. This is mainly due to the increase in power computations and the evolution in the theoretical field of robotics and control. Even though there is plenty of information in the current literature on this topics, it is not easy to find clear concepts of how to proceed in order to design and implement a controller for a robot. In general, the design of a controller requires of a complete understanding and knowledge of the system to be controlled. Therefore, for advanced control techniques the systems must be first identified. Once again this particular objective is cumbersome and is never straight forward requiring of great expertise and some criteria must be adopted. On the other hand, the particular problem of designing a controller is even more complex when dealing with Parallel Manipulators (PM), since their closed-loop structures give rise to a highly nonlinear system. Under this basis the current work is developed, which intends to resume and gather all the concepts and experiences involve for the control of an Hydraulic Parallel Manipulator. The main objective of this thesis is to provide a guide remarking all the steps involve in the designing of advanced control technique for PMs. The analysis of the PM under study is minced up to the core of the mechanism: the hydraulic actuators. The actuators are modeled and experimental identified. Additionally, some consideration regarding traditional PID controllers are presented and an adaptive controller is finally implemented. From a macro perspective the kinematic and dynamic model of the PM are presented. Based on the model of the system and extending the adaptive controller of the actuator, a control strategy for the PM is developed and its performance is analyzed with simulation.

Assessing the accessibility impact of transport policy by a land-use and transport interaction model - The case of Madrid

Accessibility is an essential concept widely used to evaluate the impact of transport and land-use strategies in urban planning and policy making. Accessibility is typically evaluated by using separately a transport model or a land-use model. This paper embeds two accessibility indicators (i.e., potential and adaptive accessibility) in a land use and transport interaction (LUTI) model in order to assess transport policies implementation. The first aim is to define the adaptive accessibility, considering the competition factor at territorial level (e.g. workplaces and workers). The second aim is to identify the optimal implementation scenario of policy measures using potential and adaptive accessibility indicators. The analysis of the results in terms of social welfare and accessibility changes closes the paper. Two transport policy measures are applied in Madrid region: a cordon toll and increase bus frequency. They have been simulated through the MARS model (Metropolitan Activity Relocation Simulator, i.e. LUTI model). An optimisation procedure is performed by MARS for maximizing the value of the objective function in order to find the optimal policy implementation (first best). Both policy measures are evaluated in terms of accessibility. Results show that the introduction of the accessibility indicators (potential and adaptive) influence the optimal value of the toll price and bus frequency level, generating different results in terms of social welfare. Mapping the difference between potential and adaptive accessibility indicator shows that the main changes occur in areas where there is a strong competition among different land-use opportunities.

Objetos tenaces : adaptive reuse en Meatpacking, 1970–1985

OBJETOS TENACES versa sobre la práctica del adaptive reuse (la “reutilización adaptativa”, la reconversión) explicado a través del fenómeno de ruptura, suspensión y reconfiguración de vínculos que el objeto arquitectónico tiene con respecto a su medio económico, sociocultural, normativo y de utilitas. Se trata de un fenómeno en tres fases, muy próximo al mecanismo empleado por el readymade y por el display, que en arquitectura parte de la crisis que supone un cambio de circunstancias, un desarraigo del objeto respecto de su ámbito anterior. El mecanismo tiene vocación de intervención mínima, desde un posicionamiento actual que comprende el mundo construido como una realidad “ya hecha”, y donde las operaciones de proyecto son acciones de transformación que llevan de una situación de obsolescencia a otra de vigencia renovada, es decir, de una realidad completa a otra igualmente completa. Para ello, la investigación se sirve de tres ejemplos de un mismo entorno geográfico (el área en torno a Meatpacking, en el lado oeste del bajo Manhattan, Nueva York) y temporal (las últimas décadas del siglo XX, en las que los casos se solapan a pesar de tener desarrollos cronológicos diversos). Bell Telephone Laboratories, The West Side Improvement y The National Biscuit Company, reconvertidos a Westbeth, a The High Line y a Chelsea Market, construyen juntos el argumento por suma de conceptos, por adición de fenómenos, estudiando las cualidades y las circunstancias que hacen posible la continuidad de las estructuras, es decir, su supervivencia. El texto forma un corpus que proporciona algunas de las claves del objeto denominado como tenaz a través del análisis sincrónico del antes y el después de su puesta en crisis. A tal efecto, se exponen una serie de figuras razonadas entre las cuales figuran el lijado del barniz, los hilos compartidos, la máscara, la momificación y la vitrina, el puente sobre le desfiladero, el enfriamiento, o las reliquias, incrustaciones y adherencias, todas ellas encaminadas a explicar el proceso de renovación del artefacto como un cambio en el sentido, en el significado, y no tanto en lo material. Entre las consideraciones de la tesis está, por un lado, el carácter relativo del término tenacidad, concepto que define a los objetos que se encuentran y se reconocen en su idoneidad, que tienen capacidad de adaptación, y que precisan finalmente de grandes dosis de pragmatismo para su reconfiguración, siendo todas éstas condiciones necesarias pero sólo suficientes a posteriori. Por otro lado, el texto reflexiona acerca del intangible que comparten todos los objetos así llamados y que consiste en la condición, quizá formidable, de ser reconocibles por parte de los sucesivos intervinientes de su historia, convencidos de que su renovación supone una oportunidad; su desaparición, una pérdida. Finalmente, la investigación apunta hacia el concepto de anacronismo como justificación para los objetos que se demuestran adaptables a épocas sucesivas, y que encajan en el tiempo de forma compleja. ABSTRACT TENACIOUS OBJECTS deals with the practice of the adaptive reuse explained by means of the phenomenon of rupture, suspension and reconfiguration of links that the architectural object has vis-à-vis its economic, socio-cultural and regulatory environment, and its utilitas. It is a phenomenon structured in three stages, very close to the mechanism used by the readymade and by the display, which in architecture arise from the crisis that leads to a change in circumstances, an alienation of the object regarding its previous environment. The mechanism aims to the minimum intervention, from a current positioning which understands the world built as a reality “already made”, and where the operations of the project are transformative actions that take us from a situation of obsolescence to a situation of renewed validity, that is, from a complete reality to one just as complete. For this purpose, the research makes use of three examples in the same geographical area (the Meatpacking Discrict, in the Lower West Side of Manhattan, New York) and the same period (the last decades of the 20th century, in which the cases overlap in spite of their diverse chronological developments). Bell Telephone Laboratories, The West Side Improvement and The National Biscuit Company, reconverted into Westbeth, The High Line and Chelsea Market, make up, together, the argument as a sum of concepts, and addition of phenomena, studying the qualities and circumstances which make the continuity of the structures possible, that is, their survival. The text forms a corpus that provides some of the keys of the so-called tenacious object by means of the synchronic analysis of before and after its crisis. To this end, a series of reasoned figures is presented, including the sanding of varnish, the shared threads, the mask, the mummification and the display cabinet, the bridge over the gorge, cooling, or the relics, incrustations and adherences, all of them aimed at explaining the renovation process of the artifact as a change in the sense and the meaning, rather than the material. The considerations of this dissertation include, on the one hand, the relative nature of the term tenacity, a concept which defines the objects found and recognized in their suitability, that are capable of adapting and which finally require large doses of pragmatism for their reconfiguration, all these conditions being necessary but only sufficient a posteriori. On the other hand, the text reflects on the intangible shared among all the so-called objects, and which consists of the condition, perhaps extraordinary, of being recognized by the subsequent interveners of their history, convinced that their renovation represents an opportunity; their disappearance, a loss. Finally, the dissertation studies the concept of anachronism as a justification for the objects that prove to adapt to subsequent periods, and which fit in time in a complex manner.

Self-Learning Embedded System for Object Identification in Intelligent Infrastructure Sensors

The emergence of new horizons in the field of travel assistant management leads to the development of cutting-edge systems focused on improving the existing ones. Moreover, new opportunities are being also presented since systems trend to be more reliable and autonomous. In this paper, a self-learning embedded system for object identification based on adaptive-cooperative dynamic approaches is presented for intelligent sensor’s infrastructures. The proposed system is able to detect and identify moving objects using a dynamic decision tree. Consequently, it combines machine learning algorithms and cooperative strategies in order to make the system more adaptive to changing environments. Therefore, the proposed system may be very useful for many applications like shadow tolls since several types of vehicles may be distinguished, parking optimization systems, improved traffic conditions systems, etc.

A feedback-controlled ensemble model of the stress-responsive hypothalamo-pituitary-adrenal axis

The present work develops and implements a biomathematical statement of how reciprocal connectivity drives stress-adaptive homeostasis in the corticotropic (hypothalamo-pituitary-adrenal) axis. In initial analyses with this interactive construct, we test six specific a priori hypotheses of mechanisms linking circadian (24-h) rhythmicity to pulsatile secretory output. This formulation offers a dynamic framework for later statistical estimation of unobserved in vivo neurohormone secretion and within-axis, dose-responsive interfaces in health and disease. Explication of the core dynamics of the stress-responsive corticotropic axis based on secure physiological precepts should help to unveil new biomedical hypotheses of stressor-specific system failure.

A model of multiplicative neural responses in parietal cortex.

Visual responses of neurons in parietal area 7a are modulated by a combined eye and head position signal in a multiplicative manner. Neurons with multiplicative responses can act as powerful computational elements in neural networks. In the case of parietal cortex, multiplicative gain modulation appears to play a crucial role in the transformation of object locations from retinal to body-centered coordinates. It has proven difficult to uncover single-neuron mechanisms that account for neuronal multiplication. Here we show that multiplicative responses can arise in a network model through population effects. Specifically, neurons in a recurrently connected network with excitatory connections between similarly tuned neurons and inhibitory connections between differently tuned neurons can perform a product operation on additive synaptic inputs. The results suggest that parietal responses may be based on this architecture.