Curiosity driven reinforcement learning for motion planning on humanoids

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Urban informatics beyond data: Media architecture, placemaking, and citizen action

Since 2006, we have been conducting urban informatics research that we define as “the study, design, and practice of urban experiences across different urban contexts that are created by new opportunities of real-time, ubiquitous technology and the augmentation that mediates the physical and digital layers of people networks and urban infrastructures” [1]. Various new research initiatives under the label “urban informatics” have been started since then by universities (e.g., NYU’s Center for Urban Science and Progress) and industry (e.g., Arup, McKinsey) worldwide. Yet, many of these new initiatives are limited to what Townsend calls, “data-driven approaches to urban improvement” [2]. One of the key challenges is that any quantity of aggregated data does not easily translate directly into quality insights to better understand cities. In this talk, I will raise questions about the purpose of urban informatics research beyond data, and show examples of media architecture, participatory city making, and citizen activism. I argue for (1) broadening the disciplinary foundations that urban science approaches draw on; (2) maintaining a hybrid perspective that considers both the bird’s eye view as well as the citizen’s view, and; (3) employing design research to not be limited to just understanding, but to bring about actionable knowledge that will drive change for good.

Planning, policy and you

The policy environment for regional natural resource management (NRM) has shifted considerably since it was first introduced in the early 2000s. This workshop will explore the impact of current policy on NRM planning and action through presentations and workshop discussion. To set the scene for the workshop discussion presentations will consider: 1) the impact of evolving national and state NRM policy in Australia; 2) how Australian NRM compares to other countries; 3) governance risks to NRM delivery; and 4) regional responses to NRM delivery. During the workshop element, participants from across regions will share their experiences and explore implications of current policy on the business of regional NRM.

Promoting articulated action from diverse stakeholders in response to public policy scenarios: A case analysis of the use of ‘scenario improvisation’ method

In this paper we present a novel application of scenario methods to engage a diverse constituency of senior stakeholders, with limited time availability, in debate to inform planning and policy development. Our case study project explores post-carbon futures for the Latrobe Valley region of the Australian state of Victoria. Our approach involved initial deductive development of two ‘extreme scenarios’ by a multi-disciplinary research team, based upon an extensive research programme. Over four workshops with the stakeholder constituency, these initial scenarios were discussed, challenged, refined and expanded through an inductive process, whereby participants took ‘ownership’ of a final set of three scenarios. These were both comfortable and challenging to them. The outcomes of this process subsequently informed public policy development for the region. Whilst this process did not follow a single extant structured, multi-stage scenario approach, neither was it devoid of form. Here, we seek to theorise and codify elements of our process – which we term ‘scenario improvisation’ – such that others may adopt it.

Predictive technology and natural hazards: Risk for Australian planning authorities?

Purpose The object of this paper is to examine whether the improvements in technology that enhance community understanding of the frequency and severity of natural hazards also increased the risk of potential liability of planning authorities in negligence. In Australia, the National Strategy imposes a resilience based approach to disaster management and stresses that responsible land use planning can reduce or prevent the impact of natural hazards upon communities. Design/methodology/approach This paper analyses how the principles of negligence allocate responsibility for loss suffered by a landowner in a hazard prone area between the landowner and local government. Findings The analysis in this paper concludes that despite being able to establish a causal link between the loss suffered by a landowner and the approval of a local authority to build in a hazard prone area, it would be in the rarest of circumstances a negligence action may be proven. Research limitations/implications The focus of this paper is on planning policies and land development, not on the negligent provision of advice or information by the local authority. Practical implications This paper identifies the issues a landowner may face when seeking compensation from a local authority for loss suffered due to the occurrence of a natural hazard known or predicted to be possible in the area. Originality/value The paper establishes that as risk managers, local authorities must place reliance upon scientific modelling and predictive technology when determining planning processes in order to fulfil their responsibilities under the National Strategy and to limit any possible liability in negligence.

Simulation-based optimal sensor scheduling with application to observer trajectory planning

The sensor scheduling problem can be formulated as a controlled hidden Markov model and this paper solves the problem when the state, observation and action spaces are continuous. This general case is important as it is the natural framework for many applications. The aim is to minimise the variance of the estimation error of the hidden state w.r.t. the action sequence. We present a novel simulation-based method that uses a stochastic gradient algorithm to find optimal actions. © 2007 Elsevier Ltd. All rights reserved.

Stormsmart coasts in action: Two case studies of local level climate change adaptation

Coastal communities throughout the United States have dealt with the devastating effects of storms for centuries, however today’s threats are greater due to three factors. First, the population along the coastline has grown, and is projected to increase.i Additionally, past land use management decisions in the coastal zone have rarely led to the greatest protection from threats. Finally, climate change is predicted to affect coastal areas by accelerating current sea level rise rates and possibly increasing storm intensity.ii These factors compounded together mean that coastal communities are facing a very dangerous situation that threatens economies and human life. (PDF contains 4 pages)

The marine protected area planning process in Washington state: Recommendations for increased effectiveness

In Washington State, the Department of Natural Resources (WA DNR) is responsible for managing state-owned aquatic lands. Aquatic reserves are one of many Marine Protected Area (MPA) designations in WA State that aim to protect sensitive aquatic and ecological habitat. We analyzed the designation and early planning processes of WA State aquatic reserves, identified gaps in the processes, and recommend action to improve the WA State aquatic reserve early planning approach. (PDF contains 4 pages)

6th European Conference on Energy Efficiency and Sustainability in Architecture and Planning

[ES]Esta obra recoge las comunicaciones seleccionadas para el 6º Congreso Europeo sobre Eficiencia Energética y Sostenibilidad en Arquitectura, organizado por el grupo de investigación Calidad de Vida en Arquitectura de la Universidad del País Vasco/Euskal Herriko Unibertsitatea. El congreso, que se celebra en el marco de los XXXIV Cursos de Verano de la UPV/EHU, aborda en esta cuarta edición el tema “Ciudades en riesgo: resiliencia y redundancia”. Alrededor de este tema general se desarrollan cinco ponencias magistrales, a cargo de Margaretha Breil (Centro Euro-Mediterráneo para el Cambio Climático), Cristina Garzillo Leemhuis (ICLEI), Ignasi Fontanals (OptiCits), Juan Carlos Barrios Montenegro (Global Action Plan) y Manuel Valdés López (Ajuntament de Barcelona). Además, 24 comunicaciones seleccionadas por el comité científico presentarán trabajos de investigaciones actuales en las sesiones orales y póster. Es objetivo paralelo del congreso es fortalecer las líneas de investigación en eficiencia energética y sostenibilidad de los grupos de investigación y formación de la UPV/ EHU comprometidos con esta propuesta, con objeto de colaborar en el reforzamiento de la I D i en su ámbito de conocimiento y apoyar la apuesta específica de los Gobiernos Central y Vasco, así como de otras instituciones nacionales e internacionales respecto a las actividades de I D i en las materias relacionadas con el cambio climático, la eficiencia energética y la sostenibilidad ambiental [ENG] This work contains the selected abstracts of the 6th European Conference on Energy Efficiency and Sustainability in Architecture and Planning, organized by the research group Quality of life in Architecture of the University of the Basque Country. The conference is part of the XXXIV Summer Courses of the UPV/EHU and deals, in its fourth edition, with the topic “Cities at risk: resilience and redundancy”. Around this general theme there are five invited speakers: Margaretha Breil (Euro-Mediterranean Centre for Climate Change), Cristina Garzillo Leemhuis (ICLEI), Ignasi Fontanals (OptiCits), Juan Carlos Barrios Montenegro (Global Action Plan) y Manuel Valdés López (Barcelona City Council). 24 abstracts additional have been selected by the scientific committee that offer actual research works in presentations and posters. The purpose of the conferences is to strengthen the investigation lines in energy efficiency and sustainability of the research and education groups of the University of the Basque Country (UPV/EHU) involved, with the purpose of collaborating in the reinforcement of the I D i in its field of knowledge, and support the specific projects of the Central and Basque Governments, as well as other national and international institutions related to the I Di activities in similar fields of climate change, energy efficiency and environmental sustainability.

Planning for Conjunctive Goals

The problem of achieving conjunctive goals has been central to domain independent planning research; the nonlinear constraint-posting approach has been most successful. Previous planners of this type have been comlicated, heuristic, and ill-defined. I have combined and distilled the state of the art into a simple, precise, implemented algorithm (TWEAK) which I have proved correct and complete. I analyze previous work on domain-independent conjunctive planning; in retrospect it becomes clear that all conjunctive planners, linear and nonlinear, work the same way. The efficiency of these planners depends on the traditional add/delete-list representation for actions, which drastically limits their usefulness. I present theorems that suggest that efficient general purpose planning with more expressive action representations is impossible, and suggest ways to avoid this problem.

Planning humanitarian projects with educational focus in Guinea rural areas

Dissertação apresentada à Universidade Fernando Pessoa como parte dos requisitos para obtenção do grau de Mestre em Acção Humanitária, Cooperação e Desenvolvimento

SOVEREIGN: An Autonomous Neural System for Incrementally Learning Planned Action Sequence to Navigate Towards a Rewarded Goal

How do reactive and planned behaviors interact in real time? How are sequences of such behaviors released at appropriate times during autonomous navigation to realize valued goals? Controllers for both animals and mobile robots, or animats, need reactive mechanisms for exploration, and learned plans to reach goal objects once an environment becomes familiar. The SOVEREIGN (Self-Organizing, Vision, Expectation, Recognition, Emotion, Intelligent, Goaloriented Navigation) animat model embodies these capabilities, and is tested in a 3D virtual reality environment. SOVEREIGN includes several interacting subsystems which model complementary properties of cortical What and Where processing streams and which clarify similarities between mechanisms for navigation and arm movement control. As the animat explores an environment, visual inputs are processed by networks that are sensitive to visual form and motion in the What and Where streams, respectively. Position-invariant and sizeinvariant recognition categories are learned by real-time incremental learning in the What stream. Estimates of target position relative to the animat are computed in the Where stream, and can activate approach movements toward the target. Motion cues from animat locomotion can elicit head-orienting movements to bring a new target into view. Approach and orienting movements are alternately performed during animat navigation. Cumulative estimates of each movement are derived from interacting proprioceptive and visual cues. Movement sequences are stored within a motor working memory. Sequences of visual categories are stored in a sensory working memory. These working memories trigger learning of sensory and motor sequence categories, or plans, which together control planned movements. Predictively effective chunk combinations are selectively enhanced via reinforcement learning when the animat is rewarded. Selected planning chunks effect a gradual transition from variable reactive exploratory movements to efficient goal-oriented planned movement sequences. Volitional signals gate interactions between model subsystems and the release of overt behaviors. The model can control different motor sequences under different motivational states and learns more efficient sequences to rewarded goals as exploration proceeds.

Supporting therapist engagement in evidence based practice: Developing a continuing education programme through participatory action research

Therapists find it challenging to integrate research evidence into their clinical decision-making because it may involve modifying their existing practices. Although continuing education (CE) programmes for evidence-based practice (EBP) have employed various approaches to increase individual practitioner’s knowledge and skills, these have been shown to have little impact in changing customary behaviours. To date, there has been little attempt to actively engage therapists as collaborators in developing educational processes concerning EBP. The researcher collaborated with seven clinical therapists (one occupational therapist, four physiotherapists and two speech and language therapists) enrolled in a new post-qualification Implementing Evidence in Therapy Practice (IETP) MSc module to monitor and adapt the learning programme over ten weeks. The participating therapists actively engaged in participatory action research (PAR) iterative cycles of reflecting→ planning→ acting→ observing→ reflecting with the researcher. Mixed methods were used to evaluate the IETP module and its influence on therapists’ subsequent engagement in EBP activities. Data were gathered immediately on completion of the module and five months later. Immediate post-module findings revealed four components as being important to the therapists: 1) characteristics of the learning environment; 2) acquisition of relevant EBP skills; 3) nature of the learning process; and 4) acquiring confidence. The two themes and sub-themes which emerged from individual interviews conducted five months post-module expanded on the four components already identified. Theme 1: Experiencing the learning (sub-themes: module organisation; learning is relational; improving the module); and theme 2: Enacting the learning through a new way of being (sub-themes: criticality and reflection; self agency; modelling EBP behaviours; positioning self in an EB work culture). The therapists’ perspectives had by then shifted from that of a learner to that of a clinician constructing a new sense of self as an evidence-based practitioner. Findings from this study underline the importance of the process of socially constructed knowledge and of empowering learners through collaboratively designed continuing education programmes. In the student-driven learning environment, therapists chose repetitive skill-building and authentic problem-solving activities which reflected the complexity of the environments to which they were expected to transfer their learning. These findings have implications for educators designing EBP continuing education programmes, during which students develop professional ways of being.

Aligning conservation priorities across taxa in Madagascar with high-resolution planning tools

Globally, priority areas for biodiversity are relatively well known, yet few detailed plans exist to direct conservation action within them, despite urgent need. Madagascar, like other globally recognized biodiversity hot spots, has complex spatial patterns of endemism that differ among taxonomic groups, creating challenges for the selection of within-country priorities. We show, in an analysis of wide taxonomic and geographic breadth and high spatial resolution, that multitaxonomic rather than single-taxon approaches are critical for identifying areas likely to promote the persistence of most species. Our conservation prioritization, facilitated by newly available techniques, identifies optimal expansion sites for the Madagascar government's current goal of tripling the land area under protection. Our findings further suggest that high-resolution multitaxonomic approaches to prioritization may be necessary to ensure protection for biodiversity in other global hot spots.