Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Learning across business sectors: facets of innovation in aerospace and construction

Innovation is notoriously difficult to define and is invariably intertwined with issues of knowledge creation, continuous improvement and organisational change. An extensive literature classifies numerous types of innovation and militates against any simplistic attempt at definition. It is widely accepted that innovation is at least partly dependent upon the surrounding environment. Industry recipes and institutionally embedded practices shape the environment within which innovation occurs. Recent research directions have addressed the diffusion of innovation and its dependence upon social and institutional structures. In this respect, it is highly pertinent to compare the way that innovation is interpreted and enacted in different industrial sectors. The comparison between UK aerospace and construction is especially revealing because the two sectors are so different and therefore constitute radically different climates for innovation. Empirical research is reported based on semi-structured interviews with practitioners from both sectors. Interpretations of innovation are found to differ dramatically between aerospace and construction. Within the context of an ongoing struggle to define innovation, both industries are striving to become more innovative. The aerospace sector is found to emphasise technical innovation whereas the construction sector emphasises process innovation. An overriding cultural bias in Western economies towards technological innovation results in the common perception that aerospace is much more innovative than construction. The experienced realities of practitioners in the two sectors are much more complex.

The symbiotic roles of empirical experimentation and thought experimentation in the learning of physics

This study was an attempt to identify the epistemological roots of knowledge when students carry out hands-on experiments in physics. We found that, within the context of designing a solution to a stated problem, subjects constructed and ran thought experiments intertwined within the processes of conducting physical experiments. We show that the process of alternating between these two modes- empirically experimenting and experimenting in thought- leads towards a convergence on scientifically acceptable concepts. We call this process mutual projection. In the process of mutual projection, external representations were generated. Objects in the physical environment were represented in an imaginary world and these representations were associated with processes in the physical world. It is through this coupling that constituents of both the imaginary world and the physical world gain meaning. We further show that the external representations are rooted in sensory interaction and constitute a semi-symbolic pictorial communication system, a sort of primitive 'language', which is developed as the practical work continues. The constituents of this pictorial communication system are used in the thought experiments taking place in association with the empirical experimentation. The results of this study provide a model of physics learning during hands-on experimentation.

Dynamic automata for mobile robot learning

Researchers at the University of Reading have developed over many years some simple mobile robots that explore an environment they perceive through simple ultrasonic sensors. Information from these sensors has allowed the robots to learn the simple task of moving around while avoiding dynamic obstacles using a static set of fuzzy automata, the choice of which has been criticised, due to its arbitrary nature. This paper considers how a dynamic set of automata can overcome this criticism. In addition, a new reinforcement learning function is outlined which is both scalable to different numbers and types of sensors. The innovations compare successfully with earlier work.

Using learning networks to understand complex systems: a case study of biological, geophysical and social research in the Amazon

Developing high-quality scientific research will be most effective if research communities with diverse skills and interests are able to share information and knowledge, are aware of the major challenges across disciplines, and can exploit economies of scale to provide robust answers and better inform policy. We evaluate opportunities and challenges facing the development of a more interactive research environment by developing an interdisciplinary synthesis of research on a single geographic region. We focus on the Amazon as it is of enormous regional and global environmental importance and faces a highly uncertain future. To take stock of existing knowledge and provide a framework for analysis we present a set of mini-reviews from fourteen different areas of research, encompassing taxonomy, biodiversity, biogeography, vegetation dynamics, landscape ecology, earth-atmosphere interactions, ecosystem processes, fire, deforestation dynamics, hydrology, hunting, conservation planning, livelihoods, and payments for ecosystem services. Each review highlights the current state of knowledge and identifies research priorities, including major challenges and opportunities. We show that while substantial progress is being made across many areas of scientific research, our understanding of specific issues is often dependent on knowledge from other disciplines. Accelerating the acquisition of reliable and contextualized knowledge about the fate of complex pristine and modified ecosystems is partly dependent on our ability to exploit economies of scale in shared resources and technical expertise, recognise and make explicit interconnections and feedbacks among sub-disciplines, increase the temporal and spatial scale of existing studies, and improve the dissemination of scientific findings to policy makers and society at large. Enhancing interaction among research efforts is vital if we are to make the most of limited funds and overcome the challenges posed by addressing large-scale interdisciplinary questions. Bringing together a diverse scientific community with a single geographic focus can help increase awareness of research questions both within and among disciplines, and reveal the opportunities that may exist for advancing acquisition of reliable knowledge. This approach could be useful for a variety of globally important scientific questions.

Working with Dyslexic Learners: Authoring in a Multimedia Environment

The fundamental principles of the teaching methodology followed for dyslexic learners evolve around the need for a multisensory approach, which would advocate repetition of learning tasks in an enjoyable way. The introduction of multimedia technologies in the field of education has supported the merging of new tools (digital camera, scanner) and techniques (sounds, graphics, animation) in a meaningful whole. Dyslexic learners are now given the opportunity to express their ideas using these alternative media and participate actively in the educational process. This paper discussed the preliminary findings of a single case study of two English monolingual dyslexic children working together to create an open-ended multimedia project on a laptop computer. The project aimed to examine whether and if the multimedia environment could enhance the dyslexic learners’ skills in composition. Analysis of the data has indicated that the technological facilities gave the children the opportunity to enhance the style and content of their work for a variety of audiences and to develop responsibilities connected to authorship.

Exploring business and IT alignment mechanisms: toward a learning perspective

Numerous studies have attempted to develop strategic alignment mechanisms. The strategic alignment mechanism is broken down into two categories namely: strategy process and strategy content. Our review shows that alignment research has been carried out in isolation. We see this as having had the effect of limiting the extent to which executives can understand elements of performance. We confer with a number of researchers in postulating that using a mechanism such as multilevel learning to combine strategy content and strategy process under one metaphor can greatly facilitate, through exploration and exploitation, the understanding not only of human interactions within a firm, but also of the interaction existent between a firm and its environment. The findings in this study further support the idea of integrating strategy process and content to have a better understating of alignment maturity and impact on business performance. It also elaborates the affect of misalignment in companies on performance.

Experience of international students from developing countries in negotiating technology for learning

International students are important economically and culturally, bringing diversity and an international perspective enriching learning experiences in classrooms. With the global transformations eLearning has become an important element of students’ higher education experience in developed countries. Although students of developed countries have digital exposure at an early age, many students from developing countries, on the journey of becoming international students, are inadequately prepared for eLearning. The lack of digital skills, prior experience, cultural differences and language barriers together with the drastic changes in learning environments require international students to not only adapt to the host environment but also to negotiate technology for learning. The scarcity of research exploring the eLearning experiences of international students from developing countries and the benefits of this understanding is discussed in an effort to promote research in this area.

Implicit learning and imperceptible influence: Syncretic literacy of multilingual Chinese children.

This article reports on an ethnographic study involving the literacy practices of two multilingual Chinese children from two similar yet different cultural and linguistic contexts: Montreal and Singapore. Using syncretism as a theoretical tool, this inquiry examines how family environment and support facilitate children’s process of becoming literate in multiple languages. Informed by sociocultural theory, the inquiry looks in particular at the role of grandparents in the syncretic literacy practices of children. Through comparative analysis, the study reveals similarities and differences that, when considered together, contribute to our understanding of multilingual children’s creative forms of learning with regard to their rich literacy resources in multiple languages, the imperceptible influences of mediators, various learning styles and syncretic literacy practices.

The development of the Virtual Construction Simulator 3: an interactive simulation environment for construction management education

This paper discusses the development of the Virtual Construction Simulator (VCS) 3 - a simulation game-based educational tool for teaching construction schedule planning and management. The VCS3 simulation game engages students in learning the concepts of planning and managing construction schedules through goal driven exploration, employed strategies, and immediate feedback. Through the planning and simulation mode, students learn the difference between the as-planned and as-built schedules resulting from varying factors such as resource availability, weather and labor productivity. This paper focuses on the development of the VCS3 and its construction physics model. Challenges inherent in the process of identifying variables and their relationships to reliably represent and simulate the dynamic nature of planning and managing of construction projects are also addressed.

Social and ecological drivers of success in agri-environment schemes: the roles of farmers and environmental context

1. Agri-environment schemes remain a controversial approach to reversing biodiversity losses, partly because the drivers of variation in outcomes are poorly understood. In particular, there is a lack of studies that consider both social and ecological factors. 2. We analysed variation across 48 farms in the quality and biodiversity outcomes of agri-environmental habitats designed to provide pollen and nectar for bumblebees and butterflies or winter seed for birds. We used interviews and ecological surveys to gather data on farmer experience and understanding of agri-environment schemes, and local and landscape environmental factors. 3. Multimodel inference indicated social factors had a strong impact on outcomes and that farmer experiential learning was a key process. The quality of the created habitat was affected positively by the farmer’s previous experience in environmental management. The farmer’s confidence in their ability to carry out the required management was negatively related to the provision of floral resources. Farmers with more wildlife-friendly motivations tended to produce more floral resources, but fewer seed resources. 4. Bird, bumblebee and butterfly biodiversity responses were strongly affected by the quantity of seed or floral resources. Shelter enhanced biodiversity directly, increased floral resources and decreased seed yield. Seasonal weather patterns had large effects on both measures. Surprisingly, larger species pools and amounts of semi-natural habitat in the surrounding landscape had negative effects on biodiversity, which may indicate use by fauna of alternative foraging resources. 5. Synthesis and application. This is the first study to show a direct role of farmer social variables on the success of agri-environment schemes in supporting farmland biodiversity. It suggests that farmers are not simply implementing agri-environment options, but are learning and improving outcomes by doing so. Better engagement with farmers and working with farmers who have a history of environmental management may therefore enhance success. The importance of a number of environmental factors may explain why agri-environment outcomes are variable, and suggests some – such as the weather – cannot be controlled. Others, such as shelter, could be incorporated into agri-environment prescriptions. The role of landscape factors remains complex and currently eludes simple conclusions about large-scale targeting of schemes.

Interactive learning in UK construction practice: examining the role of BIM process standards

From a construction innovation systems perspective, firms acquire knowledge from suppliers, clients, universities and institutional environment. Building information modelling (BIM) involves these firms using new process standards. To understand the implications on interactive learning using BIM process standards, a case study is conducted with the UK operations of a multinational construction firm. Data is drawn from: a) two workshops involving the firm and a wider industry group, b) observations of practice in the BIM core team and in three ongoing projects, c) 12 semi-structured interviews; and d) secondary publications. The firm uses a set of BIM process standards (IFC, PAS 1192, Uniclass, COBie) in its construction activities. It is also involved in a pilot to implement the COBie standard, supported by technical and management standards for BIM, such as Uniclass and PAS1192. Analyses suggest that such BIM process standards unconsciously shapes the firm's internal and external interactive learning processes. Internally standards allow engineers to learn from each through visualising 3D information and talking around designs with operatives to address problems during construction. Externally, the firm participates in trial and pilot projects involving other construction firms, government agencies, universities and suppliers to learn about the standard and access knowledge to solve its specific design problems. Through its BIM manager, the firm provides feedback to standards developers and information technology suppliers. The research contributes by articulating how BIM process standards unconsciously change interactive learning processes in construction practice. Further research could investigate these findings in the wider UK construction innovation system.

'Rules of engagement’: how experiential learning facilitates the formation of a public–private partnership in Russia

Purpose – The purpose of this paper is to investigate to what extent one can apply experiential learning theory (ELT) to the public-private partnership (PPP) setting in Russia and to draw insights regarding the learning cycle ' s nature. Additionally, the paper assesses whether the PPP case confirms Kolb ' s ELT. Design/methodology/approach – The case study draws upon primary data which the authors collected by interviewing informants including a PPP operator ' s managers, lawyers from Russian law firms and an expert from the National PPP Centre. The authors accomplished data source triangulation in order to ensure a high degree of research validity. Findings – Experiential learning has resulted in a successful and a relatively fast PPP project launch without the concessionary framework. The lessons learned include the need for effective stakeholder engagement; avoiding being stuck in bureaucracy such as collaboration with Federal Ministries and anti-trust agency; avoiding application for government funding as the approval process is tangled and lengthy; attracting strategic private investors; shaping positive public perception of a PPP project; and making continuous efforts in order to effectively mitigate the public acceptance risk. Originality/value – The paper contributes to ELT by incorporating the impact of social environment in the learning model. Additionally, the paper tests the applicability of ELT to learning in the complex organisational setting, i.e., a PPP.