Techniques for batch reinforcement learning in robotics

This thesis addresses the Batch Reinforcement Learning methods in Robotics. This sub-class of Reinforcement Learning has shown promising results and has been the focus of recent research. Three contributions are proposed that aim to extend the state-of-art methods allowing for a faster and more stable learning process, such as required for learning in Robotics. The Q-learning update-rule is widely applied, since it allows to learn without the presence of a model of the environment. However, this update-rule is transition-based and does not take advantage of the underlying episodic structure of collected batch of interactions. The Q-Batch update-rule is proposed in this thesis, to process experiencies along the trajectories collected in the interaction phase. This allows a faster propagation of obtained rewards and penalties, resulting in faster and more robust learning. Non-parametric function approximations are explored, such as Gaussian Processes. This type of approximators allows to encode prior knowledge about the latent function, in the form of kernels, providing a higher level of exibility and accuracy. The application of Gaussian Processes in Batch Reinforcement Learning presented a higher performance in learning tasks than other function approximations used in the literature. Lastly, in order to extract more information from the experiences collected by the agent, model-learning techniques are incorporated to learn the system dynamics. In this way, it is possible to augment the set of collected experiences with experiences generated through planning using the learned models. Experiments were carried out mainly in simulation, with some tests carried out in a physical robotic platform. The obtained results show that the proposed approaches are able to outperform the classical Fitted Q Iteration.

UNPROVEN INNOVATION: IPADS IN K-12 ENVIRONMENTS

Nationally, the education sector spends more than 5 billion dollars annually on digital tools, “yet seldom are technology solutions factored into any viable equation for improving student academic achievement” (Moersch, 2014, p. ix). Consider the following case in point: In July 2014, Apple announced that in just 3 years, the company had sold more than13 million iPads to educational institutions worldwide (Cavanagh, 2014). Put into perspective, that represents more than 5.2 billion dollars spent by the education industry to purchase iPads, which is the equivalent of the annual salaries of 89,655 teachers (“High School Teacher: Salary,” 2014). Despite such vast expenditures, there have been very few attempts to evaluate the efficacy of these digital tools on improving academic achievement. This research involved a quantitative data review of participant (student and teacher) survey data to explore one of the country’s largest K-12 iPad implementation undertakings in an effort to identify (a) best practices and (b) lessons learned from implementing the iPad into K-12 educational environments. It should be noted that the school system forming the basis of this research already had administered and collated the surveys used in this study.

Intercultural musical learning in the era of technology

Foreign students abroad need to feel integrated in the new community, which includes complex learning processes in multicultural environments. The fact that we have experienced these processes ourselves was certainly a motivation for this research, especially knowing that we could contribute to help our fellow Portuguese brass players undergoing the same experience. From the singularity of music performance in the style of playing and communication emerge many cultural aspects, which have been developed through centuries of orchestral practice. As the new students are confronted with the aesthetic musical concepts and both professional and social practices of the country they arrive in, they strive to understand these concepts and adapt themselves to the values promoted by the new music practice. The aim of this on-going research is the study of the integration of brass music students in German universities and in the German society. Notably, through the understanding of intercultural processes experienced by the students, professors can become more aware of the challenges that concern music education. In this research all ten Portuguese brass students enrolled in any German music university in the last five years were interviewed in order to deeply understand this process. With a growing importance of the technological facilities, students are able to gather more information, to prepare themselves for the new concepts they try to embrace and to better deal with a different culture.

Guided portfolio writing as a scaffold for reflective learning in in-service contexts: a case study

Language is widely recognized as an inescapable mediating tool for professional learning, and with this text we want to contribute to a better understanding of the particular role that guided writing can play in in-service professional reflective learning. We analysed one pre-school teacher’s written portfolio, the construction of which was guided to scaffold deep thinking about (and the transference of theory into) practice during participation in an in-service program about language education. Our case study shows that the writing process sustained robust learning about professional knowing, doing and learning itself: The teacher elaborated an integrative ethical understanding of the discussed theory, fully experienced newly informed practices and assessed her own learning by using theory to confront her previous knowledge and practices. Throughout the portfolio, the learning stance revealed by her voice varied accordingly. The study illustrates the potential of guided writing to scaffold reflective learning in in-service contexts.

Self-regulated learning in higher education : strategies adopted by computer programming students

Trabalho apresentado em PAEE/ALE’2016, 8th International Symposium on Project Approaches in Engineering Education (PAEE) and 14th Active Learning in Engineering Education Workshop (ALE)

A Soft Computing Approach to Quality Evaluation of General Chemistry Learning in Higher Education

In contemporary societies higher education must shape individuals able to solve problems in a workable and simpler manner and, therefore, a multidisciplinary view of the problems, with insights in disciplines like psychology, mathematics or computer science becomes mandatory. Undeniably, the great challenge for teachers is to provide a comprehensive training in General Chemistry with high standards of quality, and aiming not only at the promotion of the student’s academic success, but also at the understanding of the competences/skills required to their future doings. Thus, this work will be focused on the development of an intelligent system to assess the Quality-of-General-Chemistry-Learning, based on factors related with subject, teachers and students.

The interaction of economic and pedagogical ideals in the context of workplace learning in Germany: a framework for empirical research – inspired by business ethics

As an effect of marketisation, the importance of workplace learning in Germany has increased. The article follows up on the long-standing discourse around the question of how economic and pedagogical ideals interact in this context. In order to develop a theoretical framework for empirical research, three major positions of the discipline of business ethics are introduced. Business ethics in more abstract ways deals with the very same question, namely how do ideas such as profit orientation interact with other norms and values? The new perspectives show that the discourse has been hitherto based on a specific understanding of economy. In order to derive an empirical answer to the research question, the question is re-formulated as follows: Which values are inherent in the decisions taken? Consequently, it suggests using the concept of ‘rationalities of justification’ for empirical research. The article shows how this concept can be applied by conducting a test run. (DIPF/Orig.)

Continuous Quality Assurance and Optimisation in Cloud-Based Virtual Enterprises

Part 18: Optimization in Collaborative Networks

A Framework for Energy-Efficiency in Smart Home Environments

Part 7: Cyber-Physical Systems

Synchronization of Heteroclinic Circuits Through Learning in Chains of Neural Motifs

The synchronization of oscillatory activity in networks of neural networks is usually implemented through coupling the state variables describing neuronal dynamics. In this study we discuss another but complementary mechanism based on a learning process with memory. A driver network motif, acting as a teacher, exhibits winner-less competition (WLC) dynamics, while a driven motif, a learner, tunes its internal couplings according to the oscillations observed in the teacher. We show that under appropriate training the learner motif can dynamically copy the coupling pattern of the teacher and thus synchronize oscillations with the teacher. Then, we demonstrate that the replication of the WLC dynamics occurs for intermediate memory lengths only. In a unidirectional chain of N motifs coupled through teacher-learner paradigm the time interval required for pattern replication grows linearly with the chain size, hence the learning process does not blow up and at the end we observe phase synchronized oscillations along the chain. We also show that in a learning chain closed into a ring the network motifs come to a consensus, i.e. to a state with the same connectivity pattern corresponding to the mean initial pattern averaged over all network motifs.

"Going outside": discussing the connection between pedagogical practices with digital tools and outdoor learning in early childhood and primary education

Technology has an important role in children's lives and education. Based on several projects developed with ICT, both in Early Childhood Education (3-6 years old) and Primary Education (6-10 years old), since 1997, the authors argue that research and educational practices need to "go outside", addressing ways to connect technology with outdoor education. The experience with the projects and initiatives developed supported a conceptual framework, developed and discussed with several partners throughout the years and theoretically informed. Three main principles or axis have emerged: strengthening Children's Participation, promoting Critical Citizenship and establishing strong Connections to Pedagogy and Curriculum. In this paper, those axis will be presented and discussed in relation to the challenge posed by Outdoor Education to the way ICT in Early Childhood and Primary Education is understood, promoted and researched. The paper is exploratory, attempting to connect theoretical and conceptual contributions from Early Childhood Pedagogy with contributions from ICT in Education. The research-based knowledge available is still scarce, mostly based on studies developed with other purposes. The paper, therefore, focus the connections and interpellations between concepts established through the theoretical framework and draws on the almost 20 years of experience with large and small scale action-research projects of ICT in schools. The more recent one is already testing the conceptual framework by supporting children in non-formal contexts to explore vineyards and the cycle of wine production with several ICT tools. Approaching Outdoor Education as an arena where pedagogical and cultural dimensions influence decisions and practices, the paper tries to argue that the three axis are relevant in supporting a stronger connection between technology and the outdoor.

Disabling and Enabling Technologies for Learning in Higher Education for All : Issues and Challenges for Whom?

Integration, inclusion, and equity constitute fundamental dimensions of democracy in post-World War II societies and their institutions. The study presented here reports upon the ways in which individuals and institutions both use and account for the roles that technologies, including ICT, play in disabling and enabling access for learning in higher education for all. Technological innovations during the 20th and 21st centuries, including ICT, have been heralded as holding significant promise for revolutionizing issues of access in societal institutions like schools, healthcare services, etc. (at least in the global North). Taking a socially oriented perspective, the study presented in this paper focuses on an ethnographically framed analysis of two datasets that critically explores the role that technologies, including ICT, play in higher education for individuals who are “differently abled” and who constitute a variation on a continuum of capabilities. Functionality as a dimension of everyday life in higher education in the 21st century is explored through the analysis of (i) case studies of two “differently abled” students in Sweden and (ii) current support services at universities in Sweden. The findings make visible the work that institutions and their members do through analyses of the organization of time and space and the use of technologies in institutional settings against the backdrop of individuals’ accountings and life trajectories. This study also highlights the relevance of multi-scale data analyses for revisiting the ways in which identity positions become framed or understood within higher education.

The Students’ Perspective of Geometry Teaching and Learning in High School

The purpose of this article is to present the results obtained from a questionnaire applied to Costa Rican high school students, in order to know their perspectives about geometry teaching and learning. The results show that geometry classes in high school education have been based on a traditional system of teaching, where the teacher presents the theory; he presents examples and exercises that should be solved by students, which emphasize in the application and memorization of formulas. As a consequence, visualization processes, argumentation and justification don’t have a preponderant role. Geometry is presented to students like a group of definitions, formulas, and theorems completely far from their reality and, where the examples and exercises don’t possess any relationship with their context. As a result, it is considered not important, because it is not applicable to real life situations. Also, the students consider that, to be successful in geometry, it is necessary to know how to use the calculator, to carry out calculations, to have capacity to memorize definitions, formulas and theorems, to possess capacity to understand the geometric drawings and to carry out clever exercises to develop a practical ability.

Perceptions of online tutorials for distance learning in mathematics and computing

We report on student and staff perceptions of synchronous online teaching and learning sessions in mathematics and computing. The study is based on two surveys of students and tutors conducted 5 years apart, and focusses on the educational experience as well as societal and accessibility dimensions. Key conclusions are that both staff and students value online sessions, to supplement face-to-face sessions, mainly for their convenience, but interaction within the sessions is limited. Students find the recording of sessions particularly helpful in their studies.

Exploring domain-informed and physics-guided learning in image-to-image translation

Image-to-image (i2i) translation networks can generate fake images beneficial for many applications in augmented reality, computer graphics, and robotics. However, they require large scale datasets and high contextual understanding to be trained correctly. In this thesis, we propose strategies for solving these problems, improving performances of i2i translation networks by using domain- or physics-related priors. The thesis is divided into two parts. In Part I, we exploit human abstraction capabilities to identify existing relationships in images, thus defining domains that can be leveraged to improve data usage efficiency. We use additional domain-related information to train networks on web-crawled data, hallucinate scenarios unseen during training, and perform few-shot learning. In Part II, we instead rely on physics priors. First, we combine realistic physics-based rendering with generative networks to boost outputs realism and controllability. Then, we exploit naive physical guidance to drive a manifold reorganization, which allowed generating continuous conditions such as timelapses.