Techniques for batch reinforcement learning in robotics

This thesis addresses the Batch Reinforcement Learning methods in Robotics. This sub-class of Reinforcement Learning has shown promising results and has been the focus of recent research. Three contributions are proposed that aim to extend the state-of-art methods allowing for a faster and more stable learning process, such as required for learning in Robotics. The Q-learning update-rule is widely applied, since it allows to learn without the presence of a model of the environment. However, this update-rule is transition-based and does not take advantage of the underlying episodic structure of collected batch of interactions. The Q-Batch update-rule is proposed in this thesis, to process experiencies along the trajectories collected in the interaction phase. This allows a faster propagation of obtained rewards and penalties, resulting in faster and more robust learning. Non-parametric function approximations are explored, such as Gaussian Processes. This type of approximators allows to encode prior knowledge about the latent function, in the form of kernels, providing a higher level of exibility and accuracy. The application of Gaussian Processes in Batch Reinforcement Learning presented a higher performance in learning tasks than other function approximations used in the literature. Lastly, in order to extract more information from the experiences collected by the agent, model-learning techniques are incorporated to learn the system dynamics. In this way, it is possible to augment the set of collected experiences with experiences generated through planning using the learned models. Experiments were carried out mainly in simulation, with some tests carried out in a physical robotic platform. The obtained results show that the proposed approaches are able to outperform the classical Fitted Q Iteration.

BET on Top Hat: challenges to improve language proficiency

Within the business context, communication and interaction tends to be considerably rooted in the use of English (as lingua franca), as well as in ICT use. Thus, professionals have to be able to speak the English language, resorting to specific, internationally recognised terminology and be proficient in the use of manifold ICT tools. In fact, the tendency is for the great majority of higher education (HE) students to own mobile devices (laptops, smartphones and/or tablets) and use them to access information and communicate/interact with content and other people. Bearing this in mind, a teaching and learning strategy was designed, in which m-learning (i.e. learning in which the delivery platform is a mobile device) was used to approach Business English Terminology (BET). The strategy was labelled as ‘BET on Top Hat’, once the selected application was Top Hat (https://tophat.com/) and the idea was for students to face it as if it were a game/challenge. In this scenario, the main goals of this exploratory study were to find evidence as to: i) the utility of m-learning activities for learning BET and ii) if and how m-learning activities can generate intrinsic motivation in students to learn BET. Participants (n=23) were enrolled in English II, a curricular unit of the 1st cycle degree in Retail Management offered at Águeda School of Technology and Management – University of Aveiro (2014/15 edition). The data gathered included the students’ results in quizzes and their answers to a short final evaluation questionnaire regarding their experience with BET on Top Hat. Consequently, data were treated and analysed resorting to descriptive statistical analysis, and, when considered pertinent, the teacher’s observation notes were also considered. The results unveil that, on the one hand, the strategy had a clear positive impact on the students’ intrinsic motivation and, on the other hand, the students’ performance as to BET use tended to improve over time.