Dynamic equilibrium through reinforcement learning

Reinforcement Learning is an area of Machine Learning that deals with how an agent should take actions in an environment such as to maximize the notion of accumulated reward. This type of learning is inspired by the way humans learn and has led to the creation of various algorithms for reinforcement learning. These algorithms focus on the way in which an agent’s behaviour can be improved, assuming independence as to their surroundings. The current work studies the application of reinforcement learning methods to solve the inverted pendulum problem. The importance of the variability of the environment (factors that are external to the agent) on the execution of reinforcement learning agents is studied by using a model that seeks to obtain equilibrium (stability) through dynamism – a Cart-Pole system or inverted pendulum. We sought to improve the behaviour of the autonomous agents by changing the information passed to them, while maintaining the agent’s internal parameters constant (learning rate, discount factors, decay rate, etc.), instead of the classical approach of tuning the agent’s internal parameters. The influence of changes on the state set and the action set on an agent’s capability to solve the Cart-pole problem was studied. We have studied typical behaviour of reinforcement learning agents applied to the classic BOXES model and a new form of characterizing the environment was proposed using the notion of convergence towards a reference value. We demonstrate the gain in performance of this new method applied to a Q-Learning agent.

Análise funcional comparativa de algoritmos de aprendizagem por reforço

De entre todos os paradigmas de aprendizagem actualmente identificados, a Aprendizagem por Reforço revela-se de especial interesse e aplicabilidade nos inúmeros processos que nos rodeiam: desde a solitária sonda que explora o planeta mais remoto, passando pelo programa especialista que aprende a apoiar a decisão médica pela experiencia adquirida, até ao cão de brincar que faz as delícias da criança interagindo com ela e adaptando-se aos seus gostos, e todo um novo mundo que nos rodeia e apela crescentemente a que façamos mais e melhor nesta área. Desde o aparecimento do conceito de aprendizagem por reforço, diferentes métodos tem sido propostos para a sua concretização, cada um deles abordando aspectos específicos. Duas vertentes distintas, mas complementares entre si, apresentam-se como características chave do processo de aprendizagem por reforço: a obtenção de experiência através da exploração do espaço de estados e o aproveitamento do conhecimento obtido através dessa mesma experiência. Esta dissertação propõe-se seleccionar alguns dos métodos propostos mais promissores de ambas as vertentes de exploração e aproveitamento, efectuar uma implementação de cada um destes sobre uma plataforma modular que permita a simulação do uso de agentes inteligentes e, através da sua aplicação na resolução de diferentes configurações de ambientes padrão, gerar estatísticas funcionais que permitam inferir conclusões que retractem entre outros aspectos a sua eficiência e eficácia comparativas em condições específicas.

Dynamical behaviour on the parameter space: new populational growth models proportional to beta densities

We present new populational growth models, generalized logistic models which are proportional to beta densities with shape parameters p and 2, where p > 1, with Malthusian parameter r. The complex dynamical behaviour of these models is investigated in the parameter space (r, p), in terms of topological entropy, using explicit methods, when the Malthusian parameter r increases. This parameter space is split into different regions, according to the chaotic behaviour of the models.

e-learning in the classroom?

As teachers, we are challenged everyday to solve pedagogical problems and we have to fight for our students’ attention in a media rich world. I will talk about how we use ICT in Initial Teacher Training and give you some insight on what we are doing. The most important benefit of using ICT in education is that it makes us reflect on our practice. There is no doubt that our classrooms need to be updated, but we need to be critical about every peace of hardware, software or service that we bring into them. It is not only because our budgets are short, but also because e‐learning is primarily about learning, not technology. Therefore, we need to have the knowledge and skills required to act in different situations, and choose the best tool for the job. Not all subjects are suitable for e‐learning, nor do all students have the skills to organize themselves their own study times. Also not all teachers want to spend time programming or learning about instructional design and metadata. The promised land of easy use of authoring tools (e.g. eXe and Reload) that will lead to all teachers become Learning Objects authors and share these LO in Repositories, all this failed, like previously HyperCard, Toolbook and others. We need to know a little bit of many different technologies so we can mobilize this knowledge when a situation requires it: integrate e‐learning technologies in the classroom, not a flipped classroom, just simple tools. Lecture capture, mobile phones and smartphones, pocket size camcorders, VoIP, VLE, live video broadcast, screen sharing, free services for collaborative work, save, share and sync your files. Do not feel stressed to use everything, every time. Just because we have a whiteboard does not mean we have to make it the centre of the classroom. Start from where you are, with your preferred subject and the tools you master. Them go slowly and try some new tool in a non‐formal situation and with just one or two students. And you don’t need to be alone: subscribe a mailing list and share your thoughts with other teachers in a dedicated forum, even better if both are part of a community of practice, and share resources. We did that for music teachers and it was a success, in two years arriving at 1.000 members. Just do it.

Enhancing reflexivity and technology in translation learning: case study on the use of ePortfolio as a training environment

In this paper we describe a casestudy of an experiment on how reflexivity and technology can enhance learning, by using ePorfolios as a training environment to develop translation skills. Translation is today a multiskilled job and translators need to assure their clients a good performance and quality, both in language and in technology domains. In order to accomplish it, for the translator all the tasks and processes he develops appear as crucial, being pretranslation and posttranslation processes equally important as the translation itself, namely as far as autonomy, reflexive and critical skills are concerned. Finally, the need and relevance for collaborative tasks and networks amongst virtual translation communities, led us to the decision of implementing ePortfolios as a tool to develop the requested skills and extend the use of Internet in translation, namely in terminology management phases, for the completion of each task, by helping students in the management of the projects deadlines, improving their knowledge on the construction and management of translation resources and deepening their awareness about the concepts related to the development and usability of ePorfolios.

Learning management systems usage on higher education institutions

Learning Management Systems (LMS) are used all over Higher Education Institutions (HEI) and the need to know and understand its adoption and usage arises. However, there is a lack of information about how LMSs are being used, which are the most adopted, whether there is a country adoption standard and which countries use more LMSs. A research team is developing a project that tries to fill this lack of information and provide the needed answers. With this purpose, on a first phase, it a survey was taken place. The results of this survey are presented in this paper. Another purpose of this paper is to disseminate the ongoing project.

Accounting education in an action-based learning environment: lessons learned

In the context of the Bologna Declaration a change is taking place in the teaching/learning paradigm. From teaching-centered education, which emphasizes the acquisition and transmission of knowledge, we now speak of learning-centered education, which is more demanding for students. This paradigm promotes a continuum of lifelong learning, where the individual needs to be able to handle knowledge, to select what is appropriate for a particular context, to learn permanently and to understand how to learn in new and rapidly changing situations. One attempt to face these challenges has been the experience of ISCAP regarding the teaching/learning of accounting in the course Managerial Simulation. This paper describes the process of teaching, learning and assessment in an action-based learning environment. After a brief general framework that focuses on education objectives, we report the strengths and limitations of this teaching/learning tool. We conclude with some lessons from the implementation of the project.

Formação combinada: reutilizando learning objects

O uso crescente da Internet (World Wide Web), e das suas potencialidades tecnológicas têm contribuído para uma proliferação de ambientes de ensino/aprendizagem, baseados em Tecnologia. A comunidade científica reúne consenso quanto às vantagens da reutilização de conteúdos de aprendizagem e à adopção de standards com vista à interoperabilidade entre conteúdos/objectos partilháveis e plataformas. Este artigo tem como objectivo reflectir sobre o desenvolvimento de uma metodologia de ensino combinada de aprendizagem com recurso a Learning Objects, no âmbito do trabalho de doutoramento.

Thinking over b‐learning strategy: the MIPO model approach

This article describes the main research results in a new methodology, in which the stages and strategies of the technology integration process are identified and described. A set of principles and recommendations are therefore presented. The MIPO model described in this paper is a result of the effort made regarding the understanding of the main success features of good practices, in the web environment, integrated in the information systems/information technology context. The initial model has been created, based on experiences and literature review. After that, it was tested in the information and technology system units at higher school and also adapted as a result of four cycles of an actionresearch work combined with a case study research. The information, concepts and procedures presented here give support to teachers and instructors, instructional designers and planning teams – anyone who wants to develop effective b‐learning instructions.

Innovation through virtual communities of Practice: motivation and constraints in the knowledge-creation process

Communities of Practice are places which provide a sound basis for organizational learning, enabling knowledge creation and acquisition thus improving organizational performance, leveraging innovation and consequently increasing competitively. Virtual Communities of Practice (VCoP‟s) can perform a central role in promoting communication and collaboration between members who are dispersed in both time and space. The ongoing case study, described here, aims to identify both the motivations and the constraints that members of an organization experience when taking part in the knowledge creating processes of the VCoP‟s to which they belong. Based on a literature review, we have identified several factors that influence such processes; they will be used to analyse the results of interviews carried out with the leaders of VCoP‟s in four multinationals. As future work, a questionnaire will be developed and administered to the other members of these VCoP‟s

Challenges for e-learning and adult students in higher education

This article examines Lifelong Learning, from the perspective of the adult learner in higher education, by presenting some of the results of a project, funded by the European Commission's Socrates Programme, LIHE, Learning in Higher Education. It is structured as follows: first, the background of the project is described, then the experiences of the adult student, concerning their induction and tuition, are presented. Some future trends concerning adults in higher education and lifelong learning are outlined and conclusions drawn.

Building a knowledge and learning society in Portugal: adult students in technological schools and higher education institutions

Lifelong learning (LLL) has received increasing attention in recent years. It implies that learning should take place at all stages of the “life cycle and it should be life-wide, that is embedded in all life contexts from the school to the work place, the home and the community” (Green, 2002, p.613). The ‘learning society’, is the vision of a society where there are recognized opportunities for learning for every person, wherever they are and however old they happen to be. Globalization and the rise of new information technologies are some of the driving forces that cause depreciation of specialised competences. This happens very quickly in terms of economic value; consequently, workers of all skills levels, during their working life, must have the opportunity to update “their technical skills and enhance general skills to keep pace with continuous technological change and new job requirements” (Fahr, 2005, p. 75). It is in this context that LLL tops the policy agenda of international bodies, national governments and non-governmental organizations, in the field of education and training, to justify the need for LLL opportunities for the population as they face contemporary employability challenges. It is in this context that the requirement and interest to analyse the behaviour patterns of adult learners has developed over the last few years

Learning in higher education: the ISEGI case

Comunicação apresentada na IRMA Information Resources Management Association International Conference, San Diego, CA, 15 19 May

Virtual communities of practice: investigating motivations and constraints in the processes of knowledge creation and transfer

With accelerated market volatility, faster response times and increased globalization, business environments are going through a major transformation and firms have intensified their search for strategies which can give them competitive advantage. This requires that companies continuously innovate, to think of new ideas that can be transformed or implemented as products, processes or services, generating value for the firm. Innovative solutions and processes are usually developed by a group of people, working together. A grouping of people that share and create new knowledge can be considered as a Community of Practice (CoP). CoP’s are places which provide a sound basis for organizational learning and encourage knowledge creation and acquisition. Virtual Communities of Practice (VCoP's) can perform a central role in promoting communication and collaboration between members who are dispersed in both time and space. Nevertheless, it is known that not all CoP's and VCoP's share the same levels of performance or produce the same results. This means that there are factors that enable or constrain the process of knowledge creation. With this in mind, we developed a case study in order to identify both the motivations and the constraints that members of an organization experience when taking part in the knowledge creating processes of VCoP's. Results show that organizational culture and professional and personal development play an important role in these processes. No interviewee referred to direct financial rewards as a motivation factor for participation in VCoPs. Most identified the difficulty in aligning objectives established by the management with justification for the time spent in the VCoP. The interviewees also said that technology is not a constraint.

Learning to manage in knowledge based organizations: the role of portfolios

Paper presented at the 8th European Conference on Knowledge Management, Barcelona, 6-7 Sep. 2008 URL: http://www.academic-conferences.org/eckm/eckm2007/eckm07-home.htm