Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

Reinforcement learning in dendritic structures

The discovery of binary dendritic events such as local NMDA spikes in dendritic subbranches led to the suggestion that dendritic trees could be computationally equivalent to a 2-layer network of point neurons, with a single output unit represented by the soma, and input units represented by the dendritic branches. Although this interpretation endows a neuron with a high computational power, it is functionally not clear why nature would have preferred the dendritic solution with a single but complex neuron, as opposed to the network solution with many but simple units. We show that the dendritic solution has a distinguished advantage over the network solution when considering different learning tasks. Its key property is that the dendritic branches receive an immediate feedback from the somatic output spike, while in the corresponding network architecture the feedback would require additional backpropagating connections to the input units. Assuming a reinforcement learning scenario we formally derive a learning rule for the synaptic contacts on the individual dendritic trees which depends on the presynaptic activity, the local NMDA spikes, the somatic action potential, and a delayed reinforcement signal. We test the model for two scenarios: the learning of binary classifications and of precise spike timings. We show that the immediate feedback represented by the backpropagating action potential supplies the individual dendritic branches with enough information to efficiently adapt their synapses and to speed up the learning process.

"It was hard to come to mutual understanding..." - Multidimensionality of social learning processes in natural resource use in India, Africa and Latin America

Sustainable natural resource use requires that multiple actors reassess their situation in a systemic perspective. This can be conceptualised as a social learning process between actors from rural communities and the experts from outside organisations. A specifically designed workshop oriented towards a systemic view of natural resource use and the enhancement of mutual learning between local and external actors, provided the background for evaluating the potentials and constraints of intensified social learning processes. Case studies in rural communities in India, Bolivia, Peru and Mali showed that changes in the narratives of the participants of the workshop followed a similar temporal sequence relatively independently from their specific contexts. Social learning processes were found to be more likely to be successful if they 1) opened new space for communicative action, allowing for an intersubjective re-definition of the present situation, 2) contributed to rebalance the relationships between social capital and social, emotional and cognitive competencies within and between local and external actors.

Moving from sustainable management to sustainable governance of natural resources: The role of social learning processes in rural India, Bolivia and Mali

The present paper discusses a conceptual, methodological and practical framework within which the limitations of the conventional notion of natural resource management (NRM) can be overcome. NRM is understood as the application of scientific ecological knowledge to resource management. By including a consideration of the normative imperatives that arise from scientific ecological knowledge and submitting them to public scrutiny, ‘sustainable management of natural resources’ can be recontextualised as ‘sustainable governance of natural resources’. This in turn makes it possible to place the politically neutralising discourse of ‘management’ in a space for wider societal debate, in which the different actors involved can deliberate and negotiate the norms, rules and power relations related to natural resource use and sustainable development. The transformation of sustainable management into sustainable governance of natural resources can be conceptualised as a social learning process involving scientists, experts, politicians and local actors, and their corresponding scientific and non-scientific knowledges. The social learning process is the result of what Habermas has described as ‘communicative action’, in contrast to ‘strategic action’. Sustainable governance of natural resources thus requires a new space for communicative action aiming at shared, intersubjectively validated definitions of actual situations and the goals and means required for transforming current norms, rules and power relations in order to achieve sustainable development. Case studies from rural India, Bolivia and Mali explore the potentials and limitations for broadening communicative action through an intensification of social learning processes at the interface of local and external knowledge. Key factors that enable or hinder the transformation of sustainable management into sustainable governance of natural resources through social learning processes and communicative action are discussed.

Interference during the implicit learning of two different motor sequences

It has been demonstrated that learning a second motor task after having learned a first task may interfere with the long-term consolidation of the first task. However, little is known about immediate changes in the representation of the motor memory in the early acquisition phase within the first minutes of the learning process. Therefore, we investigated such early interference effects with an implicit serial reaction time task in 55 healthy subjects. Each subject performed either a sequence learning task involving two different sequences, or a random control task. The results showed that learning the first sequence led to only a slight, short-lived interference effect in the early acquisition phase of the second sequence. Overall, learning of neither sequence was impaired. Furthermore, the two processes, sequence-unrelated task learning (i.e. general motor training) and the sequence learning itself did not appear to interfere with each other. In conclusion, although the long-term consolidation of a motor memory has been shown to be sensitive to other interfering memories, the present study suggests that the brain is initially able to acquire more than one new motor sequence within a short space of time without significant interference.

Constructing a user experience user experience-based mobile learning environment

Mobile learning, in the past defined as learning with mobile devices, now refers to any type of learning-on-the-go or learning that takes advantage of mobile technologies. This new definition shifted its focus from the mobility of technology to the mobility of the learner (O'Malley and Stanton 2002; Sharples, Arnedillo-Sanchez et al. 2009). Placing emphasis on the mobile learner’s perspective requires studying “how the mobility of learners augmented by personal and public technology can contribute to the process of gaining new knowledge, skills, and experience” (Sharples, Arnedillo-Sanchez et al. 2009). The demands of an increasingly knowledge based society and the advances in mobile phone technology are combining to spur the growth of mobile learning. Around the world, mobile learning is predicted to be the future of online learning, and is slowly entering the mainstream education. However, for mobile learning to attain its full potential, it is essential to develop more advanced technologies that are tailored to the needs of this new learning environment. A research field that allows putting the development of such technologies onto a solid basis is user experience design, which addresses how to improve usability and therefore user acceptance of a system. Although there is no consensus definition of user experience, simply stated it focuses on how a person feels about using a product, system or service. It is generally agreed that user experience adds subjective attributes and social aspects to a space that has previously concerned itself mainly with ease-of-use. In addition, it can include users’ perceptions of usability and system efficiency. Recent advances in mobile and ubiquitous computing technologies further underline the importance of human-computer interaction and user experience (feelings, motivations, and values) with a system. Today, there are plenty of reports on the limitations of mobile technologies for learning (e.g., small screen size, slow connection), but there is a lack of research on user experience with mobile technologies. This dissertation will fill in this gap by a new approach in building a user experience-based mobile learning environment. The optimized user experience we suggest integrates three priorities, namely a) content, by improving the quality of delivered learning materials, b) the teaching and learning process, by enabling live and synchronous learning, and c) the learners themselves, by enabling a timely detection of their emotional state during mobile learning. In detail, the contributions of this thesis are as follows: • A video codec optimized for screencast videos which achieves an unprecedented compression rate while maintaining a very high video quality, and a novel UI layout for video lectures, which together enable truly mobile access to live lectures. • A new approach in HTTP-based multimedia delivery that exploits the characteristics of live lectures in a mobile context and enables a significantly improved user experience for mobile live lectures. • A non-invasive affective learning model based on multi-modal emotion detection with very high recognition rates, which enables real-time emotion detection and subsequent adaption of the learning environment on mobile devices. The technology resulting from the research presented in this thesis is in daily use at the School of Continuing Education of Shanghai Jiaotong University (SOCE), a blended-learning institution with 35.000 students.

Learning Analytics: Challenges and Future Research Directions

In recent years, learning analytics (LA) has attracted a great deal of attention in technology-enhanced learning (TEL) research as practitioners, institutions, and researchers are increasingly seeing the potential that LA has to shape the future TEL landscape. Generally, LA deals with the development of methods that harness educational data sets to support the learning process. This paper provides a foundation for future research in LA. It provides a systematic overview on this emerging field and its key concepts through a reference model for LA based on four dimensions, namely data, environments, context (what?), stakeholders (who?), objectives (why?), and methods (how?). It further identifies various challenges and research opportunities in the area of LA in relation to each dimension.

Linking Applied Ethnobotany to Social Learning: A Participatory Tool for the Promotion of Indigenous Plants Use in Matutuíne, Southern Mozambique

In its search for pathways towards a more sustainable management of natural resources, development oriented research increasingly faces the challenge to develop new concepts and tools based on transdisciplinarity. Transdisciplinarity can, in terms of an idealized goal, be defined as a research approach that identifies and solves problems not only independently of disciplinary boundaries, but also including the knowledge and perceptions of non-scientific actors in a participatory process. In Mozambique, the Centre for Development and Environment (Berne, Switzerland), in partnership with Impacto and Helvetas (Maputo, Mozambique), has elaborated a new transdisciplinary tool to identify indigenous plants with a potential for commercialization. The tool combines methods from applied ethnobotany with participatory research in a social learning process. This approach was devised to support a development project aimed at creating alternative sources of income for rural communities of Matutuíne district, Southern Mozambique, while reducing the pressure on the natural environment. The methodology, which has been applied and tested, is innovative in that it combines important data collection through participatory research with a social learning process involving both local and external actors. This mutual learning process provides a space for complementary forms of knowledge to meet, eventually leading to the adoption of an integrated approach to natural resource management with an understanding of its ecological, socio-economic and cultural aspects; local stakeholders are included in the identification of potentials for sustainable development. Sustainable development itself, as a normative concept, can only be defined through social learning and consensus building between the local and external stakeholders.

Standards and specifications to manage accessibility issues in e-learning

Despite the acknowledged need of providing a personalized and adaptive learning process for all, current learning management systems do not properly cover personalization and accessibility issues and they are still struggling to support the reusability requirements coming from the pervasive usage of standards. There is a lack of frameworks for providing layered-based infrastructure covering the interoperability required to manage the whole range of standards, applications and services needed to meet accessibility and adaptations needs of lifelong learning services.

Integrated learning of computer applications for production engineering

A high productivity rate in Engineering is related to an efficient management of the flow of the large quantities of information and associated decision making activities that are consubstantial to the Engineering processes both in design and production contexts. Dealing with such problems from an integrated point of view and mimicking real scenarios is not given much attention in Engineering degrees. In the context of Engineering Education, there are a number of courses designed for developing specific competencies, as required by the academic curricula, but not that many in which integration competencies are the main target. In this paper, a course devoted to that aim is discussed. The course is taught in a Marine Engineering degree but the philosophy could be used in any Engineering field. All the lessons are given in a computer room in which every student can use each all the treated software applications. The first part of the course is dedicated to Project Management: the students acquire skills in defining, using Ms-PROJECT, the work breakdown structure (WBS), and the organization breakdown structure (OBS) in Engineering projects, through a series of examples of increasing complexity, ending up with the case of vessel construction. The second part of the course is dedicated to the use of a database manager, Ms-ACCESS, for managing production related information. A series of increasing complexity examples is treated ending up with the management of the pipe database of a real vessel. This database consists of a few thousand of pipes, for which a production timing frame is defined, which connects this part of the course with the first one. Finally, the third part of the course is devoted to the work with FORAN, an Engineering Production package of widespread use in the shipbuilding industry. With this package, the frames and plates where all the outfitting will be carried out are defined through cooperative work by the studens, working simultaneously in the same 3D model. In the paper, specific details about the learning process are given. Surveys have been posed to the students in order to get feed-back from their experience as well as to assess their satisfaction with the learning process. Results from these surveys are discussed in the paper

Co-evolutionary and Reinforcement Learning Techniques Applied to Computer Go players

The objective of this thesis is model some processes from the nature as evolution and co-evolution, and proposing some techniques that can ensure that these learning process really happens and useful to solve some complex problems as Go game. The Go game is ancient and very complex game with simple rules which still is a challenge for the Artificial Intelligence. This dissertation cover some approaches that were applied to solve this problem, proposing solve this problem using competitive and cooperative co-evolutionary learning methods and other techniques proposed by the author. To study, implement and prove these methods were used some neural networks structures, a framework free available and coded many programs. The techniques proposed were coded by the author, performed many experiments to find the best configuration to ensure that co-evolution is progressing and discussed the results. Using co-evolutionary learning processes can be observed some pathologies which could impact co-evolution progress. In this dissertation is introduced some techniques to solve pathologies as loss of gradients, cycling dynamics and forgetting. According to some authors, one solution to solve these co-evolution pathologies is introduce more diversity in populations that are evolving. In this thesis is proposed some techniques to introduce more diversity and some diversity measurements for neural networks structures to monitor diversity during co-evolution. The genotype diversity evolved were analyzed in terms of its impact to global fitness of the strategies evolved and their generalization. Additionally, it was introduced a memory mechanism in the network neural structures to reinforce some strategies in the genes of the neurons evolved with the intention that some good strategies learned are not forgotten. In this dissertation is presented some works from other authors in which cooperative and competitive co-evolution has been applied. The Go board size used in this thesis was 9x9, but can be easily escalated to more bigger boards.The author believe that programs coded and techniques introduced in this dissertation can be used for other domains.

An approach to automatic learning assessment based on the computational theory of perceptions

E-learning systems output a huge quantity of data on a learning process. However, it takes a lot of specialist human resources to manually process these data and generate an assessment report. Additionally, for formative assessment, the report should state the attainment level of the learning goals defined by the instructor. This paper describes the use of the granular linguistic model of a phenomenon (GLMP) to model the assessment of the learning process and implement the automated generation of an assessment report. GLMP is based on fuzzy logic and the computational theory of perceptions. This technique is useful for implementing complex assessment criteria using inference systems based on linguistic rules. Apart from the grade, the model also generates a detailed natural language progress report on the achieved proficiency level, based exclusively on the objective data gathered from correct and incorrect responses. This is illustrated by applying the model to the assessment of Dijkstra’s algorithm learning using a visual simulation-based graph algorithm learning environment, called GRAPHs

Web-based education in Spanish Universities. A Comparison of Open Source E-Learning Platforms

Web-based education or „e-learning‟ has become a critical component in higher education for the last decade, replacing other distance learning methods, such as traditional computer training or correspondence learning. The number of university students who take on-line courses is continuously increasing all over the world. In Spain, nearly a 90% of the universities have an institutional e-learning platform and over 60% of the traditional on-site courses use this technology as a supplement to the traditional face-to-face classes. This new form of learning allows the disappearance of geographical barriers and enables students to schedule their own learning process, among some other advantages. On-line education is developed through specific software called „e-learning platform‟ or „virtual learning environment‟ (VLE). A considerable number of web-based tools to deliver distance courses are currently available. Open source software packages such as Moodle, Sakai, dotLRN or Dokeos are the most commonly used in the virtual campuses of Spanish universities. This paper analyzes the possibilities that virtual learning environments provide university teachers and learners and offers a technical comparison among some of the most popular e-learning learning platforms.

Self Learning and self evaluation exercises database in basic mechanics: a theoretical-practical approach

In the School of Mines of the Technical University of Madrid (UPM) the first course of different degrees has been implemented and adapted to the European Higher Educational Area (EHEA). In all of the degrees there is a first semester course which gathers all the contents of basic mechanics: from the first kinematics concepts to the rigid solid plane motion Before the Bologna process took place, the authors had established the final assessment of the theoretical contents through open questions of theoretical-practical character In the present work, the elaboration of a wide database containing theoretical-practical questions that students can access on line is presented. The questions are divided in thirteen different questionnaires composed of a number of questions randomly chosen from a certain group in the database. Each group corresponds to a certain learning objective that the student knows. After answering the questionnaire and checking the grade assigned according to the performance of the student, the pupils can see the correct response displayed on the screen and widely explained by the professors. This represents a 10% of the final grade. As the student can access the questionnaires as many times as they want, the main goal is the self-assessment of each learning objective and therefore, getting the students involved in their own learning process so they can decide how much time they need to acquire the required level.