Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-Temporal Credit Assignment in Neuronal Population Learning

n learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.

How Porous are the Walls that Separate Us?: Transformative Service-Learning, Women’s Incarceration, and the Unsettled Self

In this article, we refine a politics of thinking from the margins by exploring a pedagogical model that advances transformative notions of service learning as social justice teaching. Drawing on a recent course we taught involving both incarcerated women and traditional college students, we contend that when communication among differentiated and stratified parties occurs, one possible result is not just a view of the other but also a transformation of the self and other. More specifically, we suggest that an engaged feminist praxis of teaching incarcerated women together with college students helps illuminate the porous nature of fixed markers that purport to reveal our identities (e.g., race and gender), to emplace our bodies (e.g., within institutions, prison gates, and walls), and to specify our locations (e.g., cultural, geographic, socialeconomic). One crucial theoretical insight our work makes clear is that the model of social justice teaching to which we aspired necessitates re-conceptualizing ourselves as students and professors whose subjectivities are necessarily relational and emergent.

Service-Centric Networking

Content-centric networking is a novel paradigm for the Future Internet that treats content as a first class citizen. This paper argues that content-centric networking should be generalized towards a service-centric networking scheme. We propose a service-centric networking design based on an object-oriented approach, in which content and services are considered objects. We show implementation architectures for example services and how these can benefit from service-oriented networking.

The Repairing of Misconceptions through the Use of Inquiry-Based Activities

Misconceptions exist in all fields of learning and develop through a person’s preconception of how the world works. Students with misconceptions in chemical engineering are not capable of correctly transferring knowledge to a new situation and will likely arrive at an incorrect solution. The purpose of this thesis was to repair misconceptions in thermodynamics by using inquiry-based activities. Inquiry-based learning is a method of teaching that involves hands-on learning and self-discovery. Previous work has shown inquiry-based methods result in better conceptual understanding by students relative to traditional lectures. The thermodynamics activities were designed to guide students towards the correct conceptual understanding through observing a preconception fail to hold up through an experiment or simulation. The developed activities focus on the following topics in thermodynamics: “internal energy versus enthalpy”, “equilibrium versus steady state”, and “entropy”. For each topic, two activities were designed to clarify the concept and assure it was properly grasped. Each activity was coupled with an instructions packet containing experimental procedure as well as pre- and post-analysis questions, which were used to analyze the effect of the activities on the students’ responses. Concept inventories were used to monitor students’ conceptual understanding at the beginning and end of the semester. The results did not show a statistically significant increase in the overall concept inventory scores for students who performed the activities compared to traditional learning. There was a statistically significant increase in concept area scores for “internal energy versus enthalpy” and “equilibrium versus steady state”. Although there was not a significant increase in concept inventory scores for “entropy”, written analyses showed most students’ misconceptions were repaired. Students transferred knowledge effectively and retained most of the information in the concept areas of “internal energy versus enthalpy” and “equilibrium versus steady state”.

Learning Paradigms Exemplified by Virtual Experiments with Honey Bees

Learning is based on rules that can be elucidated by behavioural experiments. This article focuses on virtual experiments, in which non-associative learning (habituation, sensitization) and principles of associative learning (contiguity, inhibitory learning, generalization, overshadowing, positive and negative patterning) can be examined using 'virtual' honey bees in PER (Proboscis Reaction Extension) conditioning experiments. Users can develop experimental designs, simulate and document the experiments and find explanations and suggestions for the analysis of the learning experiments. The virtual experiments are based on video sequences and data from actual learning experiments. The bees' responses are determined by probability-based learning profiles.

Pedagogical quality in e-learning

The article is concerned with design and use of e-learning technology to develop education qualitatively. The purpose is to develop a framework for a pedagogical evaluation of e-learning technology. The approach is that evaluation and design must be grounded in a learning theoretical approach, and it is argued that it is necessary to make a reflection of technology in relation to activities, learning principles, and a learning theory in order to qualitatively develop education. The article presents three frameworks developed on the basis of cognitivism, radical constructivism and activity theory. Finally, on the basis of the frameworks, the article discusses e-learning technology and, more specifically, design of virtual learning environments and learning objects. It is argued that e-learning technology is not pedagogically neutral, and that it is therefore necessary to focus on design of technology that explicitly supports a certain pedagogical approach. Further, it is argued that design should direct its focus away from organisation of content and towards design of activities.

Teachers' professional development in a community

The goal of this article was to study teachers' professional development related to web-based learning in the context of the teacher community. The object was to learn in what kind of networks teachers share the knowledge of web-based learning and what are the factors in the community that support or challenge teachers professional development of web-based learning. The findings of the study revealed that there are teachers who are especially active, called the central actors in this study, in the teacher community who collaborate and share knowledge of web-based learning. These central actors share both technical and pedagogical knowledge of web-based learning in networks that include both internal and external relations in the community and involve people, artefacts and a variety of media. Furthermore, the central actors appear to bridge different fields of teaching expertise in their community. According to the central actors' experiences the important factors that support teachers' professional development of web-based learning in the community are; the possibility to learn from colleagues and from everyday working practices, an emotionally safe atmosphere, the leader's personal support and community-level commitment. Also, the flexibility in work planning, challenging pupils, shared lessons with colleagues, training events in an authentic work environment and colleagues' professionalism are considered meaningful for professional development. As challenges, the knowledge sharing of web-based learning in the community needs mutual interests, transactive memory, time and facilities, peer support, a safe atmosphere and meaningful pedagogical practices. On the basis of the findings of the study it is suggested that by intensive collaboration related to web-based learning it may be possible to break the boundaries of individual teachership and create such sociocultural activities which support collaborative professional development in the teacher community. Teachers' in-service training programs should be more sensitive to the culture of teacher communities and teachers' reciprocal relations. Further, teacher trainers should design teachers' in-service training of web-based learning in co-evolution with supporting networks which include the media and artefacts as well as people.

Teachers' professional development in a community

The goal of this article was to study teachers' professional development related to web-based learning in the context of the teacher community. The object was to learn in what kind of networks teachers share the knowledge of web-based learning and what are the factors in the community that support or challenge teachers professional development of web-based learning. The findings of the study revealed that there are teachers who are especially active, called the central actors in this study, in the teacher community who collaborate and share knowledge of web-based learning. These central actors share both technical and pedagogical knowledge of web-based learning in networks that include both internal and external relations in the community and involve people, artefacts and a variety of media. Furthermore, the central actors appear to bridge different fields of teaching expertise in their community. According to the central actors' experiences the important factors that support teachers' professional development of web-based learning in the community are; the possibility to learn from colleagues and from everyday working practices, an emotionally safe atmosphere, the leader's personal support and community-level commitment. Also, the flexibility in work planning, challenging pupils, shared lessons with colleagues, training events in an authentic work environment and colleagues' professionalism are considered meaningful for professional development. As challenges, the knowledge sharing of web-based learning in the community needs mutual interests, transactive memory, time and facilities, peer support, a safe atmosphere and meaningful pedagogical practices. On the basis of the findings of the study it is suggested that by intensive collaboration related to web-based learning it may be possible to break the boundaries of individual teachership and create such sociocultural activities which support collaborative professional development in the teacher community. Teachers' in-service training programs should be more sensitive to the culture of teacher communities and teachers' reciprocal relations. Further, teacher trainers should design teachers' in-service training of web-based learning in co-evolution with supporting networks which include the media and artefacts as well as people.

Teachers' professional development in a community

The goal of this article was to study teachers' professional development related to web-based learning in the context of the teacher community. The object was to learn in what kind of networks teachers share the knowledge of web-based learning and what are the factors in the community that support or challenge teachers professional development of web-based learning. The findings of the study revealed that there are teachers who are especially active, called the central actors in this study, in the teacher community who collaborate and share knowledge of web-based learning. These central actors share both technical and pedagogical knowledge of web-based learning in networks that include both internal and external relations in the community and involve people, artefacts and a variety of media. Furthermore, the central actors appear to bridge different fields of teaching expertise in their community. According to the central actors' experiences the important factors that support teachers' professional development of web-based learning in the community are; the possibility to learn from colleagues and from everyday working practices, an emotionally safe atmosphere, the leader's personal support and community-level commitment. Also, the flexibility in work planning, challenging pupils, shared lessons with colleagues, training events in an authentic work environment and colleagues' professionalism are considered meaningful for professional development. As challenges, the knowledge sharing of web-based learning in the community needs mutual interests, transactive memory, time and facilities, peer support, a safe atmosphere and meaningful pedagogical practices. On the basis of the findings of the study it is suggested that by intensive collaboration related to web-based learning it may be possible to break the boundaries of individual teachership and create such sociocultural activities which support collaborative professional development in the teacher community. Teachers' in-service training programs should be more sensitive to the culture of teacher communities and teachers' reciprocal relations. Further, teacher trainers should design teachers' in-service training of web-based learning in co-evolution with supporting networks which include the media and artefacts as well as people.

What is Practice Research in Social Work - Definitions, Barriers and Possibilities

Practice is subject to increasing pressure to demonstrate its ability to achieve outcomes required by public policy makers. As part of this process social work practice has to engage with issues around advancing knowledge-based learning processes in a close collaboration with education and research based perspectives. This has given rise to approaches seeking to combine research methodology, field research and practical experience. Practice research is connected to both “the science of the concrete” – a field of research oriented towards subjects more than objects and “mode 2 knowledge production” – an application-oriented research where frameworks and findings are discussed by a number of partners. Practice research is defined into two approaches: practice research – collaboration between practice and research – and practitioner research – processes controlled and accomplished by practitioners. The basic stakeholders in practice research are social workers, service users, administrators, management, organisations, politicians and researchers. Accordingly, practice research is necessarily collaborative, involving a meeting point for different views, interests and needs, where complexity and dilemmas are inherent. Instead of attempting to balance or reconcile these differences, it is important to respect the differences if collaboration is to be established. The strength of both practice and research in practice research is to address these difficult challenges. The danger for both fields is to avoid and reject them.

Digital Systems for Open Access to Formal and Informal Learning

Today, Digital Systems and Services for Technology Supported Learning and Education are recognized as the key drivers to transform the way that individuals, groups and organizations “learn” and the way to “assess learning” in 21st Century. These transformations influence: Objectives - moving from acquiring new “knowledge” to developing new and relevant “competences”; Methods – moving from “classroom” based teaching to “context-aware” personalized learning; and Assessment – moving from “life-long” degrees and certifications to “on-demand” and “in-context” accreditation of qualifications. Within this context, promoting Open Access to Formal and Informal Learning, is currently a key issue in the public discourse and the global dialogue on Education, including Massive Open Online Courses (MOOCs) and Flipped School Classrooms. This volume on Digital Systems for Open Access to Formal and Informal Learning contributes to the international dialogue between researchers, technologists, practitioners and policy makers in Technology Supported Education and Learning. It addresses emerging issues related with both theory and practice, as well as, methods and technologies that can support Open Access to Formal and Informal Learning. In the twenty chapters contributed by international experts who are actively shaping the future of Educational Technology around the world, topics such as: - The evolution of University Open Courses in Transforming Learning - Supporting Open Access to Teaching and Learning of People with Disabilities - Assessing Student Learning in Online Courses - Digital Game-based Learning for School Education - Open Access to Virtual and Remote Labs for STEM Education - Teachers’ and Schools’ ICT Competence Profiling - Web-Based Education and Innovative Leadership in a K-12 International School Setting are presented. An in-depth blueprint of the promise, potential, and imminent future of the field, Digital Systems for Open Access to Formal and Informal Learning is necessary reading for researchers and practitioners, as well as, undergraduate and postgraduate students, in educational technology.

E-Learning Systems, Environments and Approaches

The volume consists of twenty-five chapters selected from among peer-reviewed papers presented at the CELDA (Cognition and Exploratory Learning in the Digital Age) 2013 Conference held in Fort Worth, Texas, USA, in October 2013 and also from world class scholars in e-learning systems, environments and approaches. The following sub-topics are included: Exploratory Learning Technologies (Part I), e-Learning social web design (Part II), Learner communities through e-Learning implementations (Part III), Collaborative and student-centered e-Learning design (Part IV). E-Learning has been, since its initial stages, a synonym for flexibility. While this dynamic nature has mainly been associated with time and space it is safe to argue that currently it embraces other aspects such as the learners’ profile, the scope of subjects that can be taught electronically and the technology it employs. New technologies also widen the range of activities and skills developed in e-Learning. Electronic learning environments have evolved past the exclusive delivery of knowledge. Technology has endowed e-Learning with the possibility of remotely fomenting problem solving skills, critical thinking and team work, by investing in information exchange, collaboration, personalisation and community building.

Learning Software-Maintenance Tasks in the Transition Phase of Offshore Outsourcing Projects: Two Learning-Theoretical Perspectives

The increasing practice of offshore outsourcing software maintenance has posed the challenge of effectively transferring knowledge to individual software engineers of the vendor. In this theoretical paper, we discuss the implications of two learning theories, the model of work-based learning (MWBL) and cognitive load theory (CLT), for knowledge transfer during the transition phase. Taken together, the theories suggest that learning mechanisms need to be aligned with the type of knowledge (tacit versus explicit), task characteristics (complexity and recurrence), and the recipients’ expertise. The MWBL proposes that learning mechanisms need to include conceptual and practical activities based on the relative importance of explicit and tacit knowledge. CLT explains how effective portfolios of learning mechanisms change over time. While jobshadowing, completion tasks, and supportive information may prevail at the outset of transition, they may be replaced by the work on conventional tasks towards the end of transition.

Estimating and quantifying uncertainties on level sets using the Vorob'ev expectation and deviation with Gaussian process models

Several methods based on Kriging have recently been proposed for calculating a probability of failure involving costly-to-evaluate functions. A closely related problem is to estimate the set of inputs leading to a response exceeding a given threshold. Now, estimating such a level set—and not solely its volume—and quantifying uncertainties on it are not straightforward. Here we use notions from random set theory to obtain an estimate of the level set, together with a quantification of estimation uncertainty. We give explicit formulae in the Gaussian process set-up and provide a consistency result. We then illustrate how space-filling versus adaptive design strategies may sequentially reduce level set estimation uncertainty.