Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Learning and Example Selection for Object and Pattern Detection

This thesis presents a learning based approach for detecting classes of objects and patterns with variable image appearance but highly predictable image boundaries. It consists of two parts. In part one, we introduce our object and pattern detection approach using a concrete human face detection example. The approach first builds a distribution-based model of the target pattern class in an appropriate feature space to describe the target's variable image appearance. It then learns from examples a similarity measure for matching new patterns against the distribution-based target model. The approach makes few assumptions about the target pattern class and should therefore be fairly general, as long as the target class has predictable image boundaries. Because our object and pattern detection approach is very much learning-based, how well a system eventually performs depends heavily on the quality of training examples it receives. The second part of this thesis looks at how one can select high quality examples for function approximation learning tasks. We propose an {em active learning} formulation for function approximation, and show for three specific approximation function classes, that the active example selection strategy learns its target with fewer data samples than random sampling. We then simplify the original active learning formulation, and show how it leads to a tractable example selection paradigm, suitable for use in many object and pattern detection problems.

Staged Competence Learning in Developmental Robotics

M.H. Lee, Q. Meng and F. Chao, 'Staged Competence Learning in Developmental Robotics', Adaptive Behavior, 15(3), pp 241-255, 2007. the full text will be available in September 2008

An active learning approach for rapid characterization of endothelial cells in human tumors.

Currently, no available pathological or molecular measures of tumor angiogenesis predict response to antiangiogenic therapies used in clinical practice. Recognizing that tumor endothelial cells (EC) and EC activation and survival signaling are the direct targets of these therapies, we sought to develop an automated platform for quantifying activity of critical signaling pathways and other biological events in EC of patient tumors by histopathology. Computer image analysis of EC in highly heterogeneous human tumors by a statistical classifier trained using examples selected by human experts performed poorly due to subjectivity and selection bias. We hypothesized that the analysis can be optimized by a more active process to aid experts in identifying informative training examples. To test this hypothesis, we incorporated a novel active learning (AL) algorithm into FARSIGHT image analysis software that aids the expert by seeking out informative examples for the operator to label. The resulting FARSIGHT-AL system identified EC with specificity and sensitivity consistently greater than 0.9 and outperformed traditional supervised classification algorithms. The system modeled individual operator preferences and generated reproducible results. Using the results of EC classification, we also quantified proliferation (Ki67) and activity in important signal transduction pathways (MAP kinase, STAT3) in immunostained human clear cell renal cell carcinoma and other tumors. FARSIGHT-AL enables characterization of EC in conventionally preserved human tumors in a more automated process suitable for testing and validating in clinical trials. The results of our study support a unique opportunity for quantifying angiogenesis in a manner that can now be tested for its ability to identify novel predictive and response biomarkers.

Implications for the scheduling of theory versus application in learning

This paper presents the findings of an experiment which looked at the effects of performing applied tasks (action learning) prior to the completion of the theoretical learning of these tasks (explanation-based learning), and vice-versa. The applied tasks took the form of laboratories for the Object-Oriented Analysis and Design (OOAD) course, theoretical learning was via lectures.

Can hypermedia aided learning (HAL) deliver?

This paper presents the results of a research project aimed at evaluating (HAL) as a mode of course delivery. More specifically the paper will deal with: • Developing a hypermedia courseware for students studying research methods; and • Evaluating hypermedia courseware as a method of delivery against traditional methods. This paper concentrates on pedagogical issues regarding computer aided learning and reports that this research gives tentative indications that hypermedia based learning (either through CD-ROM or the, as means of course delivery could be as effective as traditional modes of course delivery.

Capturing teachers' experience of learning design through case studies

This article distinguishes three dimensions to learning design: a technological infrastructure, a conceptual framework for practice that focuses on the creation of structured sequences of learning activities, and a way to represent and share practice through the use of mediating artefacts. Focusing initially on the second of these dimensions, the article reports the key findings from an exploratory study, eLIDA CAMEL. This project examined a hitherto under-researched aspect of learning design: what teachers who are new to the domain perceive to be its value as a framework for practice in the design of both flexible and classroom-based learning. Data collection comprised 13 case studies constructed from participants' self-reports. These suggest that providing students with a structured sequence of learning activities was the major value to teachers. The article additionally discusses the potential of such case studies to function as mediating artefacts for practitioners who are considering experimenting with learning design.

A case study of the effectiveness of the electronic survey as a research tool for measuring students' experience of e-learning

This paper uses a case study approach to consider the effectiveness of the electronic survey as a research tool to measure the learner voice about experiences of e-learning in a particular institutional case. Two large scale electronic surveys were carried out for the Student Experience of e-Learning (SEEL) project at the University of Greenwich in 2007 and 2008, funded by the UK Higher Education Academy (HEA). The paper considers this case to argue that, although the electronic web-based survey is a convenient method of quantitative and qualitative data collection, enabling higher education institutions swiftly to capture multiple views of large numbers of students regarding experiences of e-learning, for more robust analysis, electronic survey research is best combined with other methods of in-depth qualitative data collection. The advantages and disadvantages of the electronic survey as a research method to capture student experiences of e-learning are the focus of analysis in this short paper, which reports an overview of large-scale data collection (1,000+ responses) from two electronic surveys administered to students using surveymonkey as a web-based survey tool as part of the SEEL research project. Advantages of web-based electronic survey design include flexibility, ease of design, high degree of designer control, convenience, low costs, data security, ease of access and guarantee of confidentiality combined with researcher ability to identify users through email addresses. Disadvantages of electronic survey design include the self-selecting nature of web-enabled respondent participation, which tends to skew data collection towards students who respond effectively to email invitations. The relative inadequacy of electronic surveys to capture in-depth qualitative views of students is discussed with regard to prior recommendations from the JISC-funded Learners' Experiences of e-Learning (LEX) project, in consideration of the results from SEEL in-depth interviews with students. The paper considers the literature on web-based and email electronic survey design, summing up the relative advantages and disadvantages of electronic surveys as a tool for student experience of e-learning research. The paper concludes with a range of recommendations for designing future electronic surveys to capture the learner voice on e-learning, contributing to evidence-based learning technology research development in higher education.

Association for Learning Technology Annual Survey 2014 Data and Report

This is the first report from ALT’s new Annual Survey launched in December 2014. This survey was primarily for ALT members (individual or at an organisation which is an organisational member) it could however also be filled in by others, perhaps those interested in taking out membership. The report and data highlight emerging work areas that are important to the survey respondents. Analysis of the survey responses indicates a number of areas ALT should continue to support and develop. Priorities for the membership are ‘Intelligent use of learning technology’ and ‘Research and practice’, aligned to this is the value placed by respondent’s on by communication via the ALT Newsletter/News, social media and Research in Learning Technology. The survey also reveals ‘Data and Analytics’ and ‘Open Education’ are areas where the majority of respondents are finding are becoming increasingly important. As such our community may benefit from development opportunities ALT can provide. The survey is also a reminder that ALT has an essential role in enabling members to develop research and practice in areas which might be considered as minority interest. For example whilst the majority of respondents didn't indicate areas such as ‘Digital and Open Badges’, and ‘Game Based Learning’ as important there are still members who consider these areas are very significant and becoming increasingly valuable and as such ALT will continue to better support these groups within our community. Whilst ALT has conducted previous surveys of ALT membership this is the first iteration in this form. ALT has committed to surveying the sector on an annual basis, refining the core question set but trying to preserve an opportunity for longitudinal analysis.

Education in J.J. Rousseau: a methodological antecedent of competence based teaching

Taking into account the huge repercussion and influence that J.J. Rousseau has had on modern pedagogy, the recent tercentenary of his birth is a good opportunity to think about his outstanding relevance nowadays. This paper is a theoretical and educative research developed with an analytic and comparative hermeneutical method. The main objective is to show how some concepts of his philosophy of education have a great similarity with certain changes that the present competency based teaching is demanding, so it could be considered its methodological background. In order to achieve this objective this exposure has been divided in three parts. The first part is an analysis of Rousseau's educational theory as developed in the first three books of the Emilio, in which one of the main themes is self experience-based learning, fostering self-sufficiency, curiosity and the motivation for learning. Rousseau proposed as a method the negative education, which requires, among other conditions, a constant monitoring of the learner by the tutor. In the second part, a brief summary of the most relevant changes and characteristics of competency-based teaching is developed, as well as its purpose. The student’s participation and activity are highlighted within their own learning process through the carrying out of tasks. The new educational model involves a radical change in the curriculum, in which it is highlighted the transformation of the methodology used in the classroom as well as the role of the teacher. Finally, the aim of the third part is to offer a comparative synthesis of both proposals grouping the parallelisms found in 4 topics: origin of the two models, its aims, methodology, and change in the teaching roles.

An Efficient LS-SVM-Based Method for Fuzzy System Construction

This paper proposes an efficient learning mechanism to build fuzzy rule-based systems through the construction of sparse least-squares support vector machines (LS-SVMs). In addition to the significantly reduced computational complexity in model training, the resultant LS-SVM-based fuzzy system is sparser while offers satisfactory generalization capability over unseen data. It is well known that the LS-SVMs have their computational advantage over conventional SVMs in the model training process; however, the model sparseness is lost, which is the main drawback of LS-SVMs. This is an open problem for the LS-SVMs. To tackle the nonsparseness issue, a new regression alternative to the Lagrangian solution for the LS-SVM is first presented. A novel efficient learning mechanism is then proposed in this paper to extract a sparse set of support vectors for generating fuzzy IF-THEN rules. This novel mechanism works in a stepwise subset selection manner, including a forward expansion phase and a backward exclusion phase in each selection step. The implementation of the algorithm is computationally very efficient due to the introduction of a few key techniques to avoid the matrix inverse operations to accelerate the training process. The computational efficiency is also confirmed by detailed computational complexity analysis. As a result, the proposed approach is not only able to achieve the sparseness of the resultant LS-SVM-based fuzzy systems but significantly reduces the amount of computational effort in model training as well. Three experimental examples are presented to demonstrate the effectiveness and efficiency of the proposed learning mechanism and the sparseness of the obtained LS-SVM-based fuzzy systems, in comparison with other SVM-based learning techniques.

Machine learning methods for quantitative structure-property relationship modeling

Tese de doutoramento, Informática (Bioinformática), Universidade de Lisboa, Faculdade de Ciências, 2014

Leading as designing: how educational developers in leadership and management studies can use architectural design to facilitate student learning about delivering innovation

This paper describes a qualitative observational study of how a work based learning masters leadership development programme for middle managers in health and social care in the UK introduced students to key aspects of delivering innovation, through a formative assignment on contemporary architectural design. Action learning and activity theoretical approaches were used to enable students to explore common principles of leading the delivery of innovation. Between 2001 and 2013 a total of 89 students in 7 cohorts completed the assignment. Evaluation lent support for the view that the assignment provided a powerful learning experience for many. Several students found the creativity, determination and dedication of architects, designers and structural engineers inspirational in their ability to translate a creative idea into a completed artefact, deploy resources and negotiate complex demands of stakeholders. Others expressed varying levels of self-empowerment as regards their capacity for fostering an equivalent creativity in self and others. Theoretical approaches in addition to activity theory, including Engeström’s concepts of stabilisation knowledge and possibility knowledge, are discussed to explain these differing outcomes and to clarify the challenges and opportunities for educational developers seeking to utilise cross-disciplinary, creative approaches in curriculum design.

The European Project Semester at ISEP: Learning to Learn Engineering

The European Project Semester at ISEP (EPS@ISEP) is a one semester project-based learning programme addressed to engineering students from diverse scientific backgrounds and nationalities. The students, organized in multicultural teams, are challenged to solve real world multidisciplinary problems, accounting for 30 ECTU. The EPS package, although focused on project development (20 ECTU), includes a series of complementary seminars aimed at fostering soft, project-related and engineering transversal skills (10 ECTU). This paper presents the study plan, resources, operation and results of the EPS@ISEP that was created in 2011 to apply the best engineering education practices and promote the internationalization of ISEP. The results show that the EPS@ISEP students acquire during one semester the scientific, technical and soft competences necessary to propose, design and implement a solution for a multidisciplinary problem.

An interactive learning approach to accounting classes

This working paper explores the use of interactive learning tools, such as business simulations, to facilitate the active learning process in accounting classes. Although business simulations were firstly introduced in the United States in the 1950s, the vast majority of accounting professors still use traditional teaching methods, based in end-of-chapter exercises and written cases. Moreover, the current students’ generation brings new challenges to the classroom related with their video, game, internet and mobile culture. Thus, a survey and an experimentation were conducted to understand, on one hand, if accounting professors are willing to adjust their teaching methods with the adoption of interactive learning tools and, on the other hand, if the adoption of interactive learning tools in accounting classes yield better academic results and levels of satisfaction among students. Students using more interactive learning approaches scored significantly higher means than others that did not. Accounting professors are clearly willing to try, at least once, the use of an accounting simulator in classes.