Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

Statistical inference for single- and multi-band probabilistic amplitude demodulation.

Amplitude demodulation is an ill-posed problem and so it is natural to treat it from a Bayesian viewpoint, inferring the most likely carrier and envelope under probabilistic constraints. One such treatment is Probabilistic Amplitude Demodulation (PAD), which, whilst computationally more intensive than traditional approaches, offers several advantages. Here we provide methods for estimating the uncertainty in the PAD-derived envelopes and carriers, and for learning free-parameters like the time-scale of the envelope. We show how the probabilistic approach can naturally handle noisy and missing data. Finally, we indicate how to extend the model to signals which contain multiple modulators and carriers.

Learning based automatic face annotation for arbitrary poses and expressions from frontal images only

Statistical approaches for building non-rigid deformable models, such as the Active Appearance Model (AAM), have enjoyed great popularity in recent years, but typically require tedious manual annotation of training images. In this paper, a learning based approach for the automatic annotation of visually deformable objects from a single annotated frontal image is presented and demonstrated on the example of automatically annotating face images that can be used for building AAMs for fitting and tracking. This approach employs the idea of initially learning the correspondences between landmarks in a frontal image and a set of training images with a face in arbitrary poses. Using this learner, virtual images of unseen faces at any arbitrary pose for which the learner was trained can be reconstructed by predicting the new landmark locations and warping the texture from the frontal image. View-based AAMs are then built from the virtual images and used for automatically annotating unseen images, including images of different facial expressions, at any random pose within the maximum range spanned by the virtually reconstructed images. The approach is experimentally validated by automatically annotating face images from three different databases. © 2009 IEEE.

Active learning of model evidence using Bayesian quadrature

Numerical integration is a key component of many problems in scientific computing, statistical modelling, and machine learning. Bayesian Quadrature is a modelbased method for numerical integration which, relative to standard Monte Carlo methods, offers increased sample efficiency and a more robust estimate of the uncertainty in the estimated integral. We propose a novel Bayesian Quadrature approach for numerical integration when the integrand is non-negative, such as the case of computing the marginal likelihood, predictive distribution, or normalising constant of a probabilistic model. Our approach approximately marginalises the quadrature model's hyperparameters in closed form, and introduces an active learning scheme to optimally select function evaluations, as opposed to using Monte Carlo samples. We demonstrate our method on both a number of synthetic benchmarks and a real scientific problem from astronomy.

Bayesian Learning of Noisy Markov Decision Processes

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller.

Statistical Evaluation of the Predictive Toxicology Challenge 2000-2001

Toivonen, H., Srinivasan, A., King, R. D., Kramer, S. and Helma, C. (2003) Statistical Evaluation of the Predictive Toxicology Challenge 2000-2001. Bioinformatics 19: 1183-1193

Team-teaching for inclusive learning: Purposes, practices and perceptions of a team-teaching initiative in Irish post-primary schools

Background. Schools unequivocally privilege solo-teaching. This research seeks to enhance our understanding of team-teaching by examining how two teachers, working in the same classroom at the same time, might or might not contribute to the promotion of inclusive learning. There are well-established policy statements that encourage change and moves towards the use of team-teaching to promote greater inclusion of students with special educational needs in mainstream schools and mainstream classrooms. What is not so well established is the practice of team-teaching in post-primary settings, with little research conducted to date on how it can be initiated and sustained, and a dearth of knowledge on how it impacts upon the students and teachers involved. Research questions and aims. In light of the paucity and inconclusive nature of the research on team-teaching to date (Hattie, 2009), the orientating question in this study asks ‘To what extent, can the introduction of a formal team-teaching initiative enhance the quality of inclusive student learning and teachers’ learning at post-primary level?’ The framing of this question emerges from ongoing political, legal and educational efforts to promote inclusive education. The study has three main aims. The first aim of this study is to gather and represent the voices and experiences of those most closely involved in the introduction of team-teaching; students, teachers, principals and administrators. The second aim is to generate a theory-informed understanding of such collaborative practices and how they may best be implemented in the future. The third aim is to advance our understandings regarding the day-to-day, and moment-to-moment interactions, between teachers and students which enable or inhibit inclusive learning. Sample. In total, 20 team-teaching dyads were formed across seven project schools. The study participants were from two of the seven project schools, Ash and Oak. It involved eight teachers and 53 students, whose age ranged from 12-16 years old, with 4 teachers forming two dyads per school. In Oak there was a class of first years (n=11) with one dyad and a class of transition year students (n=24) with the other dyad. In Ash one class group (n=18) had two dyads. The subjects in which the dyads engaged were English and Mathematics. Method. This research adopted an interpretive paradigm. The duration of the fieldwork was from April 2007 to June 2008. Research methodologies included semi-structured interviews (n=44), classroom observation (n=20), attendance at monthly teacher meetings (n=6), questionnaires and other data gathering practices which included school documentation, assessment findings and joint examination of student work samples (n=4). Results. Team-teaching involves changing normative practices, and involves placing both demands and opportunities before those who occupy classrooms (teachers and students) and before those who determine who should occupy these classrooms (principals and district administrators). This research shows how team-teaching has the potential to promote inclusive learning, and when implemented appropriately, can impact positively upon the learning experiences of both teachers and students. The results are outlined in two chapters. In chapter four, Social Capital Theory is used in framing the data, the change process of bonding, bridging and linking, and in capturing what the collaborative action of team-teaching means, asks and offers teachers; within classes, between classes, between schools and within the wider educational community. In chapter five, Positioning Theory deductively assists in revealing the moment-to-moment, dynamic and inclusive learning opportunities, that are made available to students through team-teaching. In this chapter a number of vignettes are chosen to illustrate such learning opportunities. These two theories help to reveal the counter-narrative that team-teaching offers, regarding how both teachers and students teach and learn. This counter-narrative can extend beyond the field of special education and include alternatives to the manner in which professional development is understood, implemented, and sustained in schools and classrooms. Team-teaching repositions teachers and students to engage with one another in an atmosphere that capitalises upon and builds relational trust and shared cognition. However, as this research study has found, it is wise that the purposes, processes and perceptions of team-teaching are clear to all so that team-teaching can be undertaken by those who are increasingly consciously competent and not merely accidentally adequate. Conclusions. The findings are discussed in the context of the promotion of effective inclusive practices in mainstream settings. I believe that such promotion requires more nuanced understandings of what is being asked of, and offered to, teachers and students. Team-teaching has, and I argue will increasingly have, its place in the repertoire of responses that support effective inclusive learning. To capture and extend such practice requires theoretical frameworks that facilitate iterative journeys between research, policy and practice. Research to date on team-teaching has been too focused on outcomes over short timeframes and not focused enough on the process that is team-teaching. As a consequence team-teaching has been under-used, under-valued, under-theorised and generally not very well understood. Moving from classroom to staff room and district board room, theoretical frameworks used in this research help to travel with, and understand, the initiation, engagement and early consequences of team-teaching within and across the educational landscape. Therefore, conclusions from this study have implications for the triad of research, practice and policy development where efforts to change normative practices can be matched by understandings associated with what it means to try something new/anew, and what it means to say it made a positive difference.

An active learning approach for rapid characterization of endothelial cells in human tumors.

Currently, no available pathological or molecular measures of tumor angiogenesis predict response to antiangiogenic therapies used in clinical practice. Recognizing that tumor endothelial cells (EC) and EC activation and survival signaling are the direct targets of these therapies, we sought to develop an automated platform for quantifying activity of critical signaling pathways and other biological events in EC of patient tumors by histopathology. Computer image analysis of EC in highly heterogeneous human tumors by a statistical classifier trained using examples selected by human experts performed poorly due to subjectivity and selection bias. We hypothesized that the analysis can be optimized by a more active process to aid experts in identifying informative training examples. To test this hypothesis, we incorporated a novel active learning (AL) algorithm into FARSIGHT image analysis software that aids the expert by seeking out informative examples for the operator to label. The resulting FARSIGHT-AL system identified EC with specificity and sensitivity consistently greater than 0.9 and outperformed traditional supervised classification algorithms. The system modeled individual operator preferences and generated reproducible results. Using the results of EC classification, we also quantified proliferation (Ki67) and activity in important signal transduction pathways (MAP kinase, STAT3) in immunostained human clear cell renal cell carcinoma and other tumors. FARSIGHT-AL enables characterization of EC in conventionally preserved human tumors in a more automated process suitable for testing and validating in clinical trials. The results of our study support a unique opportunity for quantifying angiogenesis in a manner that can now be tested for its ability to identify novel predictive and response biomarkers.

Memorability as a measure of processing: a unit analysis of prose and list learning.

The percentage of subjects recalling each unit in a list or prose passage is considered as a dependent measure. When the same units are recalled in different tasks, processing is assumed to be the same; when different units are recalled, processing is assumed to be different. Two collections of memory tasks are presented, one for lists and one for prose. The relations found in these two collections are supported by an extensive reanalysis of the existing prose memory literature. The same set of words were learned by 13 different groups of subjects under 13 different conditions. Included were intentional free-recall tasks, incidental free recall following lexical decision, and incidental free recall following ratings of orthographic distinctiveness and emotionality. Although the nine free-recall tasks varied widely with regard to the amount of recall, the relative probability of recall for the words was very similar among the tasks. Imagery encoding and recognition produced relative probabilities of recall that were different from each other and from the free-recall tasks. Similar results were obtained with a prose passage. A story was learned by 13 different groups of subjects under 13 different conditions. Eight free-recall tasks, which varied with respect to incidental or intentional learning, retention interval, and the age of the subjects, produced similar relative probabilities of recall, whereas recognition and prompted recall produced relative probabilities of recall that were different from each other and from the free-recall tasks. A review of the prose literature was undertaken to test the generality of these results. Analysis of variance is the most common statistical procedure in this literature. If the relative probability of recall of units varied across conditions, a units by condition interaction would be expected. For the 12 studies that manipulated retention interval, an average of 21% of the variance was accounted for by the main effect of retention interval, 17% by the main effect of units, and only 2% by the retention interval by units interaction. Similarly, for the 12 studies that varied the age of the subjects, 6% of the variance was accounted for by the main effect of age, 32% by the main effect of units, and only 1% by the interaction of age by units.(ABSTRACT TRUNCATED AT 400 WORDS)

Learning from experience

Training courses for researchers are discussed in some detail. The preparation of researchers and of statisticians for consulting sessions, and the opportunities such sessions provide for training, are considered.

The role of Minitab in teaching and learning statistics

This paper describes how the statistical package Minitab is used in teaching statistics in our undergraduate programmes in Mathematics and Statistics to enhance student learning. How the sophisticated recent versions of Minitab can be used to help students understand statistical concepts, develop their statistical thinking and gain valuable skills in performing statistical analysis are discussed.

Some issues on assessment methods and learning in mathematics and statistics

This paper looks at the application of some of the assessment methods in practice with the view to enhance students’ learning in mathematics and statistics. It explores the effective application of assessment methods and highlights the issues or problems, and ways of avoiding them, related to some of the common methods of assessing mathematical and statistical learning. Some observations made by the author on good assessment practice and useful approaches employed at his institution in designing and applying assessment methods are discussed. Successful strategies in implementing assessment methods at different levels are described.

De la cooperativa agroalimentaria a la "learning netchain". Hacia un planteamiento teorico interorganizativo e interpersonal

Se propone un planteamiento teórico/conceptual para determinar si las relaciones interorganizativas e interpersonales de la netchain de las cooperativas agroalimentarias evolucionan hacia una learning netchain. Las propuestas del trabajo muestran que el mayor grado de asociacionismo y la mayor cooperación/colaboración vertical a lo largo de la cadena están positivamente relacionados con la posición horizontal de la empresa focal más cercana del consumidor final. Esto requiere una planificación y una resolución de problemas de manera conjunta, lo que está positivamente relacionado con el mayor flujo y diversidad de la información/conocimiento obtenido y diseminado a lo largo de la netchain. Al mismo tiempo se necesita desarrollar un contexto social en el que fluya la información/conocimiento y las nuevas ideas de manera informal y esto se logra con redes personales y, principalmente, profesionales y con redes internas y, principalmente, externas. Todo esto permitirá una mayor satisfacción de los socios de la cooperativa agroalimentaria y de sus distribuidores y una mayor intensidad en I+D, convirtiéndose la netchain de la cooperativa agroalimentaria, así, en una learning netchain.

Reflexive learning in pre-service training of primary school teachers: from experience to assimilation of contents

This article presents an educational experiment carried out in the Primary School Teaching Degree at the University of Barcelona. Specifically, the article analyses the application of the “Work Corners” approach in a core subject. In a three-year action research process, trainers put into practice an innovation which enabled them to boost cooperative work and reflexive learning among trainees. Firstly, the theoretical model underpinning the project and guiding many of the actions carried out by the training team is presented. After providing detailed information on the practical development of the experiment, the data-gathering process and its results are shown. Various information-gathering strategies were used in assessing the project, such as a questionnaire, participant observation, and teachers’ diaries. The results demonstrate, amongst other things, that “work corners” offer viable and appropriate educational conditions for the articulation of theoretical and practical knowledge, for building professional knowledge, and therefore, the beginnings of a reflexive teaching practice.

A heterosynaptic learning rule for neural networks

In this article we intoduce a novel stochastic Hebb-like learning rule for neural networks that is neurobiologically motivated. This learning rule combines features of unsupervised (Hebbian) and supervised (reinforcement) learning and is stochastic with respect to the selection of the time points when a synapse is modified. Moreover, the learning rule does not only affect the synapse between pre- and postsynaptic neuron, which is called homosynaptic plasticity, but effects also further remote synapses of the pre-and postsynaptic neuron. This more complex form of synaptic plasticity has recently come under investigations in neurobiology and is called heterosynaptic plasticity. We demonstrate that this learning rule is useful in training neural networks by learning parity functions including the exclusive-or (XOR) mapping in a multilayer feed-forward network. We find, that our stochastic learning rule works well, even in the presence of noise. Importantly, the mean leaxning time increases with the number of patterns to be learned polynomially, indicating efficient learning.