Various algorithms for optimization and learning in adaptive systems

This thesis discusses various methods for learning and optimization in adaptive systems. Overall, it emphasizes the relationship between optimization, learning, and adaptive systems; and it illustrates the influence of underlying hardware upon the construction of efficient algorithms for learning and optimization. Chapter 1 provides a summary and an overview.

Chapter 2 discusses a method for using feed-forward neural networks to filter the noise out of noise-corrupted signals. The networks use back-propagation learning, but they use it in a way that qualifies as unsupervised learning. The networks adapt based only on the raw input data-there are no external teachers providing information on correct operation during training. The chapter contains an analysis of the learning and develops a simple expression that, based only on the geometry of the network, predicts performance.

Chapter 3 explains a simple model of the piriform cortex, an area in the brain involved in the processing of olfactory information. The model was used to explore the possible effect of acetylcholine on learning and on odor classification. According to the model, the piriform cortex can classify odors better when acetylcholine is present during learning but not present during recall. This is interesting since it suggests that learning and recall might be separate neurochemical modes (corresponding to whether or not acetylcholine is present). When acetylcholine is turned off at all times, even during learning, the model exhibits behavior somewhat similar to Alzheimer's disease, a disease associated with the degeneration of cells that distribute acetylcholine.

Chapters 4, 5, and 6 discuss algorithms appropriate for adaptive systems implemented entirely in analog hardware. The algorithms inject noise into the systems and correlate the noise with the outputs of the systems. This allows them to estimate gradients and to implement noisy versions of gradient descent, without having to calculate gradients explicitly. The methods require only noise generators, adders, multipliers, integrators, and differentiators; and the number of devices needed scales linearly with the number of adjustable parameters in the adaptive systems. With the exception of one global signal, the algorithms require only local information exchange.

A Data Dropout Compensation Algorithm Based on the Iterative Learning Control Methodology for Discrete-Time Systems

This paper deals with the convergence of a remote iterative learning control system subject to data dropouts. The system is composed by a set of discrete-time multiple input-multiple output linear models, each one with its corresponding actuator device and its sensor. Each actuator applies the input signals vector to its corresponding model at the sampling instants and the sensor measures the output signals vector. The iterative learning law is processed in a controller located far away of the models so the control signals vector has to be transmitted from the controller to the actuators through transmission channels. Such a law uses the measurements of each model to generate the input vector to be applied to its subsequent model so the measurements of the models have to be transmitted from the sensors to the controller. All transmissions are subject to failures which are described as a binary sequence taking value 1 or 0. A compensation dropout technique is used to replace the lost data in the transmission processes. The convergence to zero of the errors between the output signals vector and a reference one is achieved as the number of models tends to infinity.

Natural actor and belief critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs

This article presents a novel algorithm for learning parameters in statistical dialogue systems which are modeled as Partially Observable Markov Decision Processes (POMDPs). The three main components of a POMDP dialogue manager are a dialogue model representing dialogue state information; a policy that selects the system's responses based on the inferred state; and a reward function that specifies the desired behavior of the system. Ideally both the model parameters and the policy would be designed to maximize the cumulative reward. However, while there are many techniques available for learning the optimal policy, no good ways of learning the optimal model parameters that scale to real-world dialogue systems have been found yet. The presented algorithm, called the Natural Actor and Belief Critic (NABC), is a policy gradient method that offers a solution to this problem. Based on observed rewards, the algorithm estimates the natural gradient of the expected cumulative reward. The resulting gradient is then used to adapt both the prior distribution of the dialogue model parameters and the policy parameters. In addition, the article presents a variant of the NABC algorithm, called the Natural Belief Critic (NBC), which assumes that the policy is fixed and only the model parameters need to be estimated. The algorithms are evaluated on a spoken dialogue system in the tourist information domain. The experiments show that model parameters estimated to maximize the expected cumulative reward result in significantly improved performance compared to the baseline hand-crafted model parameters. The algorithms are also compared to optimization techniques using plain gradients and state-of-the-art random search algorithms. In all cases, the algorithms based on the natural gradient work significantly better. © 2011 ACM.

Research in development: learning from the CGIAR Research Program on Aquatic Agricultural Systems

This working paper aims to synthesize and share learning from the experience of adapting and operationalizing the Research in Development(RinD) approach to agricultural research in the five hubs under the The CGIAR Research Program on Aquatic Agricultural Systems. It seeks to share learning about how the approach is working in context and to explore the outcomes it is achieving through initial implementation over 3 ½ years. This learning can inform continuation of agricultural research in the second phase of the CGIAR research programs and will be useful to others aiming to implement research programs that seek to equitably build capacity to innovate in complex social-ecological systems. Each of the chapters in this working paper have shown that RinD has produced a range of outcomes that were often unexpected and broader in scope than might result from other approaches to agricultural research. RinD also produces innovations, and there is evidence that it builds capacity to innovate.

Reinforcement learning for parameter estimation in statistical spoken dialogue systems

Reinforcement techniques have been successfully used to maximise the expected cumulative reward of statistical dialogue systems. Typically, reinforcement learning is used to estimate the parameters of a dialogue policy which selects the system's responses based on the inferred dialogue state. However, the inference of the dialogue state itself depends on a dialogue model which describes the expected behaviour of a user when interacting with the system. Ideally the parameters of this dialogue model should be also optimised to maximise the expected cumulative reward. This article presents two novel reinforcement algorithms for learning the parameters of a dialogue model. First, the Natural Belief Critic algorithm is designed to optimise the model parameters while the policy is kept fixed. This algorithm is suitable, for example, in systems using a handcrafted policy, perhaps prescribed by other design considerations. Second, the Natural Actor and Belief Critic algorithm jointly optimises both the model and the policy parameters. The algorithms are evaluated on a statistical dialogue system modelled as a Partially Observable Markov Decision Process in a tourist information domain. The evaluation is performed with a user simulator and with real users. The experiments indicate that model parameters estimated to maximise the expected reward function provide improved performance compared to the baseline handcrafted parameters. © 2011 Elsevier Ltd. All rights reserved.

Satisfaction and acceptance with learning management systems

More and more often, universities make the decision to implement integrated learning management systems. Nevertheless, these technological developments are not realized without any trouble, and are achieved with more or less success and user satisfaction (Valenduc, 2000). It is why the presented study aims at identifying the factors influencing learning management system satisfaction and acceptance among students. The Technology Acceptance model created by Wixom and Todd (2005) studies information system acceptance through user satisfaction, and has the benefit of incorporating several ergonomic factors. More precisely, the survey, based on this model, investigates behavioral attitudes towards the system, perceived ease of use, perceived usefulness, as well as system satisfaction, information satisfaction and also incorporates two groups of factors affecting separately the two types of satisfaction. The study was conducted on a representative sample of 593 students from a Brussels university which had recently implemented an integrated learning management system. The results show on one hand, the impact of system reliability, accessibility, flexibility, lay-out and functionalities offered on system satisfaction. And on the other hand, the impact of information accuracy, intelligibility, relevance, exhaustiveness and actualization on information satisfaction. In conclusion, the results indicate the applicability of the theoretical model with learning management systems, and also highlight the importance of each aforementioned factor for a successful implantation of such a system in universities.

Specialized motor-driven dusp1 expression in the song systems of multiple lineages of vocal learning birds.

Mechanisms for the evolution of convergent behavioral traits are largely unknown. Vocal learning is one such trait that evolved multiple times and is necessary in humans for the acquisition of spoken language. Among birds, vocal learning is evolved in songbirds, parrots, and hummingbirds. Each time similar forebrain song nuclei specialized for vocal learning and production have evolved. This finding led to the hypothesis that the behavioral and neuroanatomical convergences for vocal learning could be associated with molecular convergence. We previously found that the neural activity-induced gene dual specificity phosphatase 1 (dusp1) was up-regulated in non-vocal circuits, specifically in sensory-input neurons of the thalamus and telencephalon; however, dusp1 was not up-regulated in higher order sensory neurons or motor circuits. Here we show that song motor nuclei are an exception to this pattern. The song nuclei of species from all known vocal learning avian lineages showed motor-driven up-regulation of dusp1 expression induced by singing. There was no detectable motor-driven dusp1 expression throughout the rest of the forebrain after non-vocal motor performance. This pattern contrasts with expression of the commonly studied activity-induced gene egr1, which shows motor-driven expression in song nuclei induced by singing, but also motor-driven expression in adjacent brain regions after non-vocal motor behaviors. In the vocal non-learning avian species, we found no detectable vocalizing-driven dusp1 expression in the forebrain. These findings suggest that independent evolutions of neural systems for vocal learning were accompanied by selection for specialized motor-driven expression of the dusp1 gene in those circuits. This specialized expression of dusp1 could potentially lead to differential regulation of dusp1-modulated molecular cascades in vocal learning circuits.

A meta-analysis of the effectiveness of intelligent tutoring systems on college students' academic learning

© 2013 American Psychological Association.This meta-analysis synthesizes research on the effectiveness of intelligent tutoring systems (ITS) for college students. Thirty-five reports were found containing 39 studies assessing the effectiveness of 22 types of ITS in higher education settings. Most frequently studied were AutoTutor, Assessment and Learning in Knowledge Spaces, eXtended Tutor-Expert System, and Web Interface for Statistics Education. Major findings include (a) Overall, ITS had a moderate positive effect on college students' academic learning (g = .32 to g = .37); (b) ITS were less effective than human tutoring, but they outperformed all other instruction methods and learning activities, including traditional classroom instruction, reading printed text or computerized materials, computer-assisted instruction, laboratory or homework assignments, and no-treatment control; (c) ITS's effectiveness did not significantly differ by different ITS, subject domain, or the manner or degree of their involvement in instruction and learning; and (d) effectiveness in earlier studies appeared to be significantly greater than that in more recent studies. In addition, there is some evidence suggesting the importance of teachers and pedagogy in ITS-assisted learning.

Web-based course support systems: an essential tool for supporting flexible learning environment

In Higher Education web-based course support systems are essential for supporting flexible learning environments. They provide tools to enable the interaction between student and tutor to reinforce transfer of theory to understanding particularly in an academic environment, therefore this paper will examine issues associated with the use of curriculum and learning resources within Web-based course support systems and the effectiveness of the resulting flexible learning environments This paper is a general discussion about flexible learning and in this case how it was applied to one of the courses at undergraduate level one. The first section will introduce what is flexible learning and the importance of flexible learning in Higher Education followed by the description of the course and why the flexible learning concepts is important in such a course and finally, how the flexibility was useful for this particular instance.