Directed Selection using Reinforcement Learning for the Examination Timetabling Problem

Traditional heuristic approaches to the Examination Timetabling Problem normally utilize a stochastic method during Optimization for the selection of the next examination to be considered for timetabling within the neighbourhood search process. This paper presents a technique whereby the stochastic method has been augmented with information from a weighted list gathered during the initial adaptive construction phase, with the purpose of intelligently directing examination selection. In addition, a Reinforcement Learning technique has been adapted to identify the most effective portions of the weighted list in terms of facilitating the greatest potential for overall solution improvement. The technique is tested against the 2007 International Timetabling Competition datasets with solutions generated within a time frame specified by the competition organizers. The results generated are better than those of the competition winner in seven of the twelve examinations, while being competitive for the remaining five examinations. This paper also shows experimentally how using reinforcement learning has improved upon our previous technique.

A heterosynaptic learning rule for neural networks

In this article we intoduce a novel stochastic Hebb-like learning rule for neural networks that is neurobiologically motivated. This learning rule combines features of unsupervised (Hebbian) and supervised (reinforcement) learning and is stochastic with respect to the selection of the time points when a synapse is modified. Moreover, the learning rule does not only affect the synapse between pre- and postsynaptic neuron, which is called homosynaptic plasticity, but effects also further remote synapses of the pre-and postsynaptic neuron. This more complex form of synaptic plasticity has recently come under investigations in neurobiology and is called heterosynaptic plasticity. We demonstrate that this learning rule is useful in training neural networks by learning parity functions including the exclusive-or (XOR) mapping in a multilayer feed-forward network. We find, that our stochastic learning rule works well, even in the presence of noise. Importantly, the mean leaxning time increases with the number of patterns to be learned polynomially, indicating efficient learning.

Road traffic congestion and public information - An experimental investigation

This paper reports laboratory experiments designed to study the impact of public information about past departure rates on congestion levels and travel costs. Our design is based on a discrete version of Arnott et al.'s (1990) bottleneck model. In all treatments, congestion occurs and the observed travel costs are quite similar to the predicted ones. Subjects' capacity to coordinate is not affected by the availability of public information on past departure rates, by the number of drivers or by the relative cost of delay. This seemingly absence of treatment effects is confirmed by our finding that a parameter-free reinforcement learning model best characterises individual behaviour.

Decentralized Computation of Pareto Optimal Pure Nash Equilibria of Boolean Games with Privacy Concerns

In Boolean games, agents try to reach a goal formulated as a Boolean formula. These games are attractive because of their compact representations. However, few methods are available to compute the solutions and they are either limited or do not take privacy or communication concerns into account. In this paper we propose the use of an algorithm related to reinforcement learning to address this problem. Our method is decentralized in the sense that agents try to achieve their goals without knowledge of the other agents’ goals. We prove that this is a sound method to compute a Pareto optimal pure Nash equilibrium for an interesting class of Boolean games. Experimental results are used to investigate the performance of the algorithm.

Cortisol, contingency learning, and memory in preterm and full-term infants

Cortisol plays an important role in learning and memory. An inverted-U shaped function has been proposed to account for the positive and negative effects of cortisol on cognitive performance and memory in adults, such that too little or too much impair but moderate amounts facilitate performance. Whether such relationships between cortisol and mental function apply to early infancy, when cortisol secretion, learning, and memory undergo rapid developmental changes, is unknown. We compared relationships between learning/memory and cortisol in preterm and full-term infants and examined whether a greater risk for adrenal insufficiency associated with prematurity produces differential cortisol-memory relationships. Learning in three-month old (corrected for gestational age) preterm and full-term infants was evaluated using a conjugate reinforcement mobile task. Memory was tested by repeating the same task 24h later. Salivary cortisol samples were collected before and 20 min after the presentation of the mobile. We found that preterm infants had lower cortisol levels and smaller cortisol responses than full-term infants. This is consistent with relative adrenal insufficiency reported in the neonatal period. Infants who showed increased cortisol levels from 0 to 20 min on Day 1 had significantly better memory, regardless of prematurity, than infants who showed decreased cortisol levels.