Learning Multirobot Hose Transportation and Deployment by Distributed Round-Robin Q-Learning

Multi-Agent Reinforcement Learning (MARL) algorithms face two main difficulties: the curse of dimensionality, and environment non-stationarity due to the independent learning processes carried out by the agents concurrently. In this paper we formalize and prove the convergence of a Distributed Round Robin Q-learning (D-RR-QL) algorithm for cooperative systems. The computational complexity of this algorithm increases linearly with the number of agents. Moreover, it eliminates environment non sta tionarity by carrying a round-robin scheduling of the action selection and execution. That this learning scheme allows the implementation of Modular State-Action Vetoes (MSAV) in cooperative multi-agent systems, which speeds up learning convergence in over-constrained systems by vetoing state-action pairs which lead to undesired termination states (UTS) in the relevant state-action subspace. Each agent's local state-action value function learning is an independent process, including the MSAV policies. Coordination of locally optimal policies to obtain the global optimal joint policy is achieved by a greedy selection procedure using message passing. We show that D-RR-QL improves over state-of-the-art approaches, such as Distributed Q-Learning, Team Q-Learning and Coordinated Reinforcement Learning in a paradigmatic Linked Multi-Component Robotic System (L-MCRS) control problem: the hose transportation task. L-MCRS are over-constrained systems with many UTS induced by the interaction of the passive linking element and the active mobile robots.

Altered ultrasonic vocalization and impaired learning and memory in Angelman syndrome mouse model with a large maternal deletion from Ube3a to Gabrb3.

Angelman syndrome (AS) is a neurobehavioral disorder associated with mental retardation, absence of language development, characteristic electroencephalography (EEG) abnormalities and epilepsy, happy disposition, movement or balance disorders, and autistic behaviors. The molecular defects underlying AS are heterogeneous, including large maternal deletions of chromosome 15q11-q13 (70%), paternal uniparental disomy (UPD) of chromosome 15 (5%), imprinting mutations (rare), and mutations in the E6-AP ubiquitin ligase gene UBE3A (15%). Although patients with UBE3A mutations have a wide spectrum of neurological phenotypes, their features are usually milder than AS patients with deletions of 15q11-q13. Using a chromosomal engineering strategy, we generated mutant mice with a 1.6-Mb chromosomal deletion from Ube3a to Gabrb3, which inactivated the Ube3a and Gabrb3 genes and deleted the Atp10a gene. Homozygous deletion mutant mice died in the perinatal period due to a cleft palate resulting from the null mutation in Gabrb3 gene. Mice with a maternal deletion (m-/p+) were viable and did not have any obvious developmental defects. Expression analysis of the maternal and paternal deletion mice confirmed that the Ube3a gene is maternally expressed in brain, and showed that the Atp10a and Gabrb3 genes are biallelically expressed in all brain sub-regions studied. Maternal (m-/p+), but not paternal (m+/p-), deletion mice had increased spontaneous seizure activity and abnormal EEG. Extensive behavioral analyses revealed significant impairment in motor function, learning and memory tasks, and anxiety-related measures assayed in the light-dark box in maternal deletion but not paternal deletion mice. Ultrasonic vocalization (USV) recording in newborns revealed that maternal deletion pups emitted significantly more USVs than wild-type littermates. The increased USV in maternal deletion mice suggests abnormal signaling behavior between mothers and pups that may reflect abnormal communication behaviors in human AS patients. Thus, mutant mice with a maternal deletion from Ube3a to Gabrb3 provide an AS mouse model that is molecularly more similar to the contiguous gene deletion form of AS in humans than mice with Ube3a mutation alone. These mice will be valuable for future comparative studies to mice with maternal deficiency of Ube3a alone.

A Formulation for Active Learning with Applications to Object Detection

We discuss a formulation for active example selection for function learning problems. This formulation is obtained by adapting Fedorov's optimal experiment design to the learning problem. We specifically show how to analytically derive example selection algorithms for certain well defined function classes. We then explore the behavior and sample complexity of such active learning algorithms. Finally, we view object detection as a special case of function learning and show how our formulation reduces to a useful heuristic to choose examples to reduce the generalization error.

On the Relationship Between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Functions

In this paper, we bound the generalization error of a class of Radial Basis Function networks, for certain well defined function learning tasks, in terms of the number of parameters and number of examples. We show that the total generalization error is partly due to the insufficient representational capacity of the network (because of its finite size) and partly due to insufficient information about the target function (because of finite number of samples). We make several observations about generalization error which are valid irrespective of the approximation scheme. Our result also sheds light on ways to choose an appropriate network architecture for a particular problem.

Regulation of topographic projection in the brain: Elf-1 in the hippocamposeptal system.

The hippocampus and septum play central roles in one of the most important spheres of brain function: learning and memory. Although their topographic connections have been known for two decades and topography may be critical for cognitive functions, the basis for hippocamposeptal topographic projection is unknown. We now report for the first time that Elf-1, a membrane-bound eph family ligand, is a candidate molecular tag for the genesis of the hippocamposeptal topographic projection. Elf-1 is expressed in an increasing gradient from dorsal to ventral septum. Furthermore, Elf-1 selectively allows growth of neurites from topographically appropriate lateral hippocampal neurons, while inhibiting neurite outgrowth by medial hippocampal neurons. Complementary to the expression of Elf-1, an eph family receptor, Bsk, is expressed in the hippocampus in a lateral to medial gradient, consistent with a function as a receptor for Elf-1. Further, Elf-1 specifically bound Bsk, eliciting tyrosine kinase activity. We conclude that the Elf-1/Bsk ligand-receptor pair exhibits traits of a chemoaffinity system for the organization of hippocamposeptal topographic projections.

A comparison of picture description abilities in individuals with vascular subcortical lesions and Huntington's Disease

The lexical-semantic and syntactic abilities of a group of individuals with chronic nonthalamic subcortical (NS) lesions following stroke (n = 6) were investigated using the Western Aphasia Battery (WAB) picture description task [Kertesz, A. (1982). The Western aphasia battery. New York: Grune and Stratton] and compared with those of a group of subjects with Huntington's Disease (HD) (n = 6) and a nonneurologically impaired control group (n = 6) matched for age, sex, and educational level. The performance of the NS and HD subjects did not differ significantly from the well controls on measures of lexical-semantic abilities. NS and HD subjects provided as much information about the target picture as control subjects, but produced fewer action information units. Analysis of syntactic abilities revealed that the HD subjects produced significantly more grammatical errors than both the NS and control subjects and that the NS group performed in a similar manner to control subjects. These findings are considered in terms of current theories of subcortical language function Learning outcomes: As a result of this activity, the reader will obtain information about the debate surrounding the role of subcortical language mechanisms and be provided with new information on the comparative picture description abilities of individuals with known vascular and degenerative subcortical pathologies and healthy control participants. (c) 2005 Elsevier Inc. All rights reserved.

Why people underestimate y when extrapolating in linear functions

E. L. DeLosh, J. R. Busemeyer, and M. A. McDaniel (1997) found that when learning a positive, linear relationship between a continuous predictor (x) and a continuous criterion (y), trainees tend to underestimate y on items that ask the trainee to extrapolate. In 3 experiments, the authors examined the phenomenon and found that the tendency to underestimate y is reliable only in the so-called lower extrapolation region-that is, new values of x that lie between zero and the edge of the training region. Existing models of function learning, such as the extrapolation-association model (DeLosh et al., 1997) and the population of linear experts model (M. L. Kalish, S. Lewandowsky, & J. Kruschke, 2004), cannot account for these results. The authors show that with minor changes, both models can predict the correct pattern of results.

Organização nuclear do sistema serotonérgico no encéfalo de morcegos Artibeus planirostris

Serotonin or 5-hydroxytryptamine (5-HT) is a substance found in many tissues of the body, including the nervous system acting as a neurotransmitter. Within the neuro-axis, the location of the majority of the 5-HT neurons is superimposed with raphe nuclei of the brain stem, in the median line or its vicinity, so that neuronal 5-HT can be considered a marker of the raphe nuclei. Serotonergic neurons are projected to almost all areas of the brain. Studies show the participation of serotonin in regulating the temperature, feeding behavior, sexual behavior, biological rhythms, sleep, locomotor function, learning, among others. The anatomy of these groups has been revised in many species, including mouse, rabbit, cat and primates, but never before in a bat species from South America. This study aimed to characterize the serotonergic clusters in the brain of the bat Artibeus planirostris through immunohistochemistry for serotonin. Seven adult bat males of Artibeus planirostris species (Microchiroptera, Mammalia) were used in this study. The animals were anesthetized, transcardially perfused and their brains were removed. Coronal sections of the frozen brain of bats were obtained in sliding microtome and subjected to immunohistochemistry for 5-HT. Delimit the caudal linear (CLi), dorsal (DR), median (MnR), paramedian (PMnR), pontine (PNR), magnus (MgR), pallidus (RPA) and obscurus (ROb) raphe nucleus, in addition to the groups B9 and rostral and caudal ventrolateral (RVL/CVL). The serotonergic groups of this kind of cheiroptera present morphology and cytoarchitecture relatively similar to that described in rodents and primates, confirming the phylogenetic stability of these cell clusters.

Self-regulated learning and executive function : exploring the relationships in a sample of adolescent males

This study investigated relationships between SRL and EF in a sample of 254 school-aged adolescent males. Two hypotheses were tested: that self-reported measures of SRL and EF are closely related and that as different aspects of EF mature during adolescence, the corresponding components of SRL should also improve, leading to an age-related increase in the correlation between EF and SRL. Two self-report instruments were used: the strategies for self-regulated learning survey (SSRLS) and the behavioural rating instrument of executive function (BRIEF). Strong correlations between the measures of EF and SRL were found, especially in areas associated with metacognitive processes. Correlations between EF and SRL were found, with weaker correlations between behavioural regulation and SRL were found to be weaker for the younger participants in the sample while the relationship between EF and SRL appears to grow stronger during the initial years of high school even though self-reported levels of EF along with motivation for SRL and important components of SRL such as goal setting and planning were found to decrease with age. Decreasing levels of motivation for learning during adolescence are speculated to moderate the deployment of SRL and EF in a school context.

Early mathematical learning: Number processing skills and executive function at 5 and 8 years of age

This research investigated differences and associations in performance in number processing and executive function for children attending primary school in a large Australian metropolitan city. In a cross-sectional study, performance of 25 children in the first full-time year of school, (Prep; mean age = 5.5 years) and 21 children in Year 3 (mean age = 8.5 years) completed three number processing tasks and three executive function tasks. Year 3 children consistently outperformed the Prep year children on measures of accuracy and reaction time, on the tasks of number comparison, calculation, shifting, and inhibition but not on number line estimation. The components of executive function (shifting, inhibition, and working memory) showed different patterns of correlation to performance on number processing tasks across the early years of school. Findings could be used to enhance teachers’ understanding about the role of the cognitive processes employed by children in numeracy learning, and so inform teachers’ classroom practices.

Reinforcement Learning With Function Approximation for Traffic Signal Control

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e. g., the work of Abdulhai et al., on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai et al. and Cools et al., as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai et al. We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Learning polyhedral classifiers using logistic function

In this paper we propose a new algorithm for learning polyhedral classifiers. In contrast to existing methods for learning polyhedral classifier which solve a constrained optimization problem, our method solves an unconstrained optimization problem. Our method is based on a logistic function based model for the posterior probability function. We propose an alternating optimization algorithm, namely, SPLA1 (Single Polyhedral Learning Algorithm1) which maximizes the loglikelihood of the training data to learn the parameters. We also extend our method to make it independent of any user specified parameter (e.g., number of hyperplanes required to form a polyhedral set) in SPLA2. We show the effectiveness of our approach with experiments on various synthetic and real world datasets and compare our approach with a standard decision tree method (OC1) and a constrained optimization based method for learning polyhedral sets.

A novel Q-learning algorithm with function approximation for constrained Markov decision processes

We present a novel multi-timescale Q-learning algorithm for average cost control in a Markov decision process subject to multiple inequality constraints. We formulate a relaxed version of this problem through the Lagrange multiplier method. Our algorithm is different from Q-learning in that it updates two parameters - a Q-value parameter and a policy parameter. The Q-value parameter is updated on a slower time scale as compared to the policy parameter. Whereas Q-learning with function approximation can diverge in some cases, our algorithm is seen to be convergent as a result of the aforementioned timescale separation. We show the results of experiments on a problem of constrained routing in a multistage queueing network. Our algorithm is seen to exhibit good performance and the various inequality constraints are seen to be satisfied upon convergence of the algorithm.