Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Reactive Recruitment of Attentional Control in Math Anxiety: An ERP Study of Numeric Conflict Monitoring and Adaptation

This study uses event-related brain potentials (ERPs) to investigate the electrophysiological correlates of numeric conflict monitoring in math-anxious individuals, by analyzing whether math anxiety is related to abnormal processing in early conflict detection (as shown by the N450 component) and/or in a later, response-related stage of processing (as shown by the conflict sustained potential; Conflict-SP). Conflict adaptation effects were also studied by analyzing the effect of the previous trial"s congruence in current interference. To this end, 17 low math-anxious (LMA)and 17 high math-anxious (HMA) individuals were presented with a numerical Stroop task. Groups were extreme in math anxiety but did not differ in trait or state anxiety or in simple math ability. The interference effect of the current trial (incongruent-congruent) and the interference effect preceded by congruence and by incongruity were analyzed both for behavioral measures and for ERPs. A greater interference effect was found for response times in the HMA group than in the LMA one. Regarding ERPs, the LMA group showed a greater N450 component for the interference effect preceded by congruence than when preceded by incongruity, while the HMA group showed greater Conflict-SP amplitude for the interference effect preceded by congruence than when preceded by incongruity. Our study showed that the electrophysiological correlates of numeric interference in HMA individuals comprise the absence of a conflict adaptation effect in the first stage of conflict processing (N450) and an abnormal subsequent up-regulation of cognitive control in order to overcome the conflict (Conflict-SP). More concretely, our study shows that math anxiety is related to a reactive and compensatory recruitment of control resources that is implemented only when previously exposed to a stimuli presenting conflicting information

Developmental and tissue-specific involvement of peroxisome proliferator-activated receptor-alpha in the control of mouse uncoupling protein-3 gene expression.

Uncoupling protein-3 (UCP3) is a member of the mitochondrial carrier family expressed preferentially in skeletal muscle and heart. It appears to be involved in metabolic handling of fatty acids in a way that minimizes excessive production of reactive oxygen species. Fatty acids are powerful regulators of UCP3 gene transcription. We have found that the role of peroxisome proliferator-activated receptor-α (PPARα) on the control of UCP3 gene expression depends on the tissue and developmental stage. In adults, UCP3 mRNA expression is unaltered in skeletal muscle from PPARα-null mice both in basal conditions and under the stimulus of starvation. In contrast, UCP3 mRNA is down-regulated in adult heart both in fed and fasted PPARα-null mice. This occurs despite the increased levels of free fatty acids caused by fasting in PPARα-null mice. In neonates, PPARα-null mice show impaired UCP3 mRNA expression in skeletal muscle in response to milk intake, and this is not a result of reduced free fatty acid levels. The murine UCP3 promoter is activated by fatty acids through either PPARα or PPARδ but not by PPARγ or retinoid X receptor alone. PPARδ-dependent activation could be a potential compensatory mechanism to ensure appropriate expression of UCP3 gene in adult skeletal muscle in the absence of PPARα. However, among transcripts from other PPARα and PPARδ target genes, only those acutely induced by milk intake in wild-type neonates were altered in muscle or heart from PPARα-null neonates. Thus, PPARα-dependent regulation is required for appropriate gene regulation of UCP3 as part of the subset of fatty-acid-responsive genes in neonatal muscle and heart.