Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q_learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q_learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs

Policy gradient based Reinforcement Learning for real autonomous underwater cable tracking

This paper proposes a field application of a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot in cable tracking task. The learning system is characterized by using a direct policy search method for learning the internal state/action mapping. Policy only algorithms may suffer from long convergence times when dealing with real robotics. In order to speed up the process, the learning phase has been carried out in a simulated environment and, in a second step, the policy has been transferred and tested successfully on a real robot. Future steps plan to continue the learning process on-line while on the real robot while performing the mentioned task. We demonstrate its feasibility with real experiments on the underwater robot ICTINEU AUV

Autonomous underwater vehicle control using reinforcement learning policy search methods

Autonomous underwater vehicles (AUV) represent a challenging control problem with complex, noisy, dynamics. Nowadays, not only the continuous scientific advances in underwater robotics but the increasing number of subsea missions and its complexity ask for an automatization of submarine processes. This paper proposes a high-level control system for solving the action selection problem of an autonomous robot. The system is characterized by the use of reinforcement learning direct policy search methods (RLDPS) for learning the internal state/action mapping of some behaviors. We demonstrate its feasibility with simulated experiments using the model of our underwater robot URIS in a target following task

Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

A meta-analysis of the association of fracture risk and body mass index in women.

Several recent studies suggest that obesity may be a risk factor for fracture. The aim of this study was to investigate the association between body mass index (BMI) and future fracture risk at different skeletal sites. In prospective cohorts from more than 25 countries, baseline data on BMI were available in 398,610 women with an average age of 63 (range, 20-105) years and follow up of 2.2 million person-years during which 30,280 osteoporotic fractures (6457 hip fractures) occurred. Femoral neck BMD was measured in 108,267 of these women. Obesity (BMI ≥ 30 kg/m(2) ) was present in 22%. A majority of osteoporotic fractures (81%) and hip fractures (87%) arose in non-obese women. Compared to a BMI of 25 kg/m(2) , the hazard ratio (HR) for osteoporotic fracture at a BMI of 35 kg/m(2) was 0.87 (95% confidence interval [CI], 0.85-0.90). When adjusted for bone mineral density (BMD), however, the same comparison showed that the HR for osteoporotic fracture was increased (HR, 1.16; 95% CI, 1.09-1.23). Low BMI is a risk factor for hip and all osteoporotic fracture, but is a protective factor for lower leg fracture, whereas high BMI is a risk factor for upper arm (humerus and elbow) fracture. When adjusted for BMD, low BMI remained a risk factor for hip fracture but was protective for osteoporotic fracture, tibia and fibula fracture, distal forearm fracture, and upper arm fracture. When adjusted for BMD, high BMI remained a risk factor for upper arm fracture but was also a risk factor for all osteoporotic fractures. The association between BMI and fracture risk is complex, differs across skeletal sites, and is modified by the interaction between BMI and BMD. At a population level, high BMI remains a protective factor for most sites of fragility fracture. The contribution of increasing population rates of obesity to apparent decreases in fracture rates should be explored. © 2014 American Society for Bone and Mineral Research.

Visuo-motor and cognitive procedural learning in children with basal ganglia pathology.

We investigated procedural learning in 18 children with basal ganglia (BG) lesions or dysfunctions of various aetiologies, using a visuo-motor learning test, the Serial Reaction Time (SRT) task, and a cognitive learning test, the Probabilistic Classification Learning (PCL) task. We compared patients with early (<1 year old, n=9), later onset (>6 years old, n=7) or progressive disorder (idiopathic dystonia, n=2). All patients showed deficits in both visuo-motor and cognitive domains, except those with idiopathic dystonia, who displayed preserved classification learning skills. Impairments seem to be independent from the age of onset of pathology. As far as we know, this study is the first to investigate motor and cognitive procedural learning in children with BG damage. Procedural impairments were documented whatever the aetiology of the BG damage/dysfunction and time of pathology onset, thus supporting the claim of very early skill learning development and lack of plasticity in case of damage.

The T-381C SNP in BNP gene may be modestly associated with type 2 diabetes: an updated meta-analysis in 49 279 subjects.

A recent study reported an association between the brain natriuretic peptide (BNP) promoter T-381C polymorphism (rs198389) and protection against type 2 diabetes (T2D). As replication in several studies is mandatory to confirm genetic results, we analyzed the T-381C polymorphism in seven independent case-control cohorts and in 291 T2D-enriched pedigrees totalling 39 557 subjects of European origin. A meta-analysis of the seven case-control studies (n = 39 040) showed a nominal protective effect [odds ratio (OR) = 0.86 (0.79-0.94), P = 0.0006] of the CC genotype on T2D risk, consistent with the previous study. By combining all available data (n = 49 279), we further confirmed a modest contribution of the BNP T-381C polymorphism for protection against T2D [OR = 0.86 (0.80-0.92), P = 1.4 x 10(-5)]. Potential confounders such as gender, age, obesity status or family history were tested in 4335 T2D and 4179 normoglycemic subjects and they had no influence on T2D risk. This study provides further evidence of a modest contribution of the BNP T-381C polymorphism in protection against T2D and illustrates the difficulty of unambiguously proving modest-sized associations even with large sample sizes.

Adaptive intelligent agent approach to guide the web navigation on the PLAN-G distance learning platform

Hypermedia systems based on the Web for open distance education are becoming increasinglypopular as tools for user-driven access learning information. Adaptive hypermedia is a new direction in research within the area of user-adaptive systems, to increase its functionality by making it personalized [Eklu 961. This paper sketches a general agents architecture to include navigationaladaptability and user-friendly processes which would guide and accompany the student during hislher learning on the PLAN-G hypermedia system (New Generation Telematics Platform to Support Open and Distance Learning), with the aid of computer networks and specifically WWW technology [Marz 98-1] [Marz 98-2]. The PLAN-G actual prototype is successfully used with some informatics courses (the current version has no agents yet). The propased multi-agent system, contains two different types of adaptive autonomous software agents: Personal Digital Agents {Interface), to interacl directly with the student when necessary; and Information Agents (Intermediaries), to filtrate and discover information to learn and to adapt navigation space to a specific student

A teaching/learning support tool for introductory programming courses

In This work we present a Web-based tool developed with the aim of reinforcing teaching and learning of introductory programming courses. This tool provides support for teaching and learning. From the teacher's perspective the system introduces important gains with respect to the classical teaching methodology. It reinforces lecture and laboratory sessions, makes it possible to give personalized attention to the student, assesses the degree of participation of the students and most importantly, performs a continuous assessment of the student's progress. From the student's perspective it provides a learning framework, consisting in a help environment and a correction environment, which facilitates their personal work. With this tool students are more motivated to do programming

Long circulating half-life and high tumor selectivity of the photosensitizer meta-tetrahydroxyphenylchlorin conjugated to polyethylene glycol in nude mice grafted with a human colon carcinoma.

In a mode of nude mice bearing a human colon carcinoma xenograft, the biodistribution and tumor localization of metatetrahydroxyphenylchlorin (m-THPC) coupled to polyethylene glycol (PEG) were compared with those of the free form of this photosensitizer used in photodynamic therapy (PDT). At different times after i.v. injection of both forms of 125I-labeled photosensitizer, m-THPC-PEG gave on average a 2-fold higher tumor uptake than free m-THPC. In addition, at early times after injection, m-THPC-PEG showed a 2-fold longer blood circulating half-life and a 4-fold lower liver uptake than free m-THPC. The tumor to normal tissue ratios of radioactivity concentrations were always higher for m-THPC-PEG than for free m-THPC at any time point studied from 2 to 96 hr post-injection. Significant coefficients of correlation between direct fluorescence measurements and radioactivity counting were obtained within each organ tested. Fluorescence microscopy studies showed that m-THPC-PEG was preferentially localized near the tumor vessels, whereas m-THPC was more diffusely distributed inside the tumor tissue. To verify whether m-THPC-PEG conjugate remained phototoxic in vivo, PDT experiments were performed 72 hr after injection and showed that m-THPC-PEG was as potent as free m-THPC in the induction of tumor regression provided that the irradiation does for m-THPC-PEG conjugate was adapted to a well-tolerated 2-fold higher level. The overall results demonstrate first the possibility of improving the in vivo tumor localization of a hydrophobic dye used for PDT by coupling it to PEG and second that a photosensitizer conjugated to a macromolecule can remain phototoxic in vivo.

Correlation of meta 1 expression with culture stage, cell morphology and infectivity in Leishmania (Leishmania) amazonensis promastigotes

The parasitic protozoan Leishmania (Leishmania) amazonensis alternates between mammalian and insect hosts. In the insect host, the parasites proliferate as procyclic promastigotes andthen differentiate into metacyclic infective forms. The meta 1 gene is preferentially expressed during metacyclogenesis. Meta 1 expression profile determination along parasite growth curves revealed that the meta 1 mRNA level peaked at the early stationary phase then decreased to an intermediate level. No correlation was observed between meta 1 expression and infectivity. Conversely, infectivity correlated with the increase of apoptotic cells in the late stationary phase.

Aversive Stimuli and Functional Networks of Fear Learning in PTSD : 734

Background: One characteristic of post traumatic stress disorder is an inability to adapt to a safe environment i.e. to change behavior when predictions of adverse outcomes are not met. Recent studies have also indicated that PTSD patients have altered pain processing, with hyperactivation of the putamen and insula to aversive stimuli (Geuze et al, 2007). The present study examined neuronal responses to aversive and predicted aversive events. Methods: Twenty-four trauma exposed non-PTSD controls and nineteen subjects with PTSD underwent fMRI imaging during a partial reinforcement fear conditioning paradigm, with a mild electric shock as the unconditioned stimuli (UCS). Three conditions were analyzed: actual presentations of the UCS, events when a UCS was expected, but omitted (CS+), and events when the UCS was neither expected nor delivered (CS-). Results: The UCS evoked significant alterations in the pain matrix consisting of the brainstem, the midbrain, the thalamus, the insula, the anterior and middle cingulate and the contralateral somatosensory cortex. PTSD subjects displayed bilaterally elevated putamen activity to the electric shock, as compared to controls. In trials when USC was expected, but omitted, significant activations were observed in the brainstem, the midbrain, the anterior insula and the anterior cingulate. PTSD subjects displayed similar activations, but also elevated activations in the amygdala and the posterior insula. Conclusions: These results indicate altered fear and safety learning in PTSD, and neuronal activations are further explored in terms of functional connectivity using psychophysiological interaction analyses.

Clinical performance of cervical restorations: a meta-analysis.

OBJECTIVES: To carry out a meta-analysis in order to assess the influencing factors on retention loss and marginal discoloration of cervical restorations made of composites and glass ionomer (derivates). METHODS: The literature was searched for prospective clinical studies on cervical restorations with an observation period of at least 18 months. RESULTS: Fifty clinical studies involving 40 adhesive systems matched the inclusion criteria. On average, 10% of the cervical fillings were lost and 24% exhibited marginal discoloration after 3 years. The variability ranged from 0% to 50% for retention loss and from 0% to 74% for marginal discoloration. Hardly any secondary caries was detected. When linear mixed models with a study and experiment effect were used, the analysis revealed that the adhesive/restorative class had the most significant influence, with 2-step self-etching adhesive systems performing best and 1-step self-etching adhesive systems performing worst; 3-step etch-and-rinse systems, glass ionomers/resin-modified glass ionomers, 2-step etch-and-rinse systems and polyacid-modified resin composites were ranked in between. Restorations placed in teeth whose dentin/enamel had been prepared/roughened showed a statistically significant higher retention rate than those placed in teeth with unprepared dentin (p<0.05). Beveling of the enamel and the type of isolation used (rubberdam/cotton rolls) had no significant influence. SIGNIFICANCE: The clinical performance of cervical restorations is significantly influenced by the type of adhesive system used and/or the adhesive class to which the system belonged and whether the dentin/enamel is prepared or not. 2-Step self-etching- and 3-step etch&rinse systems shall be chosen over 1-step self-etching systems and glass ionomer derivates. The dentin (and enamel) surface shall be roughened before placement of the restoration.