Semi-online neural-Q_leaming for real-time robot learning

Reinforcement learning (RL) is a very suitable technique for robot learning, as it can learn in unknown environments and in real-time computation. The main difficulties in adapting classic RL algorithms to robotic systems are the generalization problem and the correct observation of the Markovian state. This paper attempts to solve the generalization problem by proposing the semi-online neural-Q_learning algorithm (SONQL). The algorithm uses the classic Q_learning technique with two modifications. First, a neural network (NN) approximates the Q_function allowing the use of continuous states and actions. Second, a database of the most representative learning samples accelerates and stabilizes the convergence. The term semi-online is referred to the fact that the algorithm uses the current but also past learning samples. However, the algorithm is able to learn in real-time while the robot is interacting with the environment. The paper shows simulated results with the "mountain-car" benchmark and, also, real results with an underwater robot in a target following behavior

QoS online routing and MPLS multilevel protection: a survey

A survey of MPLS protection methods and their utilization in combination with online routing methods is presented in this article. Usually, fault management methods pre-establish backup paths to recover traffic after a failure. In addition, MPLS allows the creation of different backup types, and hence MPLS is a suitable method to support traffic-engineered networks. In this article, an introduction of several label switch path backup types and their pros and cons are pointed out. The creation of an LSP involves a routing phase, which should include QoS aspects. In a similar way, to achieve a reliable network the LSP backups must also be routed by a QoS routing method. When LSP creation requests arrive one by one (a dynamic network scenario), online routing methods are applied. The relationship between MPLS fault management and QoS online routing methods is unavoidable, in particular during the creation of LSP backups. Both aspects are discussed in this article. Several ideas on how these actual technologies could be applied together are presented and compared

Adding QoS protection in order to enhance MPLS QoS routing

In this paper, a method for enhancing current QoS routing methods by means of QoS protection is presented. In an MPLS network, the segments (links) to be protected are predefined and an LSP request involves, apart from establishing a working path, creating a specific type of backup path (local, reverse or global). Different QoS parameters, such as network load balancing, resource optimization and minimization of LSP request rejection should be considered. QoS protection is defined as a function of QoS parameters, such as packet loss, restoration time, and resource optimization. A framework to add QoS protection to many of the current QoS routing algorithms is introduced. A backup decision module to select the most suitable protection method is formulated and different case studies are analyzed

Full label space reduction in MPLS networks: asymmetric merged tunneling

Traffic Engineering objective is to optimize network resource utilization. Although several works have been published about minimizing network resource utilization in MPLS networks, few of them have been focused in LSR label space reduction. This letter studies Asymmetric Merged Tunneling (AMT) as a new method for reducing the label space in MPLS network. The proposed method may be regarded as a combination of label merging (proposed in the MPLS architecture) and asymmetric tunneling (proposed recently in our previous works). Finally, simulation results are performed by comparing AMT with both ancestors. They show a great improvement in the label space reduction factor

Estimation of the probability of congestion using Monte Carlo method in OPS networks

In networks with small buffers, such as optical packet switching based networks, the convolution approach is presented as one of the most accurate method used for the connection admission control. Admission control and resource management have been addressed in other works oriented to bursty traffic and ATM. This paper focuses on heterogeneous traffic in OPS based networks. Using heterogeneous traffic and bufferless networks the enhanced convolution approach is a good solution. However, both methods (CA and ECA) present a high computational cost for high number of connections. Two new mechanisms (UMCA and ISCA) based on Monte Carlo method are proposed to overcome this drawback. Simulation results show that our proposals achieve lower computational cost compared to enhanced convolution approach with an small stochastic error in the probability estimation

Entorn de simulació per a xarxes òptiques de transport

Aquest projecte final de carrera pretén investigar i experimentar una nova línia de desenvolupament d’algorismes dinàmics. A partir de l’algorisme AntNet-QoS [2] s’ incorporen noves mesures a l’algorisme (mesura de l’amplada de banda disponible i jitter), les quals combinant amb la mesura de retard ja feta servir, permet adaptar-se millor a les condicions actuals del trànsit en la xarxa i als requeriments específics de qualitat (QoS) per part del trànsit

Improving anemia management with less frequently dosed erythropoiesis stimulating agents

Introduction and Aims: The process of delivering erythropoiesis stimulating agents (ESAs) to hemodialysis patients (HD) is complex. Many European countries are requiring centers to document this process. To date, there has not been any comprehensive description of the operational aspects of ESA delivery in Europe. The objective of the Mercurius study was to describe the entire process of ESA delivery in dialysis centers. In addition, we explored the benefits of less frequent dosing. Methods: A conceptual model was developed to classify the sub-processes in the pharmacy, dialysis unit, waste unit, and back office. Within each dialysis unit activities associated with dose determination, ordering procedures, receipt and storage of ESAs, and ESA administration were measured. Within the pharmacy, ordering from supplier, receiving and storing, and delivering ESA to the dialysis unit were measured. The amount of time and materials associated with waste disposal and back office activities were also observed. We also evaluated the impact of less frequent dosing on the resources required to perform anemia management for HD patients. Structured interviews with staff were used to develop a comprehensive list of processes, sub-processes, and activities that are routinely followed to order, register, administer, and dispose of waste associated with ESAs. Each activity was evaluated to determine if less frequent dosing influenced the amount of resources required. A model was developed to estimate the change in resources consumed using less frequent dosing regimens. Results: Eight centers from 5 European countries (Belgium, France, Italy, Sweden, and Switzerland) participated in the study. The number of HD patients in each center ranged from 42 to 707 (mean=175). Across all of the centers, patients received a variety of dosing regimens (eg, TIW, BIW, QW and Q2W). The mean (±SD) time spent for the pharmacy to order an ESA from the supplier was 6.1 (±8.7) minutes; time spent in the dialysis unit and pharmacy for receiving and storing ESPs was 5.3 (±5.3) and 10.0 (±10.9) minutes, respectively; and time spent administering each injection was 6.4 (±6.5) minutes. Switching from current dosing practices to Q2W could decrease the mean number of syringes used from 12,420 to 5,085 per year. We estimate a reduction in the number of disinfective tissues and liquids of 58% and 71%, respectively by switching from current practice to dosing ESAs Q2W. Conclusions: There was significant variation in the time that it takes to perform routine ESA activities. We estimate that a reduction in resources required to manage anemia can be obtained by reducing the frequency of administration from the current mix of ESAs. These resources could be redeployed for patient care.

Accès sans prescription aux statines: bénéfique ou délétère. [Over-the-counter statins: benefit or harmful?]

Making statin available over the counter is one of the measures proposed to correct its underuse. Since May 2004, simvastatin 10 mg is sold over the counter in Great Britain. But uncertainties persist concerning the efficacy of statin in primary prevention and at a 10 mg dose. Finally, there is a risk of side effects and drug interactions. Beyond the correction of statin underuse and the hope of coronary heart disease mortality reduction, the British decision highlighted the will to give individuals a sense of responsibility concerning their health and its financial cost. Anyway, the benefit of switching statin from prescription to over the counter should be experimentally evaluated before its introduction.

Developmental changes in lateralized inhibition of symmetric movements in children with and without Developmental Coordination Disorder.

The present study investigates developmental changes in selective inhibition of symmetric movements with a lateralized switching task from bimanual to unimanual tapping in typically developing (TD) children and with Developmental Coordination Disorder (DCD) from 7 to 10 years old. Twelve right-handed TD children and twelve gender-matched children with DCD and probable DCD produce a motor switching task in which they have (1) to synchronize with the beat of an auditory metronome to produce bimanual symmetrical tapping and (2) to selectively inhibit their left finger's tapping while continuing their right finger's tapping and conversely. We assess (1) the development of the capacity to inhibit the stopping finger (number of supplementary taps after the stopping instruction) and (2) the development of the capacity to maintain the continuing finger (changes in the mean tempo and its variability for the continuing finger's tapping) and (3) the evolution of performance through trials. Results indicate that (1) TD children present an age-related increase in the capacity to inhibit and to maintain the left finger's tapping, (2) DCD exhibits persistent difficulties to inhibit the left finger's tapping, and (3) both groups improve their capacity to inhibit the left finger's movements through trials. In conclusion, the lateralized switching task provides a simple and fine tool to reveal differences in selective inhibition of symmetric movements in TD children and children with DCD. More theoretically, the specific improvement in selective inhibition of the left finger suggests a progressive development of inter-hemispheric communication during typical development that is absent or delayed in children with DCD.

Monitoring switch-type sensors and powering autonomous sensors via inductive coupling: application to removable seats in vehicles

Aquesta tesi explora la possibilitat de fer servir enllaços inductius per a una aplicació de l’automòbil on el cablejat entre la centraleta (ECU) i els sensors o detectors és difícil o impossible. S’han proposat dos mètodes: 1) el monitoratge de sensors commutats (dos possibles estats) via acoblament inductiu i 2) la transmissió mitjançant el mateix principi físic de la potència necessària per alimentar els sensors autònoms remots. La detecció d'ocupació i del cinturó de seguretat per a seients desmuntables pot ser implementada amb sistemes sense fils passius basats en circuits ressonants de tipus LC on l'estat dels sensors determina el valor del condensador i, per tant, la freqüència de ressonància. Els canvis en la freqüència són detectats per una bobina situada en el terra del vehicle. S’ha conseguit provar el sistema en un marge entre 0.5 cm i 3 cm. Els experiments s’han dut a terme fent servir un analitzador d’impedàncies connectat a una bobina primària i sensors comercials connectats a un circuit remot. La segona proposta consisteix en transmetre remotament la potència des d’una bobina situada en el terra del vehicle cap a un dispositiu autònom situat en el seient. Aquest dispositiu monitorarà l'estat dels detectors (d'ocupació i de cinturó) i transmetrà les dades mitjançant un transceptor comercial de radiofreqüència o pel mateix enllaç inductiu. S’han avaluat les bobines necessàries per a una freqüència de treball inferior a 150 kHz i s’ha estudiat quin és el regulador de tensió més apropiat per tal d’aconseguir una eficiència global màxima. Quatre tipus de reguladors de tensió s’han analitzat i comparat des del punt de vista de l’eficiència de potència. Els reguladors de tensió de tipus lineal shunt proporcionen una eficiència de potència millor que les altres alternatives, els lineals sèrie i els commutats buck o boost. Les eficiències aconseguides han estat al voltant del 40%, 25% i 10% per les bobines a distàncies 1cm, 1.5cm, i 2cm. Les proves experimentals han mostrat que els sensors autònoms han estat correctament alimentats fins a distàncies de 2.5cm.

Clinical and Virological Efficacy of Etravirine Plus Two Active Nucleos(t)ide Analogs in an Heterogeneous HIV-Infected Population.

Etravirine (ETV) is recommended in combination with a boosted protease inhibitor plus an optimized background regimen for salvage therapy, but there is limited experience with its use in combination with two nucleos(t)ide reverse-transcriptase inhibitors (NRTIs). This multicenter study aimed to assess the efficacy of this combination in two scenarios: group A) subjects without virologic failure on or no experience with non-nucleoside reverse-transcriptase inhibitors (NNRTIs) switched due to adverse events and group B) subjects switched after a virologic failure on an efavirenz- or nevirapine-based regimen. The primary endpoint was efficacy at 52 weeks analysed by intention-to-treat. Virologic failure was defined as the inability to suppress plasma HIV-RNA to <50 copies/mL after 24 weeks on treatment, or a confirmed viral load >200 copies/mL in patients who had previously achieved a viral suppression or had an undetectable viral load at inclusion. Two hundred eighty seven patients were included. Treatment efficacy rates in group A and B were 88.0% (CI95, 83.9-92.1%) and 77.4% (CI95, 65.0-89.7%), respectively; the rates reached 97.2% (CI95, 95.1-99.3%) and 90.5% (CI95, 81.7-99.3), by on-treatment analysis. The once-a-day ETV treatment was as effective as the twice daily dosing regimen. Grade 1-2 adverse events were observed motivating a treatment switch in 4.2% of the subjects. In conclusion, ETV (once- or twice daily) plus two analogs is a suitable, well-tolerated combination both as a switching strategy and after failure with first generation NNRTIs, ensuring full drug activity. TRIAL REGISTRATION ClinicalTrials.gov NCT01437241.

Mammalian genes are transcribed with widely different bursting kinetics.

In prokaryotes and eukaryotes, most genes appear to be transcribed during short periods called transcriptional bursts, interspersed by silent intervals. We describe how such bursts generate gene-specific temporal patterns of messenger RNA (mRNA) synthesis in mammalian cells. To monitor transcription at high temporal resolution, we established various gene trap cell lines and transgenic cell lines expressing a short-lived luciferase protein from an unstable mRNA, and recorded bioluminescence in real time in single cells. Mathematical modeling identified gene-specific on- and off-switching rates in transcriptional activity and mean numbers of mRNAs produced during the bursts. Transcriptional kinetics were markedly altered by cis-regulatory DNA elements. Our analysis demonstrated that bursting kinetics are highly gene-specific, reflecting refractory periods during which genes stay inactive for a certain time before switching on again.

Field testing of arsenic in groundwater samples of Bangladesh using a test kit based on lyophilized bioreporter bacteria.

A test kit based on living, lyophilized bacterial bioreporters emitting bioluminescence as a response to arsenite and arsenate was applied during a field campaign in six villages across Bangladesh. Bioreporter field measurements of arsenic in groundwater from tube wells were in satisfying agreement with the results of spectroscopic analyses of the same samples conducted in the lab. The practicability of the bioreporter test in terms of logistics and material requirements, suitability for high sample throughput, and waste disposal was much better than that of two commercial chemical test kits that were included as references. The campaigns furthermore demonstrated large local heterogeneity of arsenic in groundwater, underscoring the use of well switching as an effective remedy to avoid high arsenic exposure.

Modelling world investment markets using threshold conditional correlation models

In this paper we propose a parsimonious regime-switching approach to model the correlations between assets, the threshold conditional correlation (TCC) model. This method allows the dynamics of the correlations to change from one state (or regime) to another as a function of observable transition variables. Our model is similar in spirit to Silvennoinen and Teräsvirta (2009) and Pelletier (2006) but with the appealing feature that it does not suffer from the course of dimensionality. In particular, estimation of the parameters of the TCC involves a simple grid search procedure. In addition, it is easy to guarantee a positive definite correlation matrix because the TCC estimator is given by the sample correlation matrix, which is positive definite by construction. The methodology is illustrated by evaluating the behaviour of international equities, govenrment bonds and major exchange rates, first separately and then jointly. We also test and allow for different parts in the correlation matrix to be governed by different transition variables. For this, we estimate a multi-threshold TCC specification. Further, we evaluate the economic performance of the TCC model against a constant conditional correlation (CCC) estimator using a Diebold-Mariano type test. We conclude that threshold correlation modelling gives rise to a significant reduction in portfolio´s variance.

Discontinuation of enfuvirtide in heavily pretreated HIV-infected individuals.

BACKGROUND: Enfuvirtide was shown to be highly effective in treatment- experienced patients. Data on discontinuation of enfuvirtide and switch to new antiretroviral drugs are scarce. We aimed to evaluate the efficacy and the impact of discontinuing and/or switching enfuvirtide on virologic and clinical parameters in clinical practice. METHODS: All HIV-infected individuals participating in the Swiss HIV Cohort Study who were treated with enfuvirtide for at least 4 weeks in combination with an optimized background antiretroviral regimen were included in this study. RESULTS: A total of 151 patients were analyzed. The median baseline CD4 cell count was 108 cells/microL (interquartile range [IQR] 50-206) and HIV RNA was 4.7 log10 copies/mL (IQR 4.1-5.2). Virologic suppression, defined as a viral load below 50 copies/mL at 12 months, was achieved by 57.6% of patients. Overall, a median CD4 cell increase of 121 cells/microL (IQR 50-189) from baseline was noted. Up to 50% of patients discontinued enfuvirtide within the first year of treatment, mainly because of the patient's choice. After discontinuation of enfuvirtide, high rates of virologic failure and clinical progression were observed, notably when CD4 cell count at stopping enfuvirtide was below 100 cells/microL and no switch to new potent antiretroviral drugs such as darunavir, maraviroc, or raltegravir was performed. CONCLUSIONS: Enfuvirtide provides high virologic and immunologic response in treatment-experienced patients in the setting of clinical practice. Enfuvirtide should not be discontinued but should be replaced by new potent antiretrovirals, particularly in case of severe immunosuppression.