Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Predicting spatial patterns of plant species richness: a comparison of direct macroecological and species stacking approaches

Aim This study compares the direct, macroecological approach (MEM) for modelling species richness (SR) with the more recent approach of stacking predictions from individual species distributions (S-SDM). We implemented both approaches on the same dataset and discuss their respective theoretical assumptions, strengths and drawbacks. We also tested how both approaches performed in reproducing observed patterns of SR along an elevational gradient.Location Two study areas in the Alps of Switzerland.Methods We implemented MEM by relating the species counts to environmental predictors with statistical models, assuming a Poisson distribution. S-SDM was implemented by modelling each species distribution individually and then stacking the obtained prediction maps in three different ways - summing binary predictions, summing random draws of binomial trials and summing predicted probabilities - to obtain a final species count.Results The direct MEM approach yields nearly unbiased predictions centred around the observed mean values, but with a lower correlation between predictions and observations, than that achieved by the S-SDM approaches. This method also cannot provide any information on species identity and, thus, community composition. It does, however, accurately reproduce the hump-shaped pattern of SR observed along the elevational gradient. The S-SDM approach summing binary maps can predict individual species and thus communities, but tends to overpredict SR. The two other S-SDM approaches the summed binomial trials based on predicted probabilities and summed predicted probabilities - do not overpredict richness, but they predict many competing end points of assembly or they lose the individual species predictions, respectively. Furthermore, all S-SDM approaches fail to appropriately reproduce the observed hump-shaped patterns of SR along the elevational gradient.Main conclusions Macroecological approach and S-SDM have complementary strengths. We suggest that both could be used in combination to obtain better SR predictions by following the suggestion of constraining S-SDM by MEM predictions.

Comparison of direct and indirect alcohol markers with PEth in blood and urine in alcohol dependent inpatients during detoxication.

The importance of direct and indirect alcohol markers to evaluate alcohol consumption in clinical and forensic settings is increasingly recognized. While some markers are used to prove abstinence from ethanol, other markers are suitable for detection of alcohol misuse. Phosphatidyl ethanol (PEth) is ranked among the latter. There is only little information about the correlation between PEth and other currently used markers (ethyl glucuronide, ethyl sulfate, carbohydrate deficient transferrin, gamma-glutamyl transpeptidase, and methanol) and about their decline during detoxification. To get more information, 18 alcohol-dependent patients in withdrawal therapy were monitored for these parameters in blood and urine for up to 19 days. There was no correlation between the different markers. PEth showed a rapid decrease at the beginning of the intervention, a slow decline after the first few days, and could still be detected after 19 days of abstinence from ethanol.

Real-time molecular epidemiology of tuberculosis by direct genotyping of smear-positive clinical specimens.

We applied MIRU-VNTR (mycobacterial interspersed repetitive-unit-variable-number tandem-repeat typing) to directly analyze the bacilli present in 61 stain-positive specimens from tuberculosis patients. A complete MIRU type (24 loci) was obtained for all but one (no amplification in one locus) of the specimens (98.4%), and the allelic values fully correlated with those obtained from the corresponding cultures. Our study is the first to demonstrate that real-time genotyping of Mycobacterium tuberculosis can be achieved, fully transforming the way in which molecular epidemiology techniques can be integrated into control programs.

Direct analysis of dried blood spots coupled with mass spectrometry: concepts and biomedical applications.

Because of the emergence of dried blood spots (DBS) as an attractive alternative to conventional venous plasma sampling in many pharmaceutical companies and clinical laboratories, different analytical approaches have been developed to enable automated handling of DBS samples without any pretreatment. Associated with selective and sensitive MS-MS detection, these procedures give good results in the rapid identification and quantification of drugs (generally less than 3 min total run time), which is desirable because of the high throughput requirements of analytical laboratories. The objective of this review is to describe the analytical concepts of current direct DBS techniques and to present their advantages and disadvantages, with particular focus on automation capacity and commercial availability. Finally, an overview of the different biomedical applications in which these concepts could be of major interest will be presented.

Genetic diversity of NS3 protease from Brazilian HCV isolates and possible implications for therapy with direct-acting antiviral drugs

The hepatitis C virus (HCV) NS3 protease has been one of the molecular targets of new therapeutic approaches. Its genomic sequence variability in Brazilian HCV isolates is poorly documented. To obtain more information on the magnitude of its genetic diversity, 114 Brazilian HCV samples were sequenced and analysed together with global reference sequences. Genetic distance (d) analyses revealed that subtype 1b had a higher degree of heterogeneity (d = 0.098) than subtypes 1a (d = 0.060) and 3a (d = 0.062). Brazilian isolates of subtype 1b were distributed in the phylogenetic tree among sequences from other countries, whereas most subtype 1a and 3a sequences clustered into a single branch. Additional characterisation of subtype 1a in clades 1 and 2 revealed that all but two Brazilian subtype 1a sequences formed a distinct and strongly supported (approximate likelihood-ratio test = 93) group of sequences inside clade 1. Moreover, this subcluster inside clade 1 presented an unusual phenotypic characteristic in relation to the presence of resistance mutations for macrocyclic inhibitors. In particular, the mutation Q80K was found in the majority of clade 1 sequences, but not in the Brazilian isolates. These data demonstrate that Brazilian HCV subtypes display a distinct pattern of genetic diversity and reinforce the importance of sequence information in future therapeutic approaches.

The hypogene iron oxide copper-gold mineralization in the Mantoverde District, Northern Chile

The Mantoverde iron oxide copper-gold (IOCC) district, northern Chile, is known for its Cu production from supergene ores. Recently, exploration outlined an additional hypogene ore resource of 440 Mt with 0.56 percent Cu, and 0.12 g/t An. The hypogene sulfide mineralization occurs mainly as chalcopyrite and pyrite, typically in specularite or magnetite-cemented breccias and associated stockworks. The host rocks underwent variably intense K feldspar alteration, chloritization, sericitization, silicification, and/or carbonatization. A district scale Na(-Ca) alteration is absent. The IOCC mineralization in the district shows a strong tectonic control by northwest- to north-northwest-trending brittle structures. Large Cu sulfide-rich veins or Cu sulfide-cemented breccias are absent. Therefore, head grades of 4 percent Cu are an exception. There is a positive correlation between Cu and An grades. Gold is probably contained mostly in chalcopyrite and pyrite. Elevated concentrations of light rare-earth elements (LREE) occur locally but are attributed to redistribution of LREE within the deposits rather than to derivation from external sources. The Cu-Au ores in the Mantoverde district are low in and have relatively low contents in heavy metals that are potentially hazardous to the environment, such as As (avg 14 ppm), Hg (<5 ppm), or Cd (<0.2 ppm). The sulfur isotope ratios of chalcopyrite from the IOCC deposits lie between -5.6 and 8.9 per mil delta(34)S(VCDT). They show systematic variations within the district, which are interpreted to reflect relative distance to inferred fluid conduits and the level of deposition within the hydrothermal system. Most initial (87)Sr/(86)Sr values of altered volcanic rocks and hydrothermal calcite from the Mantoverde district are between 0.7031 and 0.7060 and are similar to those of the igneous rocks of the region. Lead isotope ratios of chalcopyrite are consistent with Pb (and by inference Cu) derived from Early Cretaceous magmatism. The sulfur, strontium, and lead isotope data of chalcopyrite, calcite gangue, or altered host rocks, respectively, are compatible with a genetic model that involves cooling of metal and sulfur-bearing magmatic-hydrothermal fluids that mix with meteoric waters or seawater at relatively shallow crustal levels. An additional exotic sulfur input is likely, though not required, for the copper mineralization. Apart from the IOCC. deposits, there are a number of smaller magnetite(-apatite) bodies in the district. These are geologically similar to the Cu-Au-bearing magnetite bodies, but are related to splays of the north-south-trending Atacama fault zone and differ in alteration and texture.

Quantification of C4d deposition and hepatitis C virus RNA in tissue in cases of graft rejection and hepatitis C recurrence after liver transplantation

Histology is the gold standard for diagnosing acute rejection and hepatitis C recurrence after liver transplantation. However, differential diagnosis between the two can be difficult. We evaluated the role of C4d staining and quantification of hepatitis C virus (HCV) RNA levels in liver tissue. This was a retrospective study of 98 liver biopsy samples divided into four groups by histological diagnosis: acute rejection in patients undergoing liver transplant for hepatitis C (RejHCV+), HCV recurrence in patients undergoing liver transplant for hepatitis C (HCVTx+), acute rejection in patients undergoing liver transplant for reasons other than hepatitis C and chronic hepatitis C not transplanted (HCVTx-). All samples were submitted for immunohistochemical staining for C4d and HCV RNA quantification. Immunoexpression of C4d was observed in the portal vessels and was highest in the HCVTx- group. There was no difference in C4d expression between the RejHCV+ and HCVTx+ groups. However, tissue HCV RNA levels were higher in the HCVTx+ group samples than in the RejHCV+ group samples. Additionally, there was a significant correlation between tissue and serum levels of HCV RNA. The quantification of HCV RNA in liver tissue might prove to be an efficient diagnostic test for the recurrence of HCV infection.

The efficiency of direct payments versus tax reductions under uncertainty

This paper analyzes the optimal behavior of farmers in the presence of direct payments and uncertainty. In an empirical analysis for Switzerland, it confirms previously obtained theoretical results and determines the magnitude of the theoretical predicted effects. The results show that direct payments increase agricultural production between 3.7% to 4.8%. Alternatively to direct payments, the production effect of tax reductions is evaluated in order to determine its magnitude. The empirical analysis corroborates the theoretical results of the literature and demonstrates that tax reductions are also distorting, but to a substantially lesser degree if losses are not offset. However, tax reductions, independently whether losses are offset or not, lead to higher government spending than pure direct payments

Evaluation of the speed-oligo direct Mycobacterium tuberculosis assay for molecular detection of mycobacteria in clinical respiratory specimens.

We present the first evaluation of a novel molecular assay, the Speed-oligo Direct Mycobacterium tuberculosis (SO-DMT) assay, which is based on PCR combined with a dipstick for the detection of mycobacteria and the specific identification of M. tuberculosis complex (MTC) in respiratory specimens. A blind evaluation was carried out in two stages: first, under experimental conditions on convenience samples comprising 20 negative specimens, 44 smear- and culture-positive respiratory specimens, and 11 sputa inoculated with various mycobacterium-related organisms; and second, in the routine workflow of 566 fresh respiratory specimens (4.9% acid-fast bacillus [AFB] smear positives, 7.6% MTC positives, and 1.8% nontuberculous mycobacteria [NTM] culture positives) from two Mycobacterium laboratories. SO-DMT assay showed no reactivity in any of the mycobacterium-free specimens or in those with mycobacterium-related organisms. Compared to culture, the sensitivity in the selected smear-positive specimens was 0.91 (0.92 for MTC and 0.90 for NTM), and there was no molecular detection of NTM in a tuberculosis case or vice versa. With respect to culture and clinical data, the sensitivity, specificity, and positive and negative predictive values for the SO-DMT system in routine specimens were 0.76 (0.93 in smear positives [1.0 for MTC and 0.5 for NTM] and 0.56 in smear negatives [0.68 for MTC and 0.16 for NTM]), 0.99, 0.85 (1.00 in smear positives and 0.68 in smear negatives), and 0.97, respectively. Molecular misidentification of NTM cases occurred when testing 2 gastric aspirates from two children with clinically but not microbiologically confirmed lung tuberculosis. The SO-DMT assay appears to be a fast and easy alternative for detecting mycobacteria and differentiating MTC from NTM in smear-positive respiratory specimens.

Inter-informant agreement and prevalence estimates for mood syndromes: Direct interview vs. family history method.

BACKGROUND: The use of the family history method is recommended in family studies as a type of proxy interview of non-participating relatives. However, using different sources of information can result in bias as direct interviews may provide a higher likelihood of assigning diagnoses than family history reports. The aims of the present study were to: 1) compare diagnoses for threshold and subthreshold mood syndromes from interviews to those relying on information from relatives; 2) test the appropriateness of lowering the diagnostic threshold and combining multiple reports from the family history method to obtain comparable prevalence estimates to the interviews; 3) identify factors that influence the likelihood of agreement and reporting of disorders by informants. METHODS: Within a family study, 1621 informant-index subject pairs were identified. DSM-5 diagnoses from direct interviews of index subjects were compared to those derived from family history information provided by their first-degree relatives. RESULTS: 1) Inter-informant agreement was acceptable for Mania, but low for all other mood syndromes. 2) Except for Mania and subthreshold depression, the family history method provided significantly lower prevalence estimates. The gap improved for all other syndromes after lowering the threshold of the family history method. 3) Individuals who had a history of depression themselves were more likely to report depression in their relatives. LIMITATIONS: Low proportion of affected individuals for manic syndromes and lack of independence of data. CONCLUSIONS: The higher likelihood of reporting disorders by affected informants entails the risk of overestimation of the size of familial aggregation of depression.