Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Factors Related to the Selection of Surgical Versus Percutaneous Revascularization in Diabetic Patients With Multivessel Coronary Artery Disease in the BARI 2D (Bypass Angioplasty Revascularization Investigation in Type 2 Diabetes) Trial

Objectives We evaluated demographic, clinical, and angiographic factors influencing the selection of coronary artery bypass graft (CABG) surgery versus percutaneous coronary intervention (PCI) in diabetic patients with multivessel coronary artery disease (CAD) in the BARI 2D (Bypass Angioplasty Revascularization Investigation in Type 2 Diabetes) trial. Background Factors guiding selection of mode of revascularization for patients with diabetes mellitus and multivessel CAD are not clearly defined. Methods In the BARI 2D trial, the selected revascularization strategy, CABG or PCI, was based on physician discretion, declared independent of randomization to either immediate or deferred revascularization if clinically warranted. We analyzed factors favoring selection of CABG versus PCI in 1,593 diabetic patients with multivessel CAD enrolled between 2001 and 2005. Results Selection of CABG over PCI was declared in 44% of patients and was driven by angiographic factors including triple vessel disease (odds ratio [OR]: 4.43), left anterior descending stenosis >= 70% (OR: 2.86), proximal left anterior descending stenosis >= 50% (OR: 1.78), total occlusion (OR: 2.35), and multiple class C lesions (OR: 2.06) (all p < 0.005). Nonangiographic predictors of CABG included age >= 65 years (OR: 1.43, p = 0.011) and non-U.S. region (OR: 2.89, p = 0.017). Absence of prior PCI (OR: 0.45, p < 0.001) and the availability of drug-eluting stents conferred a lower probability of choosing CABG (OR: 0.60, p = 0.003). Conclusions The majority of diabetic patients with multivessel disease were selected for PCI rather than CABG. Preference for CABG over PCI was largely based on angiographic features related to the extent, location, and nature of CAD, as well as geographic, demographic, and clinical factors. (Bypass Angioplasty Revascularization Investigation in Type 2 Diabetes [BARI 2D]; NCT00006305) (J Am Coll Cardiol Intv 2009;2:384-92) (C) 2009 by the American College of Cardiology Foundation

A practical framework for the generation and selection of strategic plans of action, The relevance of timeframe considerations and a systems thinking approach

The paper proposes a methodology especially focused on the generation of strategic plans of action, emphasizing the relevance of having a structured timeframe classification for the actions. The methodology explicitly recognizes the relevance of long-term goals as strategic drivers, which must insure that the complex system is capable to effectively respond to changes in the environment. In addition, the methodology employs engineering systems techniques in order to understand the inner working of the system and to build up alternative plans of action. Due to these different aspects, the proposed approach features higher flexibility compared to traditional methods. The validity and effectiveness of the methodology has been demonstrated by analyzing an airline company composed by 5 subsystems with the aim of defining a plan of action for the next 5 years, which can either: improve efficiency, redefine mission or increase revenues.

Selection of common bean lines with high grain yield and high grain calcium and iron concentrations

Genetic improvement of common bean nutritional quality has advantages in marketing and can contribute to society as a food source. The objective of this study was to evaluate the genetic variability for grain yield, calcium and iron concentrations in grains of inbred common bean lines obtained by different breeding methods. For this, 136 F7 inbred lines were obtained using the Pedigree method and 136 F7 inbred lines were obtained using the Single-Seed Descent (SSD) method. The lines showed genetic variability for grain yield, and concentrations of calcium and iron independently of the method of advancing segregating populations. The Pedigree method allows obtaining a greater number of lines with high grain yield. Selection using the SSD method allows the identification of a larger number of lines with high concentrations of calcium and iron in grains. Weak negative correlations were found between grain yield and calcium concentration (r = -0.0994) and grain yield and iron concentration (r = -0.3926). Several lines show genetic superiority for grain yield and concentrations of calcium and iron in grains and their selection can result in new common bean cultivars with high nutritional quality.

Selection of anthracnose resistant common beans using detached leaves in partially controlled environment

The objectives of this study were to evaluate the possibility of selecting anthracnose resistant common bean plants using detached primary leaves in partially controlled environment of a greenhouse and identify differences in the reaction of genotypes to anthracnose. The common bean cultivars Ouro Negro, OuroVermelho, ManteigãoFosco 11, Rudá, Rudá-R, VP8, BRSMG Madrepérola, Pérola, MeiaNoite and BRSMG Talismãwere characterizedfor resistance to the races 65, 81 and 453 of Colletotrichum lindemuthianum and the method of detached primary leaves was compared to the method with the traditional inoculation of plants at the phenological stage V2. The lines Rudá, Rudá-R and Pérola were inoculated with the races 65 and 453 of C. lindemuthianum, aiming to assess the rate of coincidence of anthracnose severity by both inoculation methods. In general, the two methods presented similar results for the reaction of the cultivars. The use of detached primary leaves of common bean plants in the partially controlled environment was feasible for selection of plants resistant to anthracnose and has the advantages of low-needed infrastructure and reduction of resources, space and time.

Epidemiological methods for research with drug misusers: review of methods for studying prevalence and morbidity

Epidemiological studies of drug misusers have until recently relied on two main forms of sampling: probability and convenience. The former has been used when the aim was simply to estimate the prevalence of the condition and the latter when in depth studies of the characteristics, profiles and behaviour of drug users were required, but each method has its limitations. Probability samples become impracticable when the prevalence of the condition is very low, less than 0.5% for example, or when the condition being studied is a clandestine activity such as illicit drug use. When stratified random samples are used, it may be difficult to obtain a truly representative sample, depending on the quality of the information used to develop the stratification strategy. The main limitation of studies using convenience samples is that the results cannot be generalised to the whole population of drug users due to selection bias and a lack of information concerning the sampling frame. New methods have been developed which aim to overcome some of these difficulties, for example, social network analysis, snowball sampling, capture-recapture techniques, privileged access interviewer method and contact tracing. All these methods have been applied to the study of drug misuse. The various methods are described and examples of their use given, drawn from both the Brazilian and international drug misuse literature.

A proteomic approach toward the selection of proteins with enhanced intrinsic conformational stability

Journal of Proteome Research (2006)5: 2720-2726

Useful clinical features for the selection of ideal patients with strial fibrillation for mapping and catheter ablation

OBJECTIVE: To identify useful clinical characteristics for selecting patients eligible for mapping and ablation of atrial fibrillation. METHODS: We studied 9 patients with atrial fibrillation, without structural heart disease, associated with: 1) antiarrhythmic drugs, 2) symptoms of low cardiac output, and 3) intention to treat. Seven patients had paroxysmal atrial fibrillation and 2 had recurrent atrial fibrillation. RESULTS: In the 6 patients who underwent mapping (all had paroxysmal atrial fibrillation), catheter ablation was successfully carried out in superior pulmonary veins in 5 patients (the first 3 in the left superior pulmonary vein and the last 2 in the right superior pulmonary vein). One patient experienced a recurrence of atrial fibrillation after 10 days. We observed that patients who had short episodes of atrial fibrillation on 24-hour Holter monitoring before the procedure were those in whom mapping the focus of tachycardia was possible. Tachycardia was successfully suppressed in 4 of 6 patients. The cause of failure was due to the impossibility of maintaining sinus rhythm long enough for efficient mapping. CONCLUSION: Patients experiencing short episodes of atrial fibrillation during 24-hour Holter monitoring were the most eligible for mapping and ablation, with a final success rate of 66%, versus the global success rate of 44%. Patients with persistent atrial fibrillation were not good candidates for focal ablation.

Protein recovery, separation and purification: selection of optimal techniques using an expert system

The paper discusses the utilization of new techniques ot select processes for protein recovery, separation and purification. It describesa rational approach that uses fundamental databases of proteins molecules to simplify the complex problem of choosing high resolution separation methods for multi component mixtures. It examines the role of modern computer techniques to help solving these questions.

Power and selection of contract terms: The case from the Brazilian orange juice sector

ABSTRACT We propose a model to explain how contract terms are selected in the presence of a form of economic power: contract power. The orange juice sector is used to illustrate an analysis that demonstrates the effects of contract power on the economic organization of the sector. We define contract power as the ability to exploit contractual gaps or failures of contractual provisions, which are strategically left incomplete. Empirical evidence from content analysis of antitrust documents supports the logic of contract power in the orange juice sector in three forms: avoiding changes to payment methods from weight to solid contents (quality); using information asymmetries to manipulate indexes that calculate the formula of orange prices; and deliberately harvesting oranges late in order to dehydrate the fruit, which consequently reduces weight and price. The paper contributes to understanding the selection of contract terms and the debate about how antitrust offices can deal with this issue.

Impact of Pre-Existing Immunity on the Selection of Rare Adenovirus Vector Candidates: Implications for HIV Vaccine Development

Background: Adenovirus serotype 5 (Ad5) phase IIb vaccine trial (STEP) was prematurely stopped due to a lack of efficacy and two-fold higher incidence of HIV infection among Ad5 seropositive vaccine recipients. We have recently demonstrated that Ad5 immune complexes (Ad5 ICs)-mediated activation of the dendritic cell (DC)-T cell axis was associated with the enhancement of HIV infection in vitro. Although the direct role of Ad5 neutralizing antibodies (NAbs) in the increase of HIV susceptibility during the STEP trial is still under debate, vector-specific NAbs remain a major hurdle for vector-based gene therapies or vaccine strategies. To surmount this obstacle, vectors based on ''rare'' Ad serotypes including Ad6, Ad26, Ad36 and Ad41 were engineered.Methods: The present study aimed to determine whether Ad ICmediated DC maturation could be circumvented using these Advector candidates.Results: We found that all Ad vectors tested forming ICs with plasma containing serotype-specific NAbs had the capacity to 1) mature human DCs as monitored by the up-regulation of costimulatory molecules and the release of pro-inflammatory cytokines (TNF-a), via the stabilization of Ad capsid at endosomal but not lysosomal pH rendering Ad DNA/TLR9 interactions possible and 2) potentiate Ad-specific CD4 and CD8 T cell responses.Conclusion: In conclusion, despite a conserved DC maturation potential, the low prevalence of serotype-specific NAbs renders rare Ad vectors attractive for vaccine strategies.

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background: The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114). Methods: 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item. Results: 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.Conclusions: Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.

Monoclonal antibodies to carcinoembryonic antigen: ionic strength as a factor in the selection of antibodies for immunoscintigraphy.

As part of an ongoing effort to improve the technique of immunoscintigraphy for the detection of human carcinomas with radiolabeled monoclonal antibodies (MABs) to carcinoembryonic antigen (CEA), we have developed a series of MABs to CEA and have studied the effects of low- and physiological molarity buffers on their CEA binding and affinity, as well as their cross-reactivity with granulocyte glycoprotein(s). These in vitro results in different buffer systems were then correlated with the use of these MABs to CEA in the detection of human colon carcinoma grafts in nude mice. Our results show that the binding of CEA by some MABs is influenced by ionic strength and that this may be an important factor in their successful use for the immunolocalization of carcinomas in vivo.

Myocardial flow reserve by Rb-82 cardiac PET improves the selection of patients eligible for invasive coronary angiography

Aim: We asked whether myocardial flow reserve (MFR) by Rb-82 cardiac PET improve the selection of patients eligible for invasive coronary angiography (ICA). Material and Methods: We enrolled 26 consecutive patients with suspected or known coronary artery disease who performed dynamic Rb-82 PET/CT and (ICA) within 60 days; 4 patients who underwent revascularization or had any cardiovascular events between PET and ICA were excluded. Myocardial blood flow at rest (rMBF), at stress with adenosine (sMBF) and myocardial flow reserve (MFR=sMBF/rMBF) were estimated using the 1-compartment Lortie model (FlowQuant) for each coronary arteries territories. Stenosis severity was assessed using computer-based automated edge detection (QCA). MFR was divided in 3 groups: G1:MFR<1.5, G2:1.5≤MFR<2 and G3:2≤MFR. Stenosis severity was graded as non-significant (<50% or FFR ≥0.8), intermediate (50%≤stenosis<70%) and severe (≥70%). Correlation between MFR and percentage of stenosis were assessed using a non-parametric Spearman test. Results: In G1 (44 vessels), 17 vessels (39%) had a severe stenosis, 11 (25%) an intermediate one, and 16 (36%) no significant stenosis. In G2 (13 vessels), 2 (15%) vessels presented a severe stenosis, 7 (54%) an intermediate one, and 4 (31%) no significant stenosis. In G3 (9 vessels), 0 vessel presented a severe stenosis, 1 (11%) an intermediate one, and 8 (89%) no significant stenosis. Of note, among 11 patients with 3-vessel low MFR<1.5 (G1), 9/11 (82%) had at least one severe stenosis and 2/11 (18%) had at least one intermediate stenosis. There was a significant inverse correlation between stenosis severity and MFR among all 66 territories analyzed (rho= -0.38, p=0.002). Conclusion: Patients with MFR>2 could avoid ICA. Low MFR (G1, G2) on a vessel-based analysis seems to be a poor predictor of severe stenosis severity. Patients with 3-vessel low MFR would benefit from ICA as they are likely to present a significant stenosis in at least one vessel.

Automatic selection of tidal volume, respiratory frequency and minute ventilation in intubated ICU patients as start up procedure for closed-loop controlled ventilation.

OBJECTIVE: Before a patient can be connected to a mechanical ventilator, the controls of the apparatus need to be set up appropriately. Today, this is done by the intensive care professional. With the advent of closed loop controlled mechanical ventilation, methods will be needed to select appropriate start up settings automatically. The objective of our study was to test such a computerized method which could eventually be used as a start-up procedure (first 5-10 minutes of ventilation) for closed-loop controlled ventilation. DESIGN: Prospective Study. SETTINGS: ICU's in two adult and one children's hospital. PATIENTS: 25 critically ill adult patients (age > or = 15 y) and 17 critically ill children selected at random were studied. INTERVENTIONS: To stimulate 'initial connection', the patients were disconnected from their ventilator and transiently connected to a modified Hamilton AMADEUS ventilator for maximally one minute. During that time they were ventilated with a fixed and standardized breath pattern (Test Breaths) based on pressure controlled synchronized intermittent mandatory ventilation (PCSIMV). MEASUREMENTS AND MAIN RESULTS: Measurements of airway flow, airway pressure and instantaneous CO2 concentration using a mainstream CO2 analyzer were made at the mouth during application of the Test-Breaths. Test-Breaths were analyzed in terms of tidal volume, expiratory time constant and series dead space. Using this data an initial ventilation pattern consisting of respiratory frequency and tidal volume was calculated. This ventilation pattern was compared to the one measured prior to the onset of the study using a two-tailed paired t-test. Additionally, it was compared to a conventional method for setting up ventilators. The computer-proposed ventilation pattern did not differ significantly from the actual pattern (p > 0.05), while the conventional method did. However the scatter was large and in 6 cases deviations in the minute ventilation of more than 50% were observed. CONCLUSIONS: The analysis of standardized Test Breaths allows automatic determination of an initial ventilation pattern for intubated ICU patients. While this pattern does not seem to be superior to the one chosen by the conventional method, it is derived fully automatically and without need for manual patient data entry such as weight or height. This makes the method potentially useful as a start up procedure for closed-loop controlled ventilation.