How to assess the external validity of therapeutic trials: a conceptual approach

Background External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. Methods Methodological review to address the concept of external validity. Results External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. Conclusion Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.

Effectiveness of integrated care including therapeutic assertive community treatment in severe schizophrenia spectrum and bipolar I disorders: the 24-month follow-up ACCESS II study

OBJECTIVE The ACCESS treatment model offers assertive community treatment embedded in an integrated care program to patients with psychoses. Compared to standard care and within a controlled study, it proved to be more effective in terms of service disengagement and illness outcomes in patients with schizophrenia spectrum disorders over 12 months. ACCESS was implemented into clinical routine and its effectiveness assessed over 24 months in severe schizophrenia spectrum disorders and bipolar I disorder with psychotic features (DSM-IV) in a cohort study. METHOD All 115 patients treated in ACCESS (from May 2007 to October 2009) were included in the ACCESS II study. The primary outcome was rate of service disengagement. Secondary outcomes were change of psychopathology, severity of illness, psychosocial functioning, quality of life, satisfaction with care, medication nonadherence, length of hospital stay, and rates of involuntary hospitalization. RESULTS Only 4 patients (3.4%) disengaged with the service. Another 11 (9.6%) left because they moved outside the catchment area. Patients received a mean of 1.6 outpatient contacts per week. Involuntary admissions decreased from 34.8% in the 2 previous years to 7.8% during ACCESS (P < .001). Mixed models repeated-measures analyses revealed significant improvements among all patients in psychopathology (effect size d = 0.64, P < .001), illness severity (d = 0.84, P = .03), functioning level (d = 0.65, P < .001), quality of life (d = 0.50, P < .001), and client satisfaction (d = 0.11, P < .001). At 24 months, 78.3% were fully adherent to medication, compared to 25.2% at baseline (P = .002). CONCLUSIONS ACCESS was successfully implemented in clinical routine and maintained excellent rates of service engagement and other outcomes in patients with schizophrenia spectrum disorders or bipolar I disorder with psychotic features over 24 months. TRIAL REGISTRATION ClinicalTrials.gov identifier: NCT01888627.

Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study

Objective To examine the presence and extent of small study effects in clinical osteoarthritis research. Design Meta-epidemiological study. Data sources 13 meta-analyses including 153 randomised trials (41 605 patients) that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patients’ reported pain as an outcome. Methods We compared estimated benefits of treatment between large trials (at least 100 patients per arm) and small trials, explored funnel plots supplemented with lines of predicted effects and contours of significance, and used three approaches to estimate treatment effects: meta-analyses including all trials irrespective of sample size, meta-analyses restricted to large trials, and treatment effects predicted for large trials. Results On average, treatment effects were more beneficial in small than in large trials (difference in effect sizes −0.21, 95% confidence interval −0.34 to −0.08, P=0.001). Depending on criteria used, six to eight funnel plots indicated small study effects. In six of 13 meta-analyses, the overall pooled estimate suggested a clinically relevant, significant benefit of treatment, whereas analyses restricted to large trials and predicted effects in large trials yielded smaller non-significant estimates. Conclusions Small study effects can often distort results of meta-analyses. The influence of small trials on estimated treatment effects should be routinely assessed.

The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study

OBJECTIVE: To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials. DESIGN: Meta-epidemiological study based on a collection of meta-analyses of randomised trials. DATA SOURCES: 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome. METHODS: Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed. RESULTS: 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference -0.13, 95% confidence interval -0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (tau(2)=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity. CONCLUSION: Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

OBJECTIVE: To examine whether the association of inadequate or unclear allocation concealment and lack of blinding with biased estimates of intervention effects varies with the nature of the intervention or outcome. DESIGN: Combined analysis of data from three meta-epidemiological studies based on collections of meta-analyses. DATA SOURCES: 146 meta-analyses including 1346 trials examining a wide range of interventions and outcomes. MAIN OUTCOME MEASURES: Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates. RESULTS: In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses. CONCLUSIONS: The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.

Molecular Characterization of Three Canine Models of Human Rare Bone Diseases: Caffey, van den Ende-Gupta, and Raine Syndromes.

One to two percent of all children are born with a developmental disorder requiring pediatric hospital admissions. For many such syndromes, the molecular pathogenesis remains poorly characterized. Parallel developmental disorders in other species could provide complementary models for human rare diseases by uncovering new candidate genes, improving the understanding of the molecular mechanisms and opening possibilities for therapeutic trials. We performed various experiments, e.g. combined genome-wide association and next generation sequencing, to investigate the clinico-pathological features and genetic causes of three developmental syndromes in dogs, including craniomandibular osteopathy (CMO), a previously undescribed skeletal syndrome, and dental hypomineralization, for which we identified pathogenic variants in the canine SLC37A2 (truncating splicing enhancer variant), SCARF2 (truncating 2-bp deletion) and FAM20C (missense variant) genes, respectively. CMO is a clinical equivalent to an infantile cortical hyperostosis (Caffey disease), for which SLC37A2 is a new candidate gene. SLC37A2 is a poorly characterized member of a glucose-phosphate transporter family without previous disease associations. It is expressed in many tissues, including cells of the macrophage lineage, e.g. osteoclasts, and suggests a disease mechanism, in which an impaired glucose homeostasis in osteoclasts compromises their function in the developing bone, leading to hyperostosis. Mutations in SCARF2 and FAM20C have been associated with the human van den Ende-Gupta and Raine syndromes that include numerous features similar to the affected dogs. Given the growing interest in the molecular characterization and treatment of human rare diseases, our study presents three novel physiologically relevant models for further research and therapy approaches, while providing the molecular identity for the canine conditions.

Homeopathic Preparations to Control the Rosy Apple Aphid (Dysaphis plantaginea Pass.)

A laboratory model system with the rosy apple aphid (Dysaphis plantaginea Pass.) on apple seedlings was developed to study the effects of homeopathic preparations on this apple pest. The assessment included the substance Lycopodium clavatum and a nosode of the rosy apple aphid. Each preparation was applied on the substrate surface as aqueous solution of granules (6c, 15c, or 30c). Controls were aqueous solutions of placebo granules or pure water. In eight independent, randomized, and blinded experiments under standardized conditions in growth chambers, the development of aphids on treated and untreated apple seedlings was observed over 17 days, each. Six experiments were determined to assess the effects of a strict therapeutic treatment; two experiments were designed to determine the effects of a combined preventative and therapeutic treatment. After application of the preparations, the number of juvenile offspring and the damage on apple seedlings were assessed after 7 and 17 days, respectively. In addition, after 17 days, the seedling weight was measured. In the final evaluation of the six strictly therapeutic trials after 17 days, the number of juvenile offspring was reduced after application of L. clavatum 15c (-17%, p = 0.002) and nosode 6c (-14%, p = 0.02) compared to the pure water control. No significant effects were observed for leaf damage or fresh weight for any application. In the two experiments with combined preventative and therapeutic treatment, no significant effects were observed in any measured parameter. Homeopathic remedies may be effective in plant-pest systems. The magnitude of observed effects seems to be larger than in models with healthy plants, which renders plant-pest systems promising candidates for homeopathic basic research. For successful application in agriculture, however, the effect is not yet sufficient. This calls for further optimization concerning homeopathic remedy selection, potency level, dosage, and application routes.

Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study

Objectives To examine the extent of multiplicity of data in trial reports and to assess the impact of multiplicity on meta-analysis results. Design Empirical study on a cohort of Cochrane systematic reviews. Data sources All Cochrane systematic reviews published from issue 3 in 2006 to issue 2 in 2007 that presented a result as a standardised mean difference (SMD). We retrieved trial reports contributing to the first SMD result in each review, and downloaded review protocols. We used these SMDs to identify a specific outcome for each meta-analysis from its protocol. Review methods Reviews were eligible if SMD results were based on two to ten randomised trials and if protocols described the outcome. We excluded reviews if they only presented results of subgroup analyses. Based on review protocols and index outcomes, two observers independently extracted the data necessary to calculate SMDs from the original trial reports for any intervention group, time point, or outcome measure compatible with the protocol. From the extracted data, we used Monte Carlo simulations to calculate all possible SMDs for every meta-analysis. Results We identified 19 eligible meta-analyses (including 83 trials). Published review protocols often lacked information about which data to choose. Twenty-four (29%) trials reported data for multiple intervention groups, 30 (36%) reported data for multiple time points, and 29 (35%) reported the index outcome measured on multiple scales. In 18 meta-analyses, we found multiplicity of data in at least one trial report; the median difference between the smallest and largest SMD results within a meta-analysis was 0.40 standard deviation units (range 0.04 to 0.91). Conclusions Multiplicity of data can affect the findings of systematic reviews and meta-analyses. To reduce the risk of bias, reviews and meta-analyses should comply with prespecified protocols that clearly identify time points, intervention groups, and scales of interest.

Visual disturbance following sclerotherapy for varicose veins, reticular veins and telangiectasias: a systematic literature review

The objective of the study was to review the literature reporting visual disturbance (VD) following sclerotherapy for varicose veins. Underlying mechanisms will be discussed. A literature search of the databases Medline and Google Scholar was performed. Original articles including randomized trials, case series and case reports reporting VD in humans following sclerotherapy for varicose veins were included. Additional references were also obtained if they had been referenced in related publications. The search yielded 4948 results of which 25 reports were found to meet the inclusion criteria. In larger series with at least 500 included patients the prevalence of VD following sclerotherapy ranges from 0.09% to 2%. In most reports foam sclerotherapy was associated with VD (19); exclusive use of liquid sclerosant was reported in two cases, some reports included foam and liquid sclerosant (4). There were no persistent visual disorders reported. VD occurred with polidocanol and sodium tetradecyl sulphate in different concentrations (0.25-3%). Various forms of foam preparation including various ways of foam production and the liquid - air ratio (1 or 2 parts of liquid mixed with 3, 4 or 5 parts of air) were reported in association with the occurrence of VD. VDs following sclerotherapy for varicose veins are rare and all reported events were transient. Bubble embolism or any kind of embolism seems unlikely to be the only underlying mechanism. A systemic inflammatory response following sclerotherapy has been suggested. Further research to clarify the mechanism of action of sclerosants is required.

Clinical presentation and diagnostic delay in bullous pemphigoid: a prospective nationwide cohort

Prospective systematic analyses of the clinical presentation of bullous pemphigoid (BP) are lacking. Little is known about the time required for its diagnosis. Knowledge of the disease spectrum is important for diagnosis, management and inclusion of patients in therapeutic trials.

Visual disturbance following sclerotherapy for varicose veins, reticular veins and telangiectasias: a systematic literature review.

The objective of the study was to review the literature reporting visual disturbance (VD)following sclerotherapy for varicose veins. Underlying mechanisms will be discussed. A literature search of the databases Medline and Google Scholar was performed. Original articles including randomized trials, case series and case reports reporting VD in humans following sclerotherapy for varicose veins were included. Additional references were also obtained if they had been referenced in related publications. The search yielded 4948 results of which 25 reports were found to meet the inclusion criteria. In larger series with at least 500 included patients the prevalence of VD following sclerotherapy ranges from 0.09% to 2%. In most reports foam sclerotherapy was associated with VD (19); exclusive use of liquid sclerosant was reported in two cases, some reports included foam and liquid sclerosant (4). There were no persistent visual disorders reported. VD occurred with polidocanol and sodium tetradecyl sulphate in different concentrations (0.25–3%). Various forms of foam preparation including various ways of foam production and the liquid –air ratio (1 or 2 parts of liquid mixed with 3, 4 or 5 parts of air) were reported in association with the occurrence of VD. VDs following sclerotherapy for varicose veins are rare and all reported events were transient. Bubble embolism or any kind of embolism seems unlikely to be the only underlying mechanism. A systemic inflammatory response following sclerotherapy has been suggested. Further research to clarify the mechanism of action of sclerosants is required.

Pemphigus. S2 Guideline for diagnosis and treatment--guided by the European Dermatology Forum (EDF) in cooperation with the European Academy of Dermatology and Venereology (EADV)

BACKGROUND Pemphigus encompasses a group of life-threatening autoimmune bullous diseases characterized by blisters and erosions of the mucous membranes and skin. Before the era of immunosuppressive treatment, the prognosis of pemphigus was almost fatal. Due to its rarity, only few prospective controlled therapeutic trials are available. OBJECTIVES For this reason, a group of European dermatologists with a long-standing interest and expertise in basic and clinical pemphigus research has sought to define diagnostic and therapeutic guidelines for the management of patients with pemphigus. RESULTS This group identified the statements of major agreement or disagreement regarding the diagnostic and therapeutic management of pemphigus. The revised final version of the pemphigus guideline was finally passed on to the European Dermatology Forum (EDF) for a final consensus with the European Academy of Dermatology and Venereology (EADV) and the European Union of Medical Specialists (UEMS).

Current state of evidence on 'off-label' therapeutic options for systemic lupus erythematosus, including biological immunosuppressive agents, in Germany, Austria and Switzerland--a consensus report

Systemic lupus erythematosus (SLE) can be a severe and potentially life-threatening disease that often represents a therapeutic challenge because of its heterogeneous organ manifestations. Only glucocorticoids, chloroquine and hydroxychloroquine, azathioprine, cyclophosphamide and very recently belimumab have been approved for SLE therapy in Germany, Austria and Switzerland. Dependence on glucocorticoids and resistance to the approved therapeutic agents, as well as substantial toxicity, are frequent. Therefore, treatment considerations will include 'off-label' use of medication approved for other indications. In this consensus approach, an effort has been undertaken to delineate the limits of the current evidence on therapeutic options for SLE organ disease, and to agree on common practice. This has been based on the best available evidence obtained by a rigorous literature review and the authors' own experience with available drugs derived under very similar health care conditions. Preparation of this consensus document included an initial meeting to agree upon the core agenda, a systematic literature review with subsequent formulation of a consensus and determination of the evidence level followed by collecting the level of agreement from the panel members. In addition to overarching principles, the panel have focused on the treatment of major SLE organ manifestations (lupus nephritis, arthritis, lung disease, neuropsychiatric and haematological manifestations, antiphospholipid syndrome and serositis). This consensus report is intended to support clinicians involved in the care of patients with difficult courses of SLE not responding to standard therapies by providing up-to-date information on the best available evidence.

Biological, diagnostic and therapeutic relevance of the MET receptor signaling in head and neck cancer.

Head and neck cancer constitutes the 6th most common malignancy worldwide and affects the crucial anatomical structures and physiological functions of the upper aerodigestive tract. Classical therapeutic strategies such as surgery and radiotherapy carry substantial toxicity and functional impairment. Moreover, the loco-regional control rates as well as overall survival still need to be improved in subgroups of patients. The scatter-factor/hepatocyte growth factor receptor tyrosine kinase MET is an established effector in the promotion, maintenance and progression of malignant transformation in a wide range of human malignancies, and has been gaining considerable interest in head and neck cancer over the last 15 years. Aberrant MET activation due to overexpression, mutations, tumor-stroma paracrine loops, and cooperative/redundant signaling has been shown to play prominent roles in epithelial-to-mesenchymal transition, angiogenesis, and responses to anti-cancer therapeutic modalities. Accumulating preclinical and translational evidence highly supports the increasing interest of MET as a biomarker for lymph node and distant metastases, as well as a potential marker of stratification for responses to ionizing radiation. The relevance of MET as a therapeutic molecular target in head and neck cancer described in preclinical studies remains largely under-evaluated in clinical trials, and therefore inconclusive. Also in the context of anti-cancer targeted therapy, a large body of preclinical data suggests a central role for MET in treatment resistance towards multiple therapeutic modalities in malignancies of the head and neck region. These findings, as well as the potential use of combination therapies including MET inhibitors in these tumors, need to be further explored.