Appraising the quality of randomized controlled trials: inter-rater reliability for the OTseeker evidence database

Rationale and aims 'OTseeker' is an online database of randomized controlled trials (RCTs) and systematic reviews relevant to occupational therapy. RCTs are critically appraised and rated for quality using the 'PEDro' scale. We aimed to investigate the inter-rater reliability of the PEDro scale before and after revising rating guidelines. Methods In study 1, five raters scored 100 RCTs using the original PEDro scale guidelines. In study 2, two raters scored 40 different RCTs using revised guidelines. All RCTs were randomly selected from the OTseeker database. Reliability was calculated using Kappa and intraclass correlation coefficients [ICC (model 2,1)]. Results Inter-rater reliability was 'good to excellent' in the first study (Kappas >= 0.53; ICCs >= 0.71). After revising the rating guidelines, the reliability levels were equivalent or higher to those previously obtained (Kappas >= 0.53; ICCs >= 0.89), except for the item, 'groups similar at baseline', which still had moderate reliability (Kappa = 0.53). In study 2, two PEDro scale items, which had their definitions revised, 'less than 15% dropout' and 'point measures and variability', showed higher reliability. In both studies, the PEDro items with the lowest reliability were 'groups similar at baseline' (Kappas = 0.53), 'less than 15% dropout' (Kappas

A Comparison of Methodologies from Two Longitudinal Community-Based Randomized Controlled Trials of Similar Interventions in Palliative Care: What Worked and What Did Not

Background: Methodological challenges such as recruitment problems and participant burden make clinical trials in palliative care difficult. In 2001-2004, two community-based randomized controlled trials (RCTs) of case conferences in palliative care settings were independently conducted in Australia-the Queensland Case Conferences trial (QCC) and the Palliative Care Trial (PCT). Design: A structured comparative study of the QCC and PCT was conducted, organized by known practical and organizational barriers to clinical trials in palliative care. Results: Differences in funding dictated study designs and recruitment success; PCT had 6 times the budget of QCC. Sample size attainment. Only PCT achieved the sample size goal. QCC focused on reducing attrition through gatekeeping while PCT maximized participation through detailed recruitment strategies and planned for significant attrition. Testing sustainable interventions. QCC achieved a higher percentage of planned case conferences; the QCC strategy required minimal extra work for clinicians while PCT superimposed conferences on normal work schedules. Minimizing participant burden. Differing strategies of data collection were implemented to reduce participant burden. QCC had short survey instruments. PCT incorporated all data collection into normal clinical nursing encounters. Other. Both studies had acceptable withdrawal rates. Intention-to-treat analyses are planned. Both studies included substudies to validate new outcome measures. Conclusions: Health service interventions in palliative care can be studied using RCTs. Detailed comparative information of strategies, successes and challenges can inform the design of future trials. Key lessons include adequate funding, recruitment focus, sustainable interventions, and mechanisms to minimize participant burden.

Prosecution and radical discourse during the 1790s: The case of the Scottish sedition trials

This essay looks at the sedition trials in Scotland during the 1790s to examine how prosecution was exploited by radicals as a forum for political expression. As the government instituted a concerted campaign against radical activism, an increasing number of reformers faced trial on sedition and treason in this period. The courtroom emerged as an alternative venue for political discourse and this essay will explore some of the ways by which radicals challenged the dominant discursive and performative elements of trial proceedings. (c) 2005 Elsevier Ltd. All rights reserved.

An integrated analysis of five double-blind, randomized controlled trials evaluating the safety and efficacy of a hyaluronan product for intra-articular injection in osteoarthritis of the kneel

Objective: Five double-blind, randomized, saline-controlled trials (RCTs) were included in the United States marketing application for an intra-articular hyaluronan (IA-HA) product for the treatment of osteoarthritis (OA) of the knee. We report an integrated analysis of the primary Case Report Form (CRF) data from these trials. Method. Trials were similar in design, patient population and outcome measures - all included the Lequesne Algofunctional Index (LI), a validated composite index of pain and function, evaluating treatment over 3 months. Individual patient data were pooled; a repeated measures analysis of covariance was performed in the intent-to-treat (ITT) population. Analyses utilized both fixed and random effects models. Safety data from the five RCTs were summarized. Results: A total of 1155 patients with radiologically confirmed knee OA were enrolled: 619 received three or five IA-HA injections; 536 received. placebo saline injections. In the active and control groups, mean ages were 61.8 and 61.4 years; 62.4% and 58.8% were women; baseline total Lequesne scores 11.03 and 11.30, respectively. Integrated analysis of the pooled data set found a statistically significant reduction (P < 0.001) in total Lequesne score with hyaluronan (HA) (-2.68) vs placebo (-2.00); estimated difference -0.68 (95% CI: -0.56 to -0.79), effect size 0.20. Additional modeling approaches confirmed robustness of the analyses. Conclusions: This integrated analysis demonstrates that multiple design factors influence the results of RCTs assessing efficacy of intra-articular (IA) therapies, and that integrated analyses based on primary data differ from meta-analyses using transformed data. (C) 2006 OsteoArthritis Research Society International. Published by Elsevier Ltd. All rights reserved.

Small-scale randomized controlled trials need more powerful methods of mediational analysis than the Baron-Kenny method

Objective: To devise more-effective physical activity interventions, the mediating mechanisms yielding behavioral change need to be identified. The Baron-Kenny method is most commonly used. but has low statistical power and May not identify mechanisms of behavioral change in small-to-medium size Studies. More powerful statistical tests are available, Study Design and Setting: Inactive adults (N = 52) were randomized to either a print or a print-plus-telephone intervention. Walking and exercise-related social support Were assessed at baseline, after file intervention, and 4 weeks later. The Baron-Kenny and three alternative methods of mediational analysis (Freedman-Schatzkin; MacKinnon et al.: bootstrap method) were used to examine the effects of social support on initial behavior change and maintenance. Results: A significant mediational effect of social support on initial behavior change was indicated by the MacKinnon et al., bootstrap. and. marginally. Freedman-Schatzkin methods, but not by the Baron-Kenny method. No significant mediational effecl of social support on maintenance of walking was found. Conclusions: Methodologically rigorous intervention studies to identify mediators of change in physical activity are costly and labor intensive, and may not be feasible with large samples. The Use of statistically powerful tests of mediational effects in small-scale studies can inform the development of more effective interventions. (C) 2006 Elsevier Inc. All rights reserved.

The effect of St John's wort extracts on CYP3A: a systematic review of prospective clinical trials

Aim The aim of this systematic review was to assess the quality and outcomes of clinical trials investigating the effect of St John's wort extracts on the metabolism of drugs by CYP3A. Methods Prospective clinical trials assessing the effect of St John's wort (SJW) extracts on metabolism by CYP3A were identified through computer-based searches (from their inception to May 2005) of Medline, Cinahl, PsycINFO, AMED, Current Contents and Embase, hand-searches of bibliographies of relevant papers and consultation with manufacturers and researchers in the field. Two reviewers selected trials for inclusion, independently extracted data and recorded details on study design. Results Thirty-one studies met the eligibility criteria. More than two-thirds of the studies employed a before-and-after design, less than one-third of the studies used a crossover design, and only three studies were double-blind and placebo controlled. In 12 studies the SJW extract had been assayed, and 14 studies stated the specific SJW extract used. Results from 26 studies, including all of the 19 studies that used high-dose hyperforin extracts (> 10 mg day(-1)), had outcomes consistent with CYP3A induction. The three studies using low-dose hyperforin extracts (< 4 mg day(-1)) demonstrated no significant effect on CYP3A. Conclusion There is reasonable evidence to suggest that high-dose hyperforin SJW extracts induce CYP3A. More studies are required to determine whether decreased CYP3A induction occurs after low-dose hyperforin extracts. Future studies should adopt study designs with a control phase or control group, identify the specific SJW extract employed and provide quantitative analyses of key constituents.

Cautionary tales in the interpretation of systematic reviews of therapy trials

This is the second in a series of articles emphasizing the cautions in the interpretation of health-care studies. Systematic reviews are presented as comprehensive, unbiased summaries of evidence and are often referred to by clinicians, guideline developers and health policy-makers. Their strengths and limitations, and how their results can be subject to bias and misinterpretation, are discussed.