Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

A Near-Real-Time Automatic Orbit Determination System for COSMIC and Its Follow-On Satellite Mission: Analysis of Orbit and Clock Errors on Radio Occultation

The COSMIC-2 mission is a follow-on mission of the Constellation Observing System for Meteorology, Ionosphere, and Climate (COSMIC) with an upgraded payload for improved radio occultation (RO) applications. The objective of this paper is to develop a near-real-time (NRT) orbit determination system, called NRT National Chiao Tung University (NCTU) system, to support COSMIC-2 in atmospheric applications and verify the orbit product of COSMIC. The system is capable of automatic determinations of the NRT GPS clocks and LEO orbit and clock. To assess the NRT (NCTU) system, we use eight days of COSMIC data (March 24-31, 2011), which contain a total of 331 GPS observation sessions and 12 393 RO observable files. The parallel scheduling for independent GPS and LEO estimations and automatic time matching improves the computational efficiency by 64% compared to the sequential scheduling. Orbit difference analyses suggest a 10-cm accuracy for the COSMIC orbits from the NRT (NCTU) system, and it is consistent as the NRT University Corporation for Atmospheric Research (URCA) system. The mean velocity accuracy from the NRT orbits of COSMIC is 0.168 mm/s, corresponding to an error of about 0.051 μrad in the bending angle. The rms differences in the NRT COSMIC clock and in GPS clocks between the NRT (NCTU) and the postprocessing products are 3.742 and 1.427 ns. The GPS clocks determined from a partial ground GPS network [from NRT (NCTU)] and a full one [from NRT (UCAR)] result in mean rms frequency stabilities of 6.1E-12 and 2.7E-12, respectively, corresponding to range fluctuations of 5.5 and 2.4 cm and bending angle errors of 3.75 and 1.66 μrad .

Risk factors for patient-reported medical errors in eleven countries

OBJECTIVES The aim of this study was to identify common risk factors for patient-reported medical errors across countries. In country-level analyses, differences in risks associated with error between health care systems were investigated. The joint effects of risks on error-reporting probability were modelled for hypothetical patients with different health care utilization patterns. DESIGN Data from the Commonwealth Fund's 2010 lnternational Survey of the General Public's Views of their Health Care System's Performance in 11 Countries. SETTING Representative population samples of 11 countries were surveyed (total sample = 19,738 adults). Utilization of health care, coordination of care problems and reported errors were assessed. Regression analyses were conducted to identify risk factors for patients' reports of medical, medication and laboratory errors across countries and in country-specific models. RESULTS Error was reported by 11.2% of patients but with marked differences between countries (range: 5.4-17.0%). Poor coordination of care was reported by 27.3%. The risk of patient-reported error was determined mainly by health care utilization: Emergency care (OR = 1.7, P < 0.001), hospitalization (OR = 1.6, P < 0.001) and the number of providers involved (OR three doctors = 2.0, P < 0.001) are important predictors. Poor care coordination is the single most important risk factor for reporting error (OR = 3.9, P < 0.001). Country-specific models yielded common and country-specific predictors for self-reported error. For high utilizers of care, the probability that errors are reported rises up to P = 0.68. CONCLUSIONS Safety remains a global challenge affecting many patients throughout the world. Large variability exists in the frequency of patient-reported error across countries. To learn from others' errors is not only essential within countries but may also prove a promising strategy internationally.