Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

Comparison of co-located independent ground-based middle atmospheric wind and temperature measurements with numerical weather prediction models

High-resolution, ground-based and independent observations including co-located wind radiometer, lidar stations, and infrasound instruments are used to evaluate the accuracy of general circulation models and data-constrained assimilation systems in the middle atmosphere at northern hemisphere midlatitudes. Systematic comparisons between observations, the European Centre for Medium-Range Weather Forecasts (ECMWF) operational analyses including the recent Integrated Forecast System cycles 38r1 and 38r2, the NASA’s Modern-Era Retrospective Analysis for Research and Applications (MERRA) reanalyses, and the free-running climate Max Planck Institute–Earth System Model–Low Resolution (MPI-ESM-LR) are carried out in both temporal and spectral dom ains. We find that ECMWF and MERRA are broadly consistent with lidar and wind radiometer measurements up to ~40 km. For both temperature and horizontal wind components, deviations increase with altitude as the assimilated observations become sparser. Between 40 and 60 km altitude, the standard deviation of the mean difference exceeds 5 K for the temperature and 20 m/s for the zonal wind. The largest deviations are observed in winter when the variability from large-scale planetary waves dominates. Between lidar data and MPI-ESM-LR, there is an overall agreement in spectral amplitude down to 15–20 days. At shorter time scales, the variability is lacking in the model by ~10 dB. Infrasound observations indicate a general good agreement with ECWMF wind and temperature products. As such, this study demonstrates the potential of the infrastructure of the Atmospheric Dynamics Research Infrastructure in Europe project that integrates various measurements and provides a quantitative understanding of stratosphere-troposphere dynamical coupling for numerical weather prediction applications.

On the initiation of boudinage and folding instabilities in layered elasto-visco-plastic solids - Insights from natural microstructures and numerical simulations

The present understanding of the initiation of boudinage and folding structures is based on viscosity contrasts and stress exponents, considering an intrinsically unstable state of the layer. The criterion of localization is believed to be prescribed by geometry-material interactions, which are often encountered in natural structures. An alternative localization phenomenon has been established for ductile materials, in which instability emerges for critical material parameters and loading rates from homogeneous conditions. In this thesis, conditions are sought under which this type of instability prevails and whether localization in geological materials necessarily requires a trigger by geometric imperfections. The relevance of critical deformation conditions, material parameters and the spatial configuration of instabilities are discussed in a geological context. In order to analyze boudinage geometries, a numerical eigenmode analysis is introduced. This method allows determining natural frequencies and wavelengths of a structure and inducing perturbations on these frequencies. In the subsequent coupled thermo-mechanical simulations, using a grain size evolution and end-member flow laws, localization emerges when material softening through grain size sensitive viscous creep sets in. Pinch-and-swell structures evolve along slip lines through a positive feedback between the matrix response and material bifurcations inside the layer, independent from the mesh-discretization length scale. Since boudinage and folding are considered to express the same general instability, both structures should arise independently of the sign of the loading conditions and for identical material parameters. To this end, the link between material to energy instabilities is approached by means of bifurcation analyses of the field equations and finite element simulations of the coupled system of equations. Boudinage and folding structures develop at the same critical energy threshold, where dissipative work by temperature-sensitive creep overcomes the diffusive capacity of the layer. This finding provides basis for a unified theory for strain localization in layered ductile materials. The numerical simulations are compared to natural pinch-and-swell microstructures, tracing the adaption of grain sizes, textures and creep mechanisms in calcite veins. The switch from dislocation to diffusion creep relates to strain-rate weakening, which is induced by dissipated heat from grain size reduction, and marks the onset of continuous necking. The time-dependent sequence uncovers multiple steady states at different time intervals. Microstructurally and mechanically stable conditions are finally expressed in the pinch-and-swell end members. The major outcome of this study is that boudinage and folding can be described as the same coupled energy-mechanical bifurcation, or as one critical energy attractor. This finding allows the derivation of critical deformation conditions and fundamental material parameters directly from localized structures in the field.