Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study

OBJECTIVE: To study the inter-observer variation related to extraction of continuous and numerical rating scale data from trial reports for use in meta-analyses. DESIGN: Observer agreement study. DATA SOURCES: A random sample of 10 Cochrane reviews that presented a result as a standardised mean difference (SMD), the protocols for the reviews and the trial reports (n=45) were retrieved. DATA EXTRACTION: Five experienced methodologists and five PhD students independently extracted data from the trial reports for calculation of the first SMD result in each review. The observers did not have access to the reviews but to the protocols, where the relevant outcome was highlighted. The agreement was analysed at both trial and meta-analysis level, pairing the observers in all possible ways (45 pairs, yielding 2025 pairs of trials and 450 pairs of meta-analyses). Agreement was defined as SMDs that differed less than 0.1 in their point estimates or confidence intervals. RESULTS: The agreement was 53% at trial level and 31% at meta-analysis level. Including all pairs, the median disagreement was SMD=0.22 (interquartile range 0.07-0.61). The experts agreed somewhat more than the PhD students at trial level (61% v 46%), but not at meta-analysis level. Important reasons for disagreement were differences in selection of time points, scales, control groups, and type of calculations; whether to include a trial in the meta-analysis; and data extraction errors made by the observers. In 14 out of the 100 SMDs calculated at the meta-analysis level, individual observers reached different conclusions than the originally published review. CONCLUSIONS: Disagreements were common and often larger than the effect of commonly used treatments. Meta-analyses using SMDs are prone to observer variation and should be interpreted with caution. The reliability of meta-analyses might be improved by having more detailed review protocols, more than one observer, and statistical expertise.

Evaluation of the accuracy of multilevel analysis parameter estimates for intra-cluster correlation in correlated binary data

Many studies in biostatistics deal with binary data. Some of these studies involve correlated observations, which can complicate the analysis of the resulting data. Studies of this kind typically arise when a high degree of commonality exists between test subjects. If there exists a natural hierarchy in the data, multilevel analysis is an appropriate tool for the analysis. Two examples are the measurements on identical twins, or the study of symmetrical organs or appendages such as in the case of ophthalmic studies. Although this type of matching appears ideal for the purposes of comparison, analysis of the resulting data while ignoring the effect of intra-cluster correlation has been shown to produce biased results.^ This paper will explore the use of multilevel modeling of simulated binary data with predetermined levels of correlation. Data will be generated using the Beta-Binomial method with varying degrees of correlation between the lower level observations. The data will be analyzed using the multilevel software package MlwiN (Woodhouse, et al, 1995). Comparisons between the specified intra-cluster correlation of these data and the estimated correlations, using multilevel analysis, will be used to examine the accuracy of this technique in analyzing this type of data. ^

Optimization of Optical Follow-up Strategies Based on Covariance Analysis

An in-depth study, using simulations and covariance analysis, is performed to identify the optimal sequence of observations to obtain the most accurate orbit propagation. The accuracy of the results of an orbit determination/ improvement process depends on: tracklet length, number of observations, type of orbit, astrometric error, time interval between tracklets and observation geometry. The latter depends on the position of the object along its orbit and the location of the observing station. This covariance analysis aims to optimize the observation strategy taking into account the influence of the orbit shape, of the relative object-observer geometry and the interval between observations.

A covariance analysis to optimize the optical follow-up strategies

The Astronomical Institute of the University of Bern (AIUB) is conducting several search campaigns for space debris using optical sensors. The debris objects are discovered during systematic survey observations. In general, the result of a discovery consists in only a short observation arc, or tracklet, which is used to perform a first orbit determination in order to be able to observe t he object again in subsequent follow-up observations. The additional observations are used in the orbit improvement process to obtain accurate orbits to be included in a catalogue. In order to obtain the most accurate orbit within the time available it is necessary to optimize the follow-up observations strategy. In this paper an in‐depth study, using simulations and covariance analysis, is performed to identify the optimal sequence of follow-up observations to obtain the most accurate orbit propagation to be used for the space debris catalogue maintenance. The main factors that determine the accuracy of the results of an orbit determination/improvement process are: tracklet length, number of observations, type of orbit, astrometric error of the measurements, time interval between tracklets, and the relative position of the object along its orbit with respect to the observing station. The main aim of the covariance analysis is to optimize the follow-up strategy as a function of the object-observer geometry, the interval between follow-up observations and the shape of the orbit. This an alysis can be applied to every orbital regime but particular attention was dedicated to geostationary, Molniya, and geostationary transfer orbits. Finally the case with more than two follow-up observations and the influence of a second observing station are also analyzed.

Quantitative multi-slice computed tomography assessment of the mitral valvular complex for transcatheter mitral valve interventions part 1: systematic measurement methodology and inter-observer variability

AIMS Transcatheter mitral valve replacement (TMVR) is an emerging technology with the potential to treat patients with severe mitral regurgitation at excessive risk for surgical mitral valve surgery. Multimodal imaging of the mitral valvular complex and surrounding structures will be an important component for patient selection for TMVR. Our aim was to describe and evaluate a systematic multi-slice computed tomography (MSCT) image analysis methodology that provides measurements relevant for transcatheter mitral valve replacement. METHODS AND RESULTS A systematic step-by-step measurement methodology is described for structures of the mitral valvular complex including: the mitral valve annulus, left ventricle, left atrium, papillary muscles and left ventricular outflow tract. To evaluate reproducibility, two observers applied this methodology to a retrospective series of 49 cardiac MSCT scans in patients with heart failure and significant mitral regurgitation. For each of 25 geometrical metrics, we evaluated inter-observer difference and intra-class correlation. The inter-observer difference was below 10% and the intra-class correlation was above 0.81 for measurements of critical importance in the sizing of TMVR devices: the mitral valve annulus diameters, area, perimeter, the inter-trigone distance, and the aorto-mitral angle. CONCLUSIONS MSCT can provide measurements that are important for patient selection and sizing of TMVR devices. These measurements have excellent inter-observer reproducibility in patients with functional mitral regurgitation.

Changes of Root Length and Root-to-Crown Ratio after Apical Surgery: An Analysis by Using Cone-beam Computed Tomography

INTRODUCTION Apical surgery is an important treatment option for teeth with post-treatment periodontitis. Although apical surgery involves root-end resection, no morphometric data are yet available about root-end resection and its impact on the root-to-crown ratio (RCR). The present study assessed the length of apicectomy and calculated the loss of root length and changes of RCR after apical surgery. METHODS In a prospective clinical study, cone-beam computed tomography scans were taken preoperatively and postoperatively. From these images, the crown and root lengths of 61 roots (54 teeth in 47 patients) were measured before and after apical surgery. Data were collected relative to the cementoenamel junction (CEJ) as well as to the crestal bone level (CBL). One observer took all measurements twice (to calculate the intraobserver variability), and the means were used for further analysis. The following parameters were assessed for all treated teeth as well as for specific tooth groups: length of root-end resection and percentage change of root length, preoperative and postoperative RCRs, and percentage change of RCR after apical surgery. RESULTS The mean length of root-end resection was 3.58 ± 1.43 mm (relative to the CBL). This amounted to a loss of 33.2% of clinical and 26% of anatomic root length. There was an overall significant difference between the tooth groups (P < .05). There was also a statistically significant difference comparing mandibular and maxillary teeth (P < .05), but not for incisors/canines versus premolars/molars (P = .125). The mean preoperative and postoperative RCRs (relative to CEJ) were 1.83 and 1.35, respectively (P < .001). With regard to the CBL reference, the mean preoperative and postoperative RCRs were 1.08 and 0.71 (CBL), respectively (P < .001). The calculated changes of RCR after apical surgery were 24.8% relative to CEJ and 33.3% relative to CBL (P < .001). Across the different tooth groups, the mean RCR was not significantly different (P = .244 for CEJ and 0.114 for CBL). CONCLUSIONS This CBCT-based study demonstrated that the RCR is significantly changed after root-end resection in apical surgery irrespective of the clinical (CBL) or anatomic (CEJ) reference levels. The lowest, and thus clinically most critical, postoperative RCR was observed in maxillary incisors. Future clinical studies need to show the impact of resection length and RCR changes on the outcome of apical surgery.

Cognitive-Behavioural Analysis System of Psychotherapy (CBASP), a drug, or their combination: differential therapeutics for persistent depressive disorder: a study protocol of an individual participant data network meta-analysis.

INTRODUCTION Despite important advances in psychological and pharmacological treatments of persistent depressive disorders in the past decades, their responses remain typically slow and poor, and differential responses among different modalities of treatments or their combinations are not well understood. Cognitive-Behavioural Analysis System of Psychotherapy (CBASP) is the only psychotherapy that has been specifically designed for chronic depression and has been examined in an increasing number of trials against medications, alone or in combination. When several treatment alternatives are available for a certain condition, network meta-analysis (NMA) provides a powerful tool to examine their relative efficacy by combining all direct and indirect comparisons. Individual participant data (IPD) meta-analysis enables exploration of impacts of individual characteristics that lead to a differentiated approach matching treatments to specific subgroups of patients. METHODS AND ANALYSIS We will search for all randomised controlled trials that compared CBASP, pharmacotherapy or their combination, in the treatment of patients with persistent depressive disorder, in Cochrane CENTRAL, PUBMED, SCOPUS and PsycINFO, supplemented by personal contacts. Individual participant data will be sought from the principal investigators of all the identified trials. Our primary outcomes are depression severity as measured on a continuous observer-rated scale for depression, and dropouts for any reason as a proxy measure of overall treatment acceptability. We will conduct a one-step IPD-NMA to compare CBASP, medications and their combinations, and also carry out a meta-regression to identify their prognostic factors and effect moderators. The model will be fitted in OpenBUGS, using vague priors for all location parameters. For the heterogeneity we will use a half-normal prior on the SD. ETHICS AND DISSEMINATION This study requires no ethical approval. We will publish the findings in a peer-reviewed journal. The study results will contribute to more finely differentiated therapeutics for patients suffering from this chronically disabling disorder. TRIAL REGISTRATION NUMBER CRD42016035886.

STUDY OF THE EFFECTS OF TOTAL MODULATION TRANSFER FUNCTION CHANGES ON OBSERVER PERFORMANCE USING CLINICAL MAMMOGRAMS

The main goal of this study was to relate physical changes in image quality measured by Modulation Transfer Function (MTF) to diagnostic accuracy.^ One Hundred and Fifty Kodak Min-R screen/film combination conventional craniocaudal mammograms obtained with the Pfizer Microfocus Mammographic system were selected from the files of the Department of Radiology, at M.D. Anderson Hospital and Tumor Institute.^ The mammograms included 88 cases with a variety of benign diagnosis and 62 cases with a variety of malignant biopsy diagnosis. The average age of the patient population was 55 years old. 70 cases presented calcifications with 30 cases having calcifications smaller than 0.5mm. 46 cases presented irregular bordered masses larger than 1 cm. 30 cases presented smooth bordered masses with 20 larger than 1 cm.^ Four separated copies of the original images were made each having a different change in the MTF using a defocusing technique whereby copies of the original were obtained by light exposure through different thicknesses (spacing) of transparent film base.^ The mammograms were randomized, and evaluated by three experienced mammographers for the degree of visibility of various anatomical breast structures and pathological lesions (masses and calicifications), subjective image quality, and mammographic interpretation.^ 3,000 separate evaluations were anayzed by several statistical techniques including Receiver Operating Characteristic curve analysis, McNemar test for differences between proportions and the Landis et al. method of agreement weighted kappa for ordinal categorical data.^ Results from the statistical analysis show: (1) There were no statistical significant differences in the diagnostic accuracy of the observers when diagnosing from mammograms with the same MTF. (2) There were no statistically significant differences in diagnostic accuracy for each observer when diagnosing from mammograms with the different MTF's used in the study. (3) There statistical significant differences in detail visibility between the copies and the originals. Detail visibility was better in the originals. (4) Feature interpretations were not significantly different between the originals and the copies. (5) Perception of image quality did not affect image interpretation.^ Continuation and improvement of this research ca be accomplished by: using a case population more sensitive to MTF changes, i.e., asymptomatic women with minimum breast cancer, more observers (including less experienced radiologists and experienced technologists) must collaborate in the study, and using a minimum of 200 benign and 200 malignant cases.^

Dielectrophoresis-based analyte separation and analysis

Dielectrophoresis—the tendency of a material of high dielectric permittivity to migrate in an electrical field gradient to a region of maximum field strength—provides an ideal motive force for manipulating small volumes of biological analytes in microfluidic microsystems. The work described in this thesis was based on the hypothesis that dielectrophoresis could be exploited to provide high-resolution cell separations in microsystems as well as a means for the electrically-controllable manipulation of solid supports for molecular analysis. To this end, a dielectrophoretic/gravitational field-flow-fractionation (DEP/G-FFF) system was developed and the separation performance evaluated using various types and sizes of polystyrene microspheres as model particles. It was shown that separation of the polystyrene beads was based on the differences in their effective dielectrophoretic properties. The ability of an improved DEP/G-FFF system to separate genetically identical, but phenotypically dissimilar cell types was demonstrated using mixtures of 6m2 mutant rat kidney cells grown under transforming and non-transforming culture conditions. Additionally, a panel of engineered dielectric microspheres was designed with specific, predetermined dielectrophoretic properties such that their dielectrophoretic behaviors would be controllable and predictable. The fabrication method involved the use of gold-coated polystyrene microsphere cores coated with a self-assembled monolayer of alkanethiol and, optionally, a self-assembled monolayer of phospholipid to form a thin-insulating-shell-over-conductive-interior structure. The successful development of the DEP/G-FFF separation system and the dielectrically engineered microspheres provides proof-of-principle demonstrations of enabling dielectrophoresis-based microsystem technology that should provide powerful new methods for the manipulation, separation and identification of analytes in many diverse fields. ^

Application of stellar photometry to the analysis of microarray images

Improvements in the analysis of microarray images are critical for accurately quantifying gene expression levels. The acquisition of accurate spot intensities directly influences the results and interpretation of statistical analyses. This dissertation discusses the implementation of a novel approach to the analysis of cDNA microarray images. We use a stellar photometric model, the Moffat function, to quantify microarray spots from nylon microarray images. The inherent flexibility of the Moffat shape model makes it ideal for quantifying microarray spots. We apply our novel approach to a Wilms' tumor microarray study and compare our results with a fixed-circle segmentation approach for spot quantification. Our results suggest that different spot feature extraction methods can have an impact on the ability of statistical methods to identify differentially expressed genes. We also used the Moffat function to simulate a series of microarray images under various experimental conditions. These simulations were used to validate the performance of various statistical methods for identifying differentially expressed genes. Our simulation results indicate that tests taking into account the dependency between mean spot intensity and variance estimation, such as the smoothened t-test, can better identify differentially expressed genes, especially when the number of replicates and mean fold change are low. The analysis of the simulations also showed that overall, a rank sum test (Mann-Whitney) performed well at identifying differentially expressed genes. Previous work has suggested the strengths of nonparametric approaches for identifying differentially expressed genes. We also show that multivariate approaches, such as hierarchical and k-means cluster analysis along with principal components analysis, are only effective at classifying samples when replicate numbers and mean fold change are high. Finally, we show how our stellar shape model approach can be extended to the analysis of 2D-gel images by adapting the Moffat function to take into account the elliptical nature of spots in such images. Our results indicate that stellar shape models offer a previously unexplored approach for the quantification of 2D-gel spots. ^

State health care reform: An analysis of strategies

The United States health care system faces significant challenges, particularly with problems of the uninsured and with the rising costs of care. These problems lead many to study and discuss strategies for reforming the health care system. Four different plans for ideal health care reform, set forth by notable scholars or organizations, are explained herein. Then, states within the United States are examined in terms of their recent efforts at health care reform. Those states proposing significant changes to their health care systems are analyzed—namely, Maine, Massachusetts, and Vermont. The strategies used in these three states are compared to the strategies laid out by the experts in order to determine which strategies are the most popular in current health care reform efforts among the states studied here. These strategies are totaled to find which organization's plan for ideal reform seems to be the most popular. The strategies of managed competition are shown to be the most popular strategies among these three state health care reforms, while the strategies of the single-payer plan discussed herein were the least popular. All three states seem to utilize strategies that build upon their previous health care system, rather than implementing strategies that completely replace the previous system. ^

Democracy in the healthcare workplace: A critical perspective

A review of existing literature revealed at least two distinct theoretical perspectives or schools of thought which are troubled by problems of the lack of participation in the workplace: Jurgen Habermas' ideal of communicative rationality (1984; 1987); and the field of workplace democracy. Whereas Habermas' ideal of communicative rationality establishes communication as necessary to attain a democratic workplace, the ideal of workplace democracy focuses on a participatory ideal in which conditions of open participation must be fulfilled in order to attain a democratic workplace. This study compared the strengths and weaknesses of the conditions proposed by Habermas with the strengths and weaknesses of the conditions selected to represent the workplace democracy ideal. Two incidents were selected for analysis which occurred within a period of one year within one large healthcare organization. The author was present as a participant-observer to assess these incidents. Each of the conditions for the ideal of communicative rationality and for the workplace democracy ideal was systematically applied to both incidents selected for analysis. The results of the analysis suggested that application of Habermas' theory provided more insight into potential distortions in communication than did the conditions selected to represent workplace democracy. Although the conditions of both models were frequently complementary and even overlapping at times, application of each theory to the same incident produced distinctly different results. ^

THE USE OF ARM CIRCUMFERENCE FOR SIMPLIFIED SCREENING FOR MALNUTRITION BY MINIMALLY TRAINED HEALTH WORKERS

Because of its simplicity and low cost, arm circumference (AC) is being used increasingly in screening for protein energy malnutrition among pre-school children in many parts of the developing world, especially where minimally trained health workers are employed. The objectives of this study were as follows: (1) To determine the relationship of the AC measure with weight for age and weight for height in the detection of malnutrition among pre-school children in a Guatemalan Indian village. (2) To determine the performance of minimally trained promoters under field conditions in measuring AC, weight and height. (3) To describe the practical aspects of taking AC measures versus weight, age and height.^ The study was conducted in San Pablo La Laguna, one of four villages situated on the shores of Lake Atitlan, Guatemala, in which a program of simplified medical care was implemented by the Institute for Nutrition for Central America and Panama (INCAP). Weight, height, AC and age data were collected for 144 chronically malnourished children. The measurements obtained by the trained investigator under the controlled conditions of the health post were correlated against one another and AC was found to have a correlation with weight for age of 0.7127 and with weight for height of 0.7911, both well within the 0.65 to 0.80 range reported in the literature. False positive and false negative analysis showed that AC was more sensitive when compared with weight for height than with weight for age. This was fortunate since, especially in areas with widespread chronic malnutrition, weight for height detects those acute cases in immediate danger of complicating illness or death. Moreover, most of the cases identified as malnourished by AC, but not by weight for height (false positives), were either young or very stunted which made their selection by AC better than weight for height. The large number of cases detected by weight for age, but not by AC (false negative rate--40%) were, however, mostly beyond the critical age period and had normal weight for heights.^ The performance of AC, weight for height and weight for age under field conditions in the hands of minimally trained health workers was also analyzed by correlating these measurements against the same criterion measurements taken under ideally controlled conditions of the health post. AC had the highest correlation with itself indicating that it deteriorated the least in the move to the field. Moreover, there was a high correlation between AC in the field and criterion weight for height (0.7509); this correlation was almost as high as that for field weight for height versus the same measure in the health post (0.7588). The implication is that field errors are so great for the compounded weight for height variable that, in the field, AC is about as good a predictor of the ideal weight for height measure.^ Minimally trained health workers made more errors than the investigator as exemplified by their lower intra-observer correlation coefficients. They consistently measured larger than the investigator for all measures. Also there was a great deal of variability between these minimally trained workers indicating that careful training and followup is necessary for the success of the AC measure.^ AC has many practical advantages compared to the other anthropometric tools. It does not require age data, which are often unreliable in these settings, and does not require sophisticated subtraction and two dimensional table-handling skills that weight for age and weight for height require. The measure is also more easily applied with less disturbance to the child and the community. The AC tape is cheap and not easily damaged or jarred out of calibration while being transported in rugged settings, as is often the case with weight scales. Moreover, it can be kept in a health worker's pocket at all times for continual use in a widespread range of settings. ^

X-ray fluorescence (XRF) scannings and grain size analysis at sites in the Gulf of Cádiz

Contourites in the Gulf of Cádiz preserve a unique archive of Mediterranean Outflow Water (MOW) variability over the past 5.3 Ma. In our study we investigate the potential of geochemical data obtained by XRF scanning to decipher bottom current processes and paleoclimatic evolution at two different sites drilled through contourite deposits in the northern Gulf of Cadiz: Site U1387, which is bathed by the upper MOW core, and Site U1389, located more proximal to the Straits of Gibraltar. The lack of major downslope transport at both locations during the Pleistocene makes them ideal locations for the purpose of our study. The results indicate that the Zr/Al ratio, representing the relative enrichment of heavy minerals (zircon) over less dense alumosilicates under strong bottom current flow, is the most useful indicator for a semi-quantitative assessment of current strength. While most elements are biased by current-related processes, the bromine record, representing organic content, preserves the most pristine climate signal rather independent of grain size changes. Hence, Br can be used for chronostratigraphy and site-to-site correlation in addition to stable isotope stratigraphy. Based on these findings we reconstructed MOW variability for Marine Isotope Stages 1-5 using the Zr/Al ratio from Site U1387. The results reveal abrupt, millennial-scale variations of MOW strength during Greenland Stadials (GS) and Interstadials (GI) with strong MOW during GS and glacial Terminations and a complex behavior during Heinrich Stadials. Millennial-scale variability persisting during periods of poorly expressed GS/GI cyclicities implies a strong internal oscillation of the Mediterranean/North Atlantic climate system.

Q method factor analysis of stakeholder social perspectives of the effectiveness of Environmental Impact Assessment public participation in the Western Cape, South Africa

Public participation is an integral part of Environmental Impact Assessment (EIA), and as such, has been incorporated into regulatory norms. Assessment of the effectiveness of public participation has remained elusive however. This is partly due to the difficulty in identifying appropriate effectiveness criteria. This research uses Q methodology to discover and analyze stakeholder's social perspectives of the effectiveness of EIAs in the Western Cape, South Africa. It considers two case studies (Main Road and Saldanha Bay EIAs) for contextual participant perspectives of the effectiveness based on their experience. It further considers the more general opinion of provincial consent regulator staff at the Department of Environmental Affairs and the Department of Planning (DEA&DP). Two main themes of investigation are drawn from the South African National Environmental Management Act imperative for effectiveness: firstly, the participation procedure, and secondly, the stakeholder capabilities necessary for effective participation. Four theoretical frameworks drawn from planning, politics and EIA theory are adapted to public participation and used to triangulate the analysis and discussion of the revealed social perspectives. They consider citizen power in deliberation, Habermas' preconditions for the Ideal Speech Situation (ISS), a Foucauldian perspective of knowledge, power and politics, and a Capabilities Approach to public participation effectiveness. The empirical evidence from this research shows that the capacity and contextual constraints faced by participants demand the legislative imperatives for effective participation set out in the NEMA. The implementation of effective public participation has been shown to be a complex, dynamic and sometimes nebulous practice. The functional level of participant understanding of the process was found to be significantly wide-ranging with consequences of unequal and dissatisfied stakeholder engagements. Furthermore, the considerable variance of stakeholder capabilities in the South African social context, resulted in inequalities in deliberation. The social perspectives revealed significant differences in participant experience in terms of citizen power in deliberation. The ISS preconditions are highly contested in both the Saldanha EIA case study and the DEA&DP social perspectives. Only one Main Road EIA case study social perspective considered Foucault's notion of governmentality as a reality in EIA public participation. The freedom of control of ones environment, based on a Capabilities approach, is a highly contested notion. Although agreed with in principle, all of the social perspectives indicate that contextual and capacity realities constrain its realisation. This research has shown that Q method can be applied to EIA public participation in South Africa and, with the appropriate research or monitoring applications it could serve as a useful feedback tool to inform best practice public participation.