Non-response bias in randomized control experiments in criminology : putting the Queensland Community Engagement Trial (QCET) under a microscope

Objectives The goal of this article is to examine whether or not the results of the Queensland Community Engagement Trial (QCET)-a randomized controlled trial that tested the impact of procedural justice policing on citizen attitudes toward police-were affected by different types of nonresponse bias. Method We use two methods (Cochrane and Elffers methods) to explore nonresponse bias: First, we assess the impact of the low response rate by examining the effects of nonresponse group differences between the experimental and control conditions and pooled variance under different scenarios. Second, we assess the degree to which item response rates are influenced by the control and experimental conditions. Results Our analysis of the QCET data suggests that our substantive findings are not influenced by the low response rate in the trial. The results are robust even under extreme conditions, and statistical significance of the results would only be compromised in cases where the pooled variance was much larger for the nonresponse group and the difference between experimental and control conditions was greatly diminished. We also find that there were no biases in the item response rates across the experimental and control conditions. Conclusion RCTs that involve field survey responses-like QCET-are potentially compromised by low response rates and how item response rates might be influenced by the control or experimental conditions. Our results show that the QCET results were not sensitive to the overall low response rate across the experimental and control conditions and the item response rates were not significantly different across the experimental and control groups. Overall, our analysis suggests that the results of QCET are robust and any biases in the survey responses do not significantly influence the main experimental findings.

Estimating equations with nonignorably missing response data

Troxel, Lipsitz, and Brennan (1997, Biometrics 53, 857-869) considered parameter estimation from survey data with nonignorable nonresponse and proposed weighted estimating equations to remove the biases in the complete-case analysis that ignores missing observations. This paper suggests two alternative modifications for unbiased estimation of regression parameters when a binary outcome is potentially observed at successive time points. The weighting approach of Robins, Rotnitzky, and Zhao (1995, Journal of the American Statistical Association 90, 106-121) is also modified to obtain unbiased estimating functions. The suggested estimating functions are unbiased only when the missingness probability is correctly specified, and misspecification of the missingness model will result in biases in the estimates. Simulation studies are carried out to assess the performance of different methods when the covariate is binary or normal. For the simulation models used, the relative efficiency of the two new methods to the weighting methods is about 3.0 for the slope parameter and about 2.0 for the intercept parameter when the covariate is continuous and the missingness probability is correctly specified. All methods produce substantial biases in the estimates when the missingness model is misspecified or underspecified. Analysis of data from a medical survey illustrates the use and possible differences of these estimating functions.

Personality, health and lifestyle in a questionnaire family study: A comparison between highly cooperative and less cooperative families

The effect of nonresponse on health and lifestyle measures has received extensive study, showing at most relatively modest effects. Nonresponse bias with respect to personality has been less thoroughly investigated. The present study uses data from responding individuals as a proxy for the missing data of their nonresponding family members to examine the presence of nonresponse bias for personality traits and disorders as well as health and lifestyle traits. We looked at the Big Five personality traits, borderline personality disorder (BPD) features, attention-deficit/hyperactivity disorder, Anger, and several measures of health (Body Mass Index, migraine) and lifestyle (smoking, alcohol use). In general, outcomes tend to be slightly more favorable for individuals from highly cooperative families compared to individuals from less cooperative families. The only significant difference was found for BPD features (p = .001). However, the absolute difference in mean scores is very small, less than 1 point for a scale ranging from 0 to 72. In conclusion, survey data on personality, health and lifestyle are relatively unbiased with respect to nonresponse.