Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications

OBJECTIVE: To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications. DESIGN: Cohort of protocols of randomised controlled trial and subsequent full journal publications. SETTING: Six research ethics committees in Switzerland, Germany, and Canada. DATA SOURCES: 894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications. RESULTS: Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis. CONCLUSIONS: Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.

Priming psychic and conjuring abilities of a magic demonstration influences event interpretation and random number generation biases

Magical ideation and belief in the paranormal is considered to represent a trait-like character; people either believe in it or not. Yet, anecdotes indicate that exposure to an anomalous event can turn skeptics into believers. This transformation is likely to be accompanied by altered cognitive functioning such as impaired judgments of event likelihood. Here, we investigated whether the exposure to an anomalous event changes individuals' explicit traditional (religious) and non-traditional (e.g., paranormal) beliefs as well as cognitive biases that have previously been associated with non-traditional beliefs, e.g., repetition avoidance when producing random numbers in a mental dice task. In a classroom, 91 students saw a magic demonstration after their psychology lecture. Before the demonstration, half of the students were told that the performance was done respectively by a conjuror (magician group) or a psychic (psychic group). The instruction influenced participants' explanations of the anomalous event. Participants in the magician, as compared to the psychic group, were more likely to explain the event through conjuring abilities while the reverse was true for psychic abilities. Moreover, these explanations correlated positively with their prior traditional and non-traditional beliefs. Finally, we observed that the psychic group showed more repetition avoidance than the magician group, and this effect remained the same regardless of whether assessed before or after the magic demonstration. We conclude that pre-existing beliefs and contextual suggestions both influence people's interpretations of anomalous events and associated cognitive biases. Beliefs and associated cognitive biases are likely flexible well into adulthood and change with actual life events.