Performing meta-analysis with incomplete statistical information in clinical trials

Background: Results from clinical trials are usually summarized in the form of sampling distributions. When full information (mean, SEM) about these distributions is given, performing meta-analysis is straightforward. However, when some of the sampling distributions only have mean values, a challenging issue is to decide how to use such distributions in meta-analysis. Currently, the most common approaches are either ignoring such trials or for each trial with a missing SEM, finding a similar trial and taking its SEM value as the missing SEM. Both approaches have drawbacks. As an alternative, this paper develops and tests two new methods, the first being the prognostic method and the second being the interval method, to estimate any missing SEMs from a set of sampling distributions with full information. A merging method is also proposed to handle clinical trials with partial information to simulate meta-analysis.

Temporal and statistical information in causal structure learning

Three experiments examined children’s and adults’ abilities to use statistical and temporal information to distinguish between common cause and causal chain structures. In Experiment 1, participants were provided with conditional probability information and/or temporal information and asked to infer the causal structure of a three-variable mechanical system that operated probabilistically. Participants of all ages preferentially relied on the temporal pattern of events in their inferences, even if this conflicted with statistical information. In Experiments 2 and 3, participants observed a series of interventions on the system, which in these experiments operated deterministically. In Experiment 2, participants found it easier to use temporal pattern information than statistical information provided as a result of interventions. In Experiment 3, in which no temporal pattern information was provided, children from 6-7 years, but not younger children, were able to use intervention information to make causal chain judgments, although they had difficulty when the structure was a common cause. The findings suggest that participants, and children in particular, may find it more difficult to use statistical information than temporal pattern information because of its demands on information processing resources. However, there may also be an inherent preference for temporal information.

When does information about causal structure improve statistical reasoning?

Base rate neglect on the mammography problem can be overcome by explicitly presenting a causal basis for the typically vague false-positive statistic. One account of this causal facilitation effect is that people make probabilistic judgements over intuitive causal models parameterized with the evidence in the problem. Poorly defined or difficult-to-map evidence interferes with this process, leading to errors in statistical reasoning. To assess whether the construction of parameterized causal representations is an intuitive or deliberative process, in Experiment 1 we combined a secondary load paradigm with manipulations of the presence or absence of an alternative cause in typical statistical reasoning problems. We found limited effects of a secondary load, no evidence that information about an alternative cause improves statistical reasoning, but some evidence that it reduces base rate neglect errors. In Experiments 2 and 3 where we did not impose a load, we observed causal facilitation effects. The amount of Bayesian responding in the causal conditions was impervious to the presence of a load (Experiment 1) and to the precise statistical information that was presented (Experiment 3). However, we found less Bayesian responding in the causal condition than previously reported. We conclude with a discussion of the implications of our findings and the suggestion that there may be population effects in the accuracy of statistical reasoning.

Communicating clinical trial outcomes: Effects of presentation method on physicians’ evaluations of new treatments

Physicians expect a treatment to be more effective when its clinical outcomes are described as relative rather than as absolute risk reductions. We examined whether effects of presentation method (relative vs. absolute risk reduction)
remain when physicians are provided the baseline risk information, a vital piece of statistical information omitted in previous studies. Using a between-subjects design, ninety five physicians were presented the risk reduction associated
with a fictitious treatment for hypertension either as an absolute risk reduction or as a relative risk reduction, with or without including baseline risk information. Physicians reported that the treatment would be more effective and that they would be more willing to prescribe it when its risk reduction was presented to them in relative rather than in absolute terms. The relative risk reduction was perceived as more effective than absolute risk reduction even when the baseline risk information was explicitly reported. We recommend that information about absolute risk reduction be made available to physicians in the reporting of clinical outcomes. Moreover, health professionals should be cognizant of the potential biasing effects of risk information presented in relative risk terms

Whose statistical reasoning is improved by information about causal structure?

People often struggle when making Bayesian probabilistic estimates on the basis of competing sources of statistical evidence. Recently, Krynski and Tenenbaum (Journal of Experimental Psychology: General, 136, 430–450, 2007) proposed that a causal Bayesian framework accounts for peoples’ errors in Bayesian reasoning and showed that, by clarifying the causal relations among the pieces of evidence, judgments on a classic statistical reasoning problem could be significantly improved. We aimed to understand whose statistical reasoning is facilitated by the causal structure intervention. In Experiment 1, although we observed causal facilitation effects overall, the effect was confined to participants high in numeracy. We did not find an overall facilitation effect in Experiment 2 but did replicate the earlier interaction between numerical ability and the presence or absence of causal content. This effect held when we controlled for general cognitive ability and thinking disposition. Our results suggest that clarifying causal structure facilitates Bayesian judgments, but only for participants with sufficient understanding of basic concepts in probability and statistics.