Quantitative evaluation in health promotion : evaluation with confidence

Over the past 10 years or so, confidence intervals have become increasingly recognised in program evaluation and quantitative health measurement generally as the preferred way of reporting the accuracy of statistical estimates. Statisticians have found that the more traditional ways of reporting results - using P-values and hypothesis tests - are often very difficult to interpret and can be misleading. This is particularly the case when sample sizes are small and results are 'negative' (ie P>0.05); in these cases, a confidence interval can communicate much more information about the sample and, by inference, about the population. Despite this trend among statisticians and health promotion evaluators towards the use of confidence intervals, it is surprisingly difficult to find succinct and reasonably simple methods to actually compute a confidence interval. This is particularly the case for proportions or percentages. Much of the data which are analysed in health promotion are binary or categorical, rather than the quantities and continuous variables often found in laboratories or other branches of science, so there is a need for health promotion evaluators to be able to present confidence intervals for percentages or proportions. However, the most popular statistical analysis computer package among health promotion professionals, SPSS does not have a routine to compute a simple confidence interval for a proportion! To address this shortcoming, I present in this paper some fairly simple strategies for computing confidence intervals for population percentages, both manually and using the right computer software.

A suite of visual languages for model-driven development of statistical surveys and services

Objective: To provide statistician end users with a visual language environment for complex statistical survey design and implementation. Methods: We have developed, in conjunction with professional statisticians, the Statistical Design Language (SDL), an integrated suite of visual languages aimed at supporting the process of designing statistical surveys, and its support environment, SDLTool. SDL comprises five diagrammatic notations: survey diagrams, data diagrams, technique diagrams, task diagrams and process diagrams. SDLTool provides an integrated environment supporting design, coordination, execution, sharing and publication of complex statistical survey techniques as web services. SDLTool allows association of model components with survey artefacts, including data sets, metadata, and statistical package analysis scripts, with the ability to execute elements of the survey design model to implement survey analysis. Results: We describe three evaluations of SDL and SDLTool: use of the notation by expert statistician to design and execute surveys; useability evaluation of the environment; and assessment of several generated statistical analysis web services. Conclusion: We have shown the effectiveness of SDLTool for supporting statistical survey design and implementation. Practice implications: We have developed a more effective approach to supporting statisticians in their survey design work.

Guidelines for developing and reporting machine learning predictive models in biomedical research : a multidisciplinary view

BACKGROUND: As more and more researchers are turning to big data for new opportunities of biomedical discoveries, machine learning models, as the backbone of big data analysis, are mentioned more often in biomedical journals. However, owing to the inherent complexity of machine learning methods, they are prone to misuse. Because of the flexibility in specifying machine learning models, the results are often insufficiently reported in research articles, hindering reliable assessment of model validity and consistent interpretation of model outputs.

OBJECTIVE: To attain a set of guidelines on the use of machine learning predictive models within clinical settings to make sure the models are correctly applied and sufficiently reported so that true discoveries can be distinguished from random coincidence.

METHODS: A multidisciplinary panel of machine learning experts, clinicians, and traditional statisticians were interviewed, using an iterative process in accordance with the Delphi method.

RESULTS: The process produced a set of guidelines that consists of (1) a list of reporting items to be included in a research article and (2) a set of practical sequential steps for developing predictive models.

CONCLUSIONS: A set of guidelines was generated to enable correct application of machine learning models and consistent reporting of model specifications and results in biomedical research. We believe that such guidelines will accelerate the adoption of big data analysis, particularly with machine learning methods, in the biomedical research community.