Methodological challenges in conducting a systematic review of community wide interventions to increase physical activity

Introduction: Evaluating the effectiveness of interventions designed to increase the physical activity in communities is often a difficult and complex task, requiring considerable expertise and investment, and often constrained by methodological limitations. These limitations, in turn, create additional challenges when these studies are used in systematic reviews as they hinder the confidence, precision and interpretation of results. The objective of this paper is to summarise the methodological challenges posed in conducting a systematic review of community-wide physical activity interventions to help inform those conducting future primary research and systematic reviews. Methods: We conducted a Cochrane systematic review of community-wide interventions to increase physical activity. We assessed the methodological quality of the included studies. We will investigate these in greater detail, particularly in relation to the potential impact on measures of effect, confidence in results, generalizability of results and general interpretation. Results: The systematic review was conducted and has been published in the Cochrane Library. A logic model was helpful in defining and interpreting the studies. Many studies of unsuitable study design were excluded; however several important methodological limitations of the primary studies evaluating community-wide physical activity interventions emerged. These included: - the failure to use validated tools to measure physical activity; - issues associated with pre and post test designs; - inadequate sampling of populations; - poor control groups; and - intervention and measurement protocols of inadequate duration. Although it is challenging to undertake rigorous evaluations of complex interventions, these issues result in significant uncertainty over the effectiveness of these interventions, and the possible factors required for a community-wide intervention to be successful. In particular, the combination of several of these limitations (e.g. un-validated tools, inadequate sampling, and short duration) is that studies may lack the sensitivity to detect any meaningful change. Multiple publications of findings for the same study also made interpretation difficult; however, interventions with parallel qualitative publications were helpful. Discussion: Evaluating community wide interventions to increase physical activity in a rigorous way is incredibly challenging. These findings reflect these challenges but have important ramifications for researchers conducting primary studies to determine the efficacy of such interventions, as well as for researchers conducting systematic reviews. This new review shows that the inadequacies of design and evaluation are continuing. It is hoped that the adoption of such suggestions may aid in the development of systematic reviews, but more importantly, in enabling translation of such findings into policy and practice.