Grading gems: Appraising the quality of research about social work and social care.

The impetus towards basing practice and policy decisions more explicitly on sound research requires tools to facilitate the systematic appraisal of the quality of research encompassing a diverse range of methods and designs. Five exemplar tools were developed and assessed in terms of their usefulness in selecting studies for inclusion in a systematic review. The widely used ‘hierarchy of evidence’ was adapted and used to appraise internal validity. Four tools were then developed to appraise the external validity dimensions of generalizability (two scales) and methods of data collection (two scales). Methods of combining the scores generated by each tool were explored. Qualitative and quantitative studies were appraised, not separated into two spheres but by using complementary tools developed to appraise different aspects of rigour. There was a high level of agreement between researchers in applying the tools to twenty-two studies on decision making by professionals about the longterm care of older people. The scales for internal validity and generalizability discriminated between the qualities of studies appropriately. The two tools to appraise data collection gave diverse results. Excluding studies that scored in the lowest category on any scale appeared to be the scoring system that was most justifiable. This approach is presented to stimulate debate about the practical application of the evidence-based initiative to social work and social care. This study may assist in developing clearer definitions and common language about appraising rigour that should further the process of selecting robust research for synthesis to inform practice and policy decisions.

The Growth, Learning and Development Study: Summary of Research, Findings and Recommendations.

The Growth, Learning and Development (GLAD) study aimed to examine how a broad range of factors influence child weight during the first year of life. Assessments were undertaken within a multidisciplinary team framework. The sample was drawn from the community and data collection was undertaken in the four Greater Belfast Trusts. Twohundred and thirty-four families took part, each receiving a total of five home visits during which physical growth, oral-motor skills and development were assessed. Psychosocial evaluation examined parent-child interaction, feeding and other parental and child characteristics using quantitative and observational techniques. This paper outlines the main findings and recommendations from the GLAD study.

Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact

There is a growing literature examining the impact of research on informing policy, and of research and policy on practice. Research and policy do not have the same types of impact on practice but can be evaluated using similar approaches. Sometimes the literature provides a platform for methodological debate but mostly it is concerned with how research can link to improvements in the process and outcomes of education, how it can promote innovative policies and practice, and how it may be successfully disseminated. Whether research-informed or research-based, policy and its implementation is often assessed on such 'hard' indicators of impact as changes in the number of students gaining five or more A to C grades in national examinations or a percentage fall in the number of exclusions in inner city schools. Such measures are necessarily crude, with large samples smoothing out errors and disguising instances of significant success or failure. Even when 'measurable' in such a fashion, however, the impact of any educational change or intervention may require a period of years to become observable. This paper considers circumstances in which short-term change may be implausible or difficult to observe. It explores how impact is currently theorized and researched and promotes the concept of 'soft' indicators of impact in circumstances in which the pursuit of conventional quantitative and qualitative evidence is rendered impractical within a reasonable cost and timeframe. Such indicators are characterized by their avowedly subjective, anecdotal and impressionistic provenance and have particular importance in the context of complex community education issues where the assessment of any impact often faces considerable problems of access. These indicators include the testimonies of those on whom the research intervention or policy focuses (for example, students, adult learners), the formative effects that are often reported (for example, by head teachers, community leaders) and media coverage. The collation and convergence of a wide variety of soft indicators (Where there is smoke …) is argued to offer a credible means of identifying subtle processes that are often neglected as evidence of potential and actual impact (… there is fire).