Principal led school improvement and teacher capacity building in the Barwon South Network : final report, May 2013

The period of interest for this report is the beginning of 2011 to the end of 2012. The period commenced when the Regional Network Leader of the Barwon South Network of schools in the Barwon South Region of the Department of Education and Early Childhood contacted the School of Education at Deakin University, Waurn Ponds Campus Geelong. The Regional Network Leader outlined a desire to engage with Deakin University to research a short-term-cycle model of school improvement to be implemented in the region. While the model was expected to be taken on by all schools in the region the research was limited to the 23 schools in the Barwon South Network with four schools to be investigated more closely for each of two years (2001 & 2012) – eight focus schools in total.

Many positive outcomes flowed from the implementation of short-term-cycle school improvement plans and their associated practices but there was wide variation in the nature and degrees of success and of the perception of the process. The research team asked the following questions of the data:

1. What aspects of the School Improvement Plan (SIP) approach were important for initiating and supporting worthwhile change?
2. What might we take from this, to provide guidance on how best to support change in teaching and learning processes in schools?

The School Improvement Plan (SIP) worked in a range of ways. At one level it was strongly focused on school leadership, and a need to improve principals’ capacity to initiate worthwhile teaching and learning processes in their schools. Underlying this intent, one might think an assumption is operation is that the leadership process involves top down decision-making and a willingness to hold staff accountable for the quality of their practice.

The second strong focus was on the translation into practice and the consequent effect on student learning, involving an emphasis on data and evidence led practice. Hence, along with the leadership focus there was a demand for the process of school improvement to reach down into students and classrooms. Thus, the SIP process inevitably involved a chain of decision-making by which student learning quality drove the intervention, and teachers responsible for this had a common view. The model therefore should not be seen as an intervention only on the principal, but rather on the school decision-making system and focus. Even though it was the principal receiving the SIP planning template, and reporting to the network, the reporting was required to include description of the operation of the school processes, of classroom processes, and of student learning. This of course placed significant constraints on principals, which may help explain the variation in responses and outcomes described above.

The findings from this study are based on multiple data sources: analysis of both open and closed survey questions which all teachers in the 23 schools in the network were invited to complete; interviews with principals, teachers and leaders in the eight case study schools; some interviews with students in the case study schools; and interviews with leaders who worked in the regional network office; and field notes from network meetings including the celebrations days. Celebrations days occurred each school term when groups of principals came together to share and celebrate the improvements and processes happening in their schools. Many of the themes emerging from the analysis of the different data sources were similar or overlapping, providing some confidence in the evidence-base for the findings.

The study, conducted over two years of data collection and analysis, has demonstrated a range of positive outcomes in at the case study schools relating to school communication and collaboration processes, professional learning of principals, leadership teams and classroom teachers. There was evidence in the survey responses and field notes from ‘celebration days’ that these outcomes were also represented in other schools in the network. The key points of change concerned the leadership processes of planning for improvement, and the rigorous attention to student data in framing teaching and learning processes. This latter point of change had the effect of basing SIP processes on a platform of evidence-based change. The research uncovered considerable anecdotal and observational evidence of improvements in student learning, in teacher accounts in interview, and presentations of student work. Interviews with students, although not as representative as the team would have liked, showed evidence of student awareness of learning goals, a key driver in the SIP improvement model. It was, however, not possible over this timescale to collect objective comparative evidence of enhanced learning outcomes.

A number of features of the short-term-cycle SIP were identified that supported positive change across the network. These were: 1) the support structures represented by the network leader and support personnel within schools, 2) the nature of the SIP model – focusing strongly on change leadership but within a collaborative structure that combined top-down and bottom-up elements, 3) the focus on data-led planning and implementation that helped drill down to explicit elements of classroom practice, and 4) the accountability regimes represented by network leader presence, and the celebration days in which principals became effectively accountable to their peers. We found that in the second year of the project, momentum was lost in the case study schools, as the network was dismantled. This raised issues also for the conduct of research in situations of systemic change.

Alongside the finding of evidence of positive outcomes in the case study schools overall, was the finding that the SIP processes and outcomes varied considerably across schools. A number of contextual factors were identified that led to this variation, including school histories of reform, principal management style, and school size and structure that made the short-term-cycle model unmanageable. In some cases there was overt resistance to the SIP model, at least in some part, and this led to an element of performativity in which the language of the SIP was conscripted to other purposes. The study found that even with functioning schools the SIP was understood differently and the processes performed differently, raising the question of whether in the study we are dealing with one SIP or many. The final take home message from the research is that schools are complex institutions, and models of school improvement need to involve both strong principled features, and flexibility in local application, if all schools’ interests in improving teaching and learning processes and outcomes are to be served.

Assessing cost-effectiveness in mental health: helping policy-makers prioritize and plan health services

Objective: We assessed, from a health sector perspective, options for change that could improve the efficiency of Australia's current mental health services by directing available resources toward 'best practice' cost-effective services.

Method: We summarize cost-effectiveness results of a range of interventions for depression, schizophrenia, attention deficit hyperactivity disorder and anxiety disorders that have been presented in previous papers in this journal. Recommendations for change are formulated after taking into account 'second-filter criteria' of equity, feasibility of implementing change, acceptability to stakeholders and the strength of the evidence. In addition, we estimate the impact on total expenditure if the recommended mental health interventions for depression and schizophrenia are to be implemented in Australia.

Results: There are cost-effective treatment options for mental disorders that are currently underutilized (e.g. cognitive–behavioural therapy (CBT) for depression and anxiety, bibliotherapy for depression, family interventions for schizophrenia and clozapine for the worst course of schizophrenia). There are also less cost-effective treatments in current practice (e.g. widespread use of olanzapine and risperidone in the treatment of established schizophrenia and, within those atypicals, a preference for olanzapine over risperidone). Feasibility of funding mechanisms and training of staff are the main second-filter issues for CBT and family interventions. Acceptability to various stakeholders is the main barrier to implementation of more cost-effective drug treatment regimens. More efficient drug intervention options identified for schizophrenia would cost A$68 million less than current practice. These savings would more than cover the estimated A$36M annual cost of delivering family interventions to the 51% of people with schizophrenia whom we estimated to be eligible and this would lead to an estimated 12% improvement in their health status. Implementing recommended strategies for depression would cost A$121M annually for the 24% of people with depression who seek care currently, but do not receive an evidence-based treatment.

Conclusions: Despite considerable methodological problems, a range of cost-effective and less cost-effective interventions for major mental disorders can be discerned. The biggest hurdle to implementation of more efficient mental health services is that this change would require reallocation of funds between interventions, between disorders and between service providers with different funding mechanisms.

Living improvement : a case study of a secondary school in England

Schools in England are being urged to personalize the curriculum and make school experiences more responsive to all students. We report on an evaluation project which investigated innovation in teaching and learning in a successful secondary school in the north of England. Data were gathered from a sample of teaching staff, a questionnaire to all students, lesson and meeting observations, and meetings with the governing body and parents: the student-guided evaluation canvassed recent and planned changes to the structure and experience of teaching and learning. This article illuminates how one school is breaking the ‘traditional schooling rules’ that limit improvement and how in doing so it is developing new insights about the nature and process of improvement. We show how staff and students have been engaged in the change process, and focus in particular on analysing the interplay between improvement as a plan, a practice and a lived experience.

Evaluation of individual program plan content and quality in a Victorian residential service

The study assesses the quality and content of Individual Program Plans for 284 Victorian adults with profound intellectual disabilities living in one residential service. A checklist was developed for improving plan quality, and measurement difficulties with checklists discussed. Recommends that a quality improvement process be implemented.

A preliminary investigation of the utility of the "Behavior support plan quality evaluation guide II" for use in Australia

Background The quality of behaviour support plans (BSPs) can be an important influence on the quality of the support provided to people with disability who show challenging behaviours. The Behavior Support Plan Quality Evaluation Guide II (BSP-QE II) is one tool that may be useful in assessing the quality of behaviour support plans. It has previously been used to assess BSPs to support children in schools in the USA. In this preliminary study, we examined the utility of the BSP-QE II for assessing behaviour support plans designed for adults with an intellectual disability in community support services in Australia.

Method  Experienced practitioners were surveyed concerning the relevance of BSP-QE II components to the evaluation of BSPs in Australia, and an audit was conducted of randomly selected BSPs submitted as a statutory requirement to the Office of the Senior Practitioner.

Results  The BSP-QE II showed evidence of acceptable construct validity and interrater reliability. There was evidence of a need for substantial improvement in the quality of BSPs, consistent with previous findings conducted using other audit tools.

Conclusions The findings support the utility of the BSP-QE II, to inform and evaluate service planning in supporting adults with intellectual disability who show challenging behaviour.