Sing & Grow : the co-existence of evaluation research and clinical practice in an early intervention music therapy project

Sing & Grow is a short term early intervention music therapy program for at risk families. Sing & Grow uses music to strengthen parent-child relationships by increasing positive parent-child interactions, assisting parents to bond with their children, and extending the repertoire of parents’ skills in relating to their child through interactive . Both the Australian and New Zealand governments are looking for evidence based research to highlight the effectiveness of funded programs in early childhood. As a government funded program, independent evaluation is a requirement of the delivery of the service. This paper explains the process involved in setting up and managing this large scale evaluation from engaging the evaluators and designing the project, to the data gathering stage. It describes the various challenges encountered and concludes that a highly collaborative and communicative partnership bet en researchers and clinicians is essential to ensure data can be gathered with minimal disturbance to clinical music therapy practice.

A formative evaluation of the Stronger Smarter Learning Communities Project. 2011 report.

This report is a formative evaluation of the operations of the DEEWR funded Stronger Smarter Learning Community (SSLC) project from September 2009 to July 2011. It is undertaken by an independent team of researchers from Queensland University of Technology, the University of Newcastle and Harvard University. It reports on findings from: documentary analysis; qualitative case studies of SSLC Hub schools; descriptive, multivariate and multilevel analysis of survey data from school leaders and teachers from SSLC Hub and Affiliate schools and from a control group of non-SSLC schools; and multilevel analysis of school-level data on SSLC Hubs, Affiliates and ACARA like-schools. Key findings from this work are that: • SSLC school leaders and teachers are reporting progress in changing school ethos around issues of: recognition of Indigenous identity, Indigenous leadership, innovative approaches to staffing and school models, Indigenous community engagement and high expectations leadership; • Many Stronger Smarter messages are reportedly having better uptake in schools with high percentages of Indigenous students; • There are no major or consistent patterns of differences between SSLC and non-SSLC schools in teacher and school leader self-reports of curriculum and pedagogy practices; and • There is no evidence to date that SSLC Hubs and Affiliates have increased attendance or increased achievement gains compared to ACARA like-schools. Twenty-one months is relatively early in this school reform project. Hence the major focus of subsequent reports will be on the documentation of comparative longitudinal gains in achievement tests and improved attendance. The 2011 and 2012 research also will model the relationships between change in school ethos/climate, changed Indigenous community relations, improved curriculum/pedagogy, and gains in Indigenous student achievement, attendance and outcomes. The key challenge for SSLC and the Stronger Smarter approach will be whether it can systematically generate change and reform in curriculum and pedagogy practices that can be empirically linked to improved student outcomes.

Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Evaluation of the "reconceptualising early mathematics learning" project

The Pattern and Structure Mathematics Awareness Project (PASMAP) has investigated the development of patterning and early algebraic reasoning among 4 to 8 year olds over a series of related studies. We assert that an awareness of mathematical pattern and structure enables mathematical thinking and simple forms of generalisation from an early age. The project aims to promote a strong foundation for mathematical development by focusing on critical, underlying features of mathematics learning. This paper provides an overview of key aspects of the assessment and intervention, and analyses of the impact of PASMAP on students’ representation, abstraction and generalisation of mathematical ideas. A purposive sample of four large primary schools, two in Sydney and two in Brisbane, representing 316 students from diverse socio-economic and cultural contexts, participated in the evaluation throughout the 2009 school year and a follow-up assessment in 2010. Two different mathematics programs were implemented: in each school, two Kindergarten teachers implemented the PASMAP and another two implemented their regular program. The study shows that both groups of students made substantial gains on the ‘I Can Do Maths’ assessment and a Pattern and Structure Assessment (PASA) interview, but highly significant differences were found on the latter with PASMAP students outperforming the regular group on PASA scores. Qualitative analysis of students’ responses for structural development showed increased levels for the PASMAP students; those categorised as low ability developed improved structural responses over a relatively short period of time.

A Summative Evaluation of the Stronger Smarter Learning Communities Project : Vol 1 and Vol 2

This is a summative evaluation of the Stronger Smarter Learning Communities (SSLC) project that examines whether and how the SSLC project had an impact on Australian state schools which adopted its models and approaches. Drawing from qualitative and quantitative data sets, it also presents the largest scale and most comprehensive analysis of Indigenous education practices and outcomes to date. It includes empirical findings on: success in changing school ethos and community engagement; challenges in progress at closure of the 'gap' in conventionally measured achievement and performance; schools' and principals' choices in curriculum and instruction; profiles of teachers' and principals' training and views on teacher education; and a strong emphasis on community and school Indigenoous voices and views on Indigenous education.

Empowering the school community for change : an evaluation of participation in the Ashgrove Healthy School Environment Project

The purpose of this study was to evaluate aspects of participation in a participatory action research project, the Ashgrove Healthy School Environment Project. Participatory action research is a form of research that creates change as an explicit part of the research process and requires the active participation of those by and for whom the research is being conducted. This study arose from concems by this researcher, who is also a co-facilitator of the project, that levels of participation were not as extensive as one might have hoped and that this seemingly low level would have a negative impact on the continuing participation of those already involved. Specifically. this evaluation sought to uncover the reasons that prompted participation, to identify structural barriers to initial involvement and to uncover participants' perceptions of the process, including barriers and opportunities. It also sought to record evidence of any shift in decision making and to draw implications about the findings that could assist the project, the school, other schools and the wider community. This evaluation involved focus group discussions and interviews with participants actively involved in the school project. The purpose was to uncover their views, feelings and perceptions about their participation and the participatory processes in use generally. It also included some examination of school documents and newsletters and as also drawn on the reflections of this 'insider' researcher, based on two years of involvement in facilitating the project. The findings that emerge from this study are heartening. Rather than feeling anxious about the long-term sustainability of the project, this researcher now feels more confident about its achievements, both in terms of the changes that have occurred in the school and about the participatory processes and levels of participation. Whilst the evaluation has identified a number of barriers, both institutional, personal and project related, it has also identified several key factors that serve to promote participation.

Paper, prices and politics : an evaluation of the Swedish support to the Bai Bang project in Vietnam : a Sida evaluation report

Sweden’s protest against the Vietnam War was given tangible form in 1969 through the decision to give economic aid to the Government of North Vietnam. The main outcome was an integrated pulp and paper mill in the Vinh Phu Province north-west of Hanoi. Known as Bai Bang after its location, the mill became the most costly, one of the longest lasting and the most controversial project in the history of Swedish development cooperation. In 1996 Bai Bang produced at its full capacity. Today the mill is exclusively managed and staffed by the Vietnamese and there are plans for future expansion. At the same time a substantial amount of money has been spent to reach these achievements. Looking back at the cumbersome history of the project the results are against many’s expectations. To learn more about the conditions for sustainable development Sida commissioned two studies of the Bai Bang project. Together they touch upon several important issues in development cooperation over a period of almost 30 years: the change of aid paradigms over time, the role of foreign policy in development cooperation, cultural obstacles, recipient responsibility versus donor led development etc. The two studies were commissioned by Sida’s Department for Evaluation and Internal Audit which is an independent department reporting directly to Sida’s Board of Directors. One study assesses the financial and economic viability of the pulp and paper mill and the broader development impact of the project in Vietnam. It has been carried out by the Centre for International Economics, an Australian private economic research agency. The other study analyses the decision-making processes that created and shaped the project over a period of two decades, and reflects on lessons from the project for development cooperation in general. This study has been carried out by the Chr. Michelsen Institute, a Norweigan independent research institution.

An evaluation of the Australian ‘reconceptualising early mathematics learning’ project : key findings and implications

The Pattern and Structure Mathematics Awareness Project (PASMAP) has investigated the development of patterning and early algebraic reasoning among 4 to 8 year olds over a series of related studies. We assert that an awareness of mathematical pattern and structure (AMPS) enables mathematical thinking and simple forms of generalization from an early age. This paper provides an overview of key findings of the Reconceptualizing Early Mathematics Learning empirical evaluation study involving 316 Kindergarten students from 4 schools. The study found highly significant differences on PASA scores for PASMAP students. Analysis of structural development showed increased levels for the PASMAP students; those categorised as low ability developed improved structural responses over a short period of time.

Evaluation of Indigenous justice programs.: final report / Project D, Safe Aboriginal Youth Patrol Programs in New South Wales and Northbridge Policy and Juvenile Aid Group in Western Australia

The Australian Government has provided funding to evaluate the effectiveness of Indigenous law and justice programs across five subject areas to identify the best approaches to tackling crime and justice issues and better inform government funding decisions in the future. This report presents the findings of subject area "D", which examined two different approaches to delivering community and night patrol services for young people: the Safe Aboriginal Youth Patrol programs in New South Wales, and the Northbridge Policy project (the Young People in Northbridge project), in Western Australia. Night patrols can address crime either directly or indirectly, by prevention work or by addressing the social causes of crime through community development.

Learnings, case studies and guidelines for establishing shared and collaborative service delivery in the non-government sector: Evaluation of the Multi-Tenant Service Centre (MTSC) Pilots Project

This report presents learnings, case studies, guidelines and resources for non-government organisations that are planning to implement shared or collaborative arrangements with other agencies. It summarises results from an evaluation of the implementation phase of the Multi-Tenant Service Centre (MTSC) Pilots Project, which was completed in June 2008. This evaluation shows that developing and implementing shared and collaborative arrangements is a complex process that presents many risks, challenges and barriers to success, but can have many potential benefits for non government organisations. As this report makes clear, there is no ‘one size fits all’ approach to this process. The MTSC Pilots Project was conducted by the Department of Communities (DoC), Queensland Government, as part of its Strengthening Non-Government Organisations strategy. The objective of the MTSC Pilots initiative was to co-locate separate service providers in an appropriately located centre, operating with effective and transparent management, which enabled service providers to improve client services. Three MTSC consortiums in Mackay, Caboolture and Toowoomba were selected as the pilots over a four year period from 2006 – 2010.

Synthetic Evaluation Project - Nematodes

The project tests synthetic hexaploid wheats for resistance to root-lesion nematodes.

Harvest strategy evaluation to optimise the sustainability and value of the Queensland scallop fishery. Queensland scallop fishery - FRDC Project No 2006/024 Final Report

Objective 1. Measure spatial and temporal trawl frequency of scallop grounds using VMS data. This will provide a relative measure of how often individual undersized scallops are caught and put through a tumbler 2. Estimate discard mortality and growth rates for saucer scallops using cage experiments. 3. Evaluate the current management measures, in particular the seasonal closure, rotational closure and seasonally varying minimum legal sizes using stock assessment and management modeling models. Recommend optimal range of management measures to ensure long-term viability and value of the Scallop fishery based on a formal management strategy evaluation. Outcomes acheived to date: 1. Improved understanding of the survival rates of discarded sub-legal scallops; 2. Preliminary von Bertalanffy growth parameters using data from tagged-and-released scallops; 3. Changing trends in vessels and fishing gear used in the Queensland scallop fishery and their effect on scallop catch rates over time using standardised catch rates quantified; 4. Increases in fishing power of vessels operating in the Queensland scallop fishery quantified; 5. Trawl intensity mapped and quantified for all Scallop Replenishment Areas; 6. Harvest Strategy Evaluations completed.

The Sports Demonstration Project – An initiative of the Australian Sports Commission: An Evaluation

The Sport Development Project (SDP) was a comprehensive youth strategy for sport in the Northern Territory aimed at diversion from ‘at-risk’ behaviours, improvement of life choices and outcomes, and strengthening youth service infrastructure through engagement in positive (sport) activities. There were five Remote Service Delivery sites that were involved in the trial of this ‘best practice’ model for delivering sport-focused diversion activities. These include: Gapuwiyak, Wadeye, Yuendumu, Gunbalunya and Nguiu.