Looking Yonder : Longitudinal Research and Evaluation of the Yonder Participative Arts-based Learning Model Report 2010-2012

Since 2007, KITE Arts Education Program @ QPAC has been engaged in a series of arts and drama-based experiences for students in selected primary schools on the edges of Brisbane and in regional Queensland. The in-school workshop experiences of the program have culminated in a performance by the children for their school community, parents and carers at the Queensland Performing Arts Centre or a regional cultural venue. In conducting an analysis of the Yonder project, the researcher aimed to provide evidence of outcomes brought about through participation by schools, school staff, students and their communities in the Yonder project. To develop longitudinal data project initiators, participants were interviewed at six-monthly intervals to establish patterns of engagement and participation. The report analyses arts-based workshops conducted by the teacher artist in edge-city Brisbane and a regional centre; interviews with teachers and school administrators from the participating schools; interviews with teacher artist and professional artists; interviews with community partners; teacher professional development workshops; community-based workshops; performance outcomes that were the culminating events of the workshop program; student work samples and student reflections on the program. This document covers data and project outputs from February 2010 to July 2012. There have been five iterations of the Yonder project since its commencement in mid-2009 — three in regional Queensland (February–April 2010; February–May 2011; February–May 2012) and two in edge-city1 Brisbane (July–September 2010; August–October 2011). This report is a result of a research partnership between Queensland Performing Arts Centre and Queensland University of Technology (QUT) Creative Industries Faculty(Drama).

Evaluation of "Eat Well Queensland"

Eat Well Queensland 2002-2012: Smart Eating for a Healthier State (EWQ) was developed by the Queensland Public Health Forum in 2002 as a 10-year strategy to improve the health of Queenslanders through better food and nutrition. This study aimed to evaluate the implementation of EWQ and identify future strategic action required. Queensland Health funded a mid-point review of EWQ in 2008, to identify achievements, gaps, barriers and emerging issues associated with EWQ. 31 key stakeholders were interviewed, 83 stakeholders responded to an online survey, 150 stakeholders attended a state-wide practitioner workshop and 209 EWQ-related project reports were assessed.

The Yonder project : literacy and social competency development in primary school students through co-creation of live performance

Since 2007 Kite Arts Education Program (KITE), based at Queensland Performing Arts Centre (QPAC), has been engaged in delivering a series of theatre-based experiences for children in low socio-economic primary schools in Queensland. The artist in residence (AIR) project titled Yonder includes performances developed by the children with the support and leadership of teacher artists from KITE for their community and parents/carers,supported by a peak community cultural institution. In 2009,Queensland Performing Arts Centre partnered with Queensland University of Technology (QUT) Creative Industries Faculty (Drama) to conduct a three-year evaluation of the Yonder project to understand the operational dynamics, artistic outputs and the educational benefits of the project. This paper outlines the research findings for children engaged in the Yonder project in the interrelated areas of literacy development and social competencies. Findings are drawn from six iterations of the project in suburban locations on the edge of Brisbane city and in regional Queensland.

Calling for help : a responsive evaluation of Child Health Line

The project examined the responsiveness of the telenursing service provided by the Child Health Line (hereinafter referred to as CHL). It aimed to provide an account of population usage of the service, the call request types and the response of the service to the calls. In so doing, the project extends the current body of knowledge pertaining to the provision of parenting support through telenursing. Approximately 900 calls to the CHL were audio-recorded over the December 2005-2006 Christmas-New Year period. A protocol was developed to code characteristics of the call, the interactional features between the caller and nurse call-taker, and the extent to which there was (a) agreement on problem definition and the plan of action and (b) interactional alignment between nurse and caller. A quantitative analysis examined the frequencies of the main topics covered in calls to the CHL and any statistical associations between types of calls, length of calls and nurse-caller alignment. In addition, a detailed qualitative analysis was conducted on a subset of calls dealing with the nurse management of calls seeking medical advice and information. Key findings include: • Overall, 74% of the calls discussed parenting and child development issues, 48% discussed health/medical issues, and 16% were information-seeking calls. • More specifically: o 21% discussed health/medical and parenting and child development issues. o 3% discussed parenting and information-seeking issues. o 5% discussed health/medical, parenting/development and information issues. o 18% exclusively focussed on health and medical issues and therefore were outside the remit of the intended scope of the CHL. These calls caused interactional dilemmas for the nurse call-takers as they simultaneously dealt with parental expectations for help and the CHL guidelines indicating that offering medical advice was outside the remit of the service. • Most frequent reasons for calling were to discuss sleep, feeding, normative infant physical functions and parenting advice. • The average length of calls to the CHL was 7 minutes. • Longer calls were more likely to involve nurse call-takers giving advice on more than one topic, the caller displaying strong emotions, the caller not specifically providing the reason for the call, and the caller discussing parenting and developmental issues. • Shorter calls were characterised by the nurse suggesting that the child receive immediate medical attention, the nurse emphasising the importance or urgency of the plan of action, the caller referring to or requesting confirmation of a diagnosis, and caller and nurse call-taker discussion of health and medical issues. • The majority of calls, 92%, achieved parent-nurse alignment by the conclusion of the call. However, 8% did not. • The 8% of calls that were not aligned require further quantitative and qualitative investigation of the interactional features. The findings are pertinent in the current context where Child Health Line now resides within 13HEALTH. These findings indicate: 1. A high demand for parenting advice. 2. Nurse call-takers have a high level of competency in dealing with calls about parenting and normal child development, which is the remit of the CHL. 3. Nurse call-takers and callers achieve a high degree of alignment when both parties agree on a course of action. 4. There is scope for developing professional practice in calls that present difficulties in terms of call content, interactional behaviour and call closure. Recommendations of the project: 1. There are numerous opportunities for further research on interactional aspects of calls to the CHL, such as further investigations of the interactional features and the association of the features to alignment and nonalignment. The rich and detailed insights into the patterns of nurse-parent interactions were afforded by the audio-recording and analysis of calls to the CHL. 2. The regular recording of calls would serve as a way of increasing understanding of the type and nature of calls received, and provide a valuable training resource. Recording and analysing calls to CHL provides insight into the operation of the service, including evidence about the effectiveness of triaging calls. 3. Training in both recognising and dealing with problem calls may be beneficial. For example, calls where the caller showed strong emotion, appeared stressed, frustrated or troubled were less likely to be rated as aligned calls. In calls where the callers described being ‘at their wits end’, or responded to each proposed suggestion with ‘I’ve tried that’, the callers were fairly resistant to advice-giving. 4. Training could focus on strategies for managing calls relating to parenting support and advice, and parental well-being. The project found that these calls were more likely to be rated as being nonaligned. 5. With the implementation of 13HEALTH, future research could compare nurse-parent interaction following the implementation of triaging. Of the calls, 21% had both medical and parenting topics discussed and 5.3% discussed medical, parenting and information topics. Added to this, in 12% of calls, there was ambiguity between the caller and nurse call-taker as to whether the problem was medical or behavioural.

In the Wake of Cyclone Yasi : Facilitation and Evaluation in Community Narrative-Driven Projects

On 3 February 2011, Cyclone Yasi struck the coast of North Queensland, causing widespread damage. The cyclone destroyed the small coastal town of Cardwell, about 165 kilometres north of Townsville, Queensland. This chapter serves as a case study of a collaborative outreach project mobilised in response to this disaster in North Queensland. A public history research team, consisting of practitioners from the Queensland University of Technology’s Creative Industries Faculty, with the support of the Oral History Association of Australia, Queensland branch, partnered with the Cardwell and District Historical Society to support the society to collect community narratives in the wake of Cyclone Yasi.

Theoretical framework for a management decision evaluation system

A theoretical framework for a construction management decision evaluation system for project selection by means of a literature review. The theory is developed by the examination of the major factors concerning the project selection decision from a deterministic viewpoint, where the decision-maker is assumed to possess 'perfect knowledge' of all the aspects involved. Four fundamental project characteristics are identified together with three meaningful outcome variables. The relationship within and between these variables are considered together with some possible solution techniques. The theory is next extended to time-related dynamic aspects of the problem leading to the implications of imperfect knowledge and a non­deterministic model. A solution technique is proposed in which Gottinger's sequential machines are utilised to model the decision process,

REFRAME : a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

How to improve pupils' literacy? A cost-effectiveness analysis of a French educational project

The Action Lecture program is an innovative teaching method run in some nursery and primary schools in Paris and designed to improve pupils’ literacy. We report the results of an evaluation of this program. We describe the experimental protocol that was built to estimate the program’s impact on several types of indicators. Data were processed following a Differences-in-Differences (DID) method. Then we use the estimation of the impact on academic achievement to conduct a cost-effectiveness analysis and take a reduction of the class size program as a benchmark. The results are positive for the Action Lecture program.

Queensland police service : an evaluation of HealthStart 2012

The HS program consists of a voluntary health screening and personalised feedback (via a website) which incorporates physical measurements, psychological stress assessment and blood tests. The following report describes the results of a research project that evaluated the effectiveness of the QPS-HS program and examines the health benefits it offers its participants. This report has three main areas that correspond to the research questions and includes three primary aims: 1.Review the literature pertaining to the health, social and economic value of wellness programs in workplaces. In particular, we reviewed policing worksites and other safety sensitive workplaces, to understand best-practice wellness programming and return on investment, in terms of value to employees, social value to the community, and economic value to employers; 2.Evaluate health outcomes of participants in the HS program, including physical measurements, such as blood tests and psychological well-being. These measures were to form an outcome evaluation and assess the effectiveness of the HS program in positively impacting physical and psychological health of HS participants; and 3.Assess employee awareness and perceptions of the HS program for a process evaluation.

Development and formative evaluation of a motorcycle rider training intervention to address risk taking

The need to address on-road motorcycle safety in Australia is important due to the disproportionately high percentage of riders and pillions killed and injured each year. One approach to preventing motorcycle-related injury is through training and education. However, motorcycle rider training lacks empirical support as an effective road safety countermeasure to reduce crash involvement. Previous reviews have highlighted that risk-taking is a contributing factor in many motorcycle crashes, rather than merely a lack of vehicle-control skills (Haworth & Mulvihill, 2005; Jonah, Dawson & Bragg, 1982; Watson et al, 1996). Hence, though the basic vehicle-handling skills and knowledge of road rules that are taught in most traditional motorcycle licence training programs may be seen as an essential condition of safe riding, they do not appear to be sufficient in terms of crash reduction. With this in mind there is considerable scope for the improvement of program focus and content for rider training and education. This program of research examined an existing traditional pre-licence motorcycle rider training program and formatively evaluated the addition of a new classroom-based module to address risky riding; the Three Steps to Safer Riding program. The pilot program was delivered in the real world context of the Q-Ride motorcycle licensing system in the state of Queensland, Australia. Three studies were conducted as part of the program of research: Study 1, a qualitative investigation of delivery practices and student learning needs in an existing rider training course; Study 2, an investigation of the extent to which an existing motorcycle rider training course addressed risky riding attitudes and motives; and Study 3, a formative evaluation of the new program. A literature review as well as the investigation of learning needs for motorcyclists in Study 1 aimed to inform the initial planning and development of the Three Steps to Safer Riding program. Findings from Study 1 suggested that the training delivery protocols used by the industry partner training organisation were consistent with a learner-centred approach and largely met the learning needs of trainee riders. However, it also found that information from the course needs to be reinforced by on-road experiences for some riders once licensed and that personal meaning for training information was not fully gained until some riding experience had been obtained. While this research informed the planning and development of the new program, a project team of academics and industry experts were responsible for the formulation of the final program. Study 2 and Study 3 were conducted for the purpose of formative evaluation and program refinement. Study 2 served primarily as a trial to test research protocols and data collection methods with the industry partner organisation and, importantly, also served to gather comparison data for the pilot program which was implemented with the same rider training organisation. Findings from Study 2 suggested that the existing training program of the partner organisation generally had a positive (albeit small) effect on safety in terms of influencing attitudes to risk taking, the propensity for thrill seeking, and intentions to engage in future risky riding. However, maintenance of these effects over time and the effects on riding behaviour remain unclear due to a low response rate upon follow-up 24 months after licensing. Study 3 was a formative evaluation of the new pilot program to establish program effects and possible areas for improvement. Study 3a examined the short term effects of the intervention pilot on psychosocial factors underpinning risky riding compared to the effects of the standard traditional training program (examined in Study 2). It showed that the course which included the Three Steps to Safer Riding program elicited significantly greater positive attitude change towards road safety than the existing standard licensing course. This effect was found immediately following training, and mean scores for attitudes towards safety were also maintained at the 12 month follow-up. The pilot program also had an immediate effect on other key variables such as risky riding intentions and the propensity for thrill seeking, although not significantly greater than the traditional standard training. A low response rate at the 12 month follow-up unfortunately prevented any firm conclusions being drawn regarding the impact of the pilot program on self-reported risky riding once licensed. Study 3a further showed that the use of intermediate outcomes such as self-reported attitudes and intentions for evaluation purposes provides insights into the mechanisms underpinning risky riding that can be changed by education and training. A multifaceted process evaluation conducted in Study 3b confirmed that the intervention pilot was largely delivered as designed, with course participants also rating most aspects of training delivery highly. The complete program of research contributed to the overall body of knowledge relating to motorcycle rider training, with some potential implications for policy in the area of motorcycle rider licensing. A key finding of the research was that psychosocial influences on risky riding can be shaped by structured education that focuses on awareness raising at a personal level and provides strategies to manage future riding situations. However, the formative evaluation was mainly designed to identify areas of improvement for the Three Steps to Safer Riding program and found several areas of potential refinement to improve future efficacy of the program. This included aspects of program content, program delivery, resource development, and measurement tools. The planned future follow-up of program participants' official crash and traffic offence records over time may lend further support for the application of the program within licensing systems. The findings reported in this thesis offer an initial indication that the Three Steps to Safer Riding is a useful resource to accompany skills-based training programs.

The structured and practical approach in development of decision support system for consultant selection in public sector infrastructure project

Nowadays, most of the infrastructure development projects undertaken are complex in nature. Practically, public clients who do not have a good understanding of the design and management may suffer severe losses, especially for infrastructure projects. There is a need for luring the right consultant to secure client's investment in infrastructure developments. Throughout the project life cycle, consultants play vital role from the inception to completion stage of a project. A few studies in Malaysia show that infrastructure projects involving irrigation and drainage have experience problems such as poor workmanship, delay and cost overrun due to the consultant's inability or the client incompetence of recruiting consultants in time. This highlights the need of aided decision making and an efficient system to select the best consultant by using Decision Support System (DSS). On the other hand, recent trends reveal that most DSS in construction only concentrate on decision model development. These models are impractical and unused as they are complicated or difficult for laymen such as project managers to utilize. Thus, this research attempts to develop an efficient DSS for consultant selection namely consultDeSS. Driven by the motivation and research aims, this study deployed Design Science Research Methodology (DSRM) dominant with a combination of case studies at the Malaysian Department of Irrigation and Drainage (DID). Two real projects involving irrigation and drainage infrastructure were used to design, implement and evaluate the artefact. The 3-tier consultDeSS was revised after the evaluation and the design was significantly improved based on user feedback. By developing desirable tools that fit client's needs will enhance the productivity and minimize conflict within groups and organisations. The tool is more usable and efficient compared to previous studies in construction. Thus, this research has demonstrated a purposeful artefact with a practical and valid structured development approach that is applicable in a variety of problems in construction discipline.

Kaleidoscope : Reframing learning and teaching evaluation through a shared stakeholder lens

Sector wide interest in Reframe: QUT’s Evaluation Framework continues with a number of institutions requesting finer details as QUT embeds the new approach to evaluation across the university in 2013. This interest, both nationally and internationally has warranted QUT’s collegial response to draw upon its experiences from developing Reframe into distilling and offering Kaleidoscope back to the sector. The word Reframe is a relevant reference for QUT’s specific re-evaluation, reframing and adoption of a new approach to evaluation; whereas Kaleidoscope reflects the unique lens through which any other institution will need to view their own cultural specificity and local context through an extensive user-led stakeholder engagement approach when introducing new approaches to learning and teaching evaluation. Kaleidoscope’s objectives are for QUT to develop its research-based stakeholder approach to distil the successful experience exhibited in the Reframe Project into a transferable set of guidelines for use by other tertiary institutions across the sector. These guidelines will assist others to design, develop, and deploy, their own culturally specific widespread organisational change informed by stakeholder engagement and organisational buy-in. It is intended that these guidelines will promote, support and enable other tertiary institutions to embark on their own evaluation projects and maximise impact. Kaleidoscope offers an institutional case study of widespread organisational change underpinned by Reframe’s (i) evidence-based methodology; (ii) research including published environmental scan, literature review (Alderman, et al., 2012), development of a conceptual model (Alderman, et al., in press 2013), project management principles (Alderman & Melanie, 2012) and national conference peer reviews; and (iii) year-long strategic project with national outreach to collaboratively engage the development of a draft set of National Guidelines. Kaleidoscope’s aims are to inform Higher Education evaluation policy development through national stakeholder engagement, the finalisation of proposed National Guidelines. In correlation with the conference paper, the authors will present a Draft Guidelines and Framework ready for external peer review by evaluation practitioners from the Higher Education sector, as part of Kaleidoscope’s dissemination strategy (Hinton & Gannaway, 2011) applying illuminative evaluation theory (Parlett & Hamilton, 1976), through conference workshops and ongoing discussions (Shapiro, et al., 1983; Jacobs, 2000). The initial National Guidelines will be distilled from the Reframe: QUT’s Evaluation Framework’s Policy, Protocols, and incorporated Business Rules. It is intended that the outcomes of Kaleidoscope are owned by and reflect sectoral engagement, including iterative evaluation through multiple avenues of dissemination and collaboration including the Higher Education sector. The dissemination strategy with the inclusion of Illuminative Evaluation methodology provides an inclusive opportunity for other institutions and stakeholders across the Higher Education sector to give voice through the information-gathering component of evaluating the draft Guidelines, providing a comprehensive understanding of the complex realities experienced across the Higher Education sector, and thereby ‘illuminating’ both the shared and unique lenses and contexts. This process will enable any final guidelines developed to have broader applicability, greater acceptance, enhanced sustainability and additional relevance benefiting the Higher Education sector, and the adoption and adaption by any single institution for their local contexts.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

Aboriginal dietary intake and a successful nutrition intervention project in a Northern Territory community

This paper summarises the development and testing of the 'store-turnover' method, a non-invasive dietary survey methodology for quantitative measurement of food and nutrient intake in remote, centralised Aboriginal communities. It then describes the use of the method in planning, implementation and evaluation of a community-based nutrition intervention project in a small Aboriginal community in the Northern Territory. During this project marked improvements in both the dietary intake of the community and biological indicators of nutritional health (including vitamin status and the degree and prevalence of several risk factors for non-communicable disease) were measured in the community over a 12-month period following the development of intervention strategies with the community. Although these specific strategies are presented, emphasis is directed towards the process involved, particularly the evaluation procedures used to monitor all stages of the project with the community.

A collaborative robotics engagement project

This paper reports on an evaluation of a collaborative robotics engagement project involving teachers from local schools and an academic from Queensland University of Technology (QUT). Engaged community projects are aimed at building stronger relationships between universities and their local communities (Sandman, Williams & Abrams, 2009). This partnership leads to mutually beneficial outcomes, builds community capacity, and can focus on aspirations and access to higher education for school students (Scull & Cuthill, 2010). The Robotics@QUT project aimed to build a partnership between local teachers and the university in order to provide students from a low SES area opportunity to engage in robotics-based Science, Technology, Engineering, and Mathematics (STEM) activities. Students from low SES regions are underrepresented at university and less likely to pursue studies in these fields (Bradley, Noonan, Nugent, & Scales, 2008). Having teachers who provide engaging STEM activities is an important motivating factor for students to enjoy STEM and do well in STEM subjects (Tytler, Osborne Williams Tytler & Clark, 2008).