Navigating theory, negotiating difference and beating the system : a critical how-to of media assistance evaluation

The wave of democratisation across Europe, Africa, Asia and Latin America in the early 1990s triggered an increase in donor funding to media assistance initiatives, primarily within good governance policy frameworks. However, few media assistance projects have managed to effectively evaluate the impacts of their work. This thesis explores how the impacts of Australian media assistance on social change and governance can be most effectively evaluated and understood. The findings of this research suggest the importance of early investment in participatory planning of evaluation designs, which are then periodically revisited. These evaluation designs should be based on a theoretically sound link between models of change, evaluative questions and methods.

Youth off the Streets "Our Place, Walgett Youth and Young Families Project" Project Evaluation.

The Project: • YOTS is a major youth specific agency established in 1991. It is a non-denominational, non-discriminatory and not-for-profit organisation, providing a wide range of services and offering a full continuum of care. It seeks to build on the strengths and positive aspects of marginalised young people and communities. • The 'Our Place, Walgett Youth and Young Families Project' further develops an existing YOTS capacity to provide services to Aboriginal young people. • The project adopted an action-research and community development model in which YOTS worked in partnership with the Youth Sub-committee of the Walgett Interagency. • Specific goals/objectives of the program were to: Coordinate youth and young family activities in partnership with local services and the community to build self-esteem, pride, resilience, motivation and skills; Contribute to the prevention and reduction of homelessness, unstable and unsafe housing and disruptive mobility (Walgett/Redfern) in youth and young families; Increase and improve collaborative engagement between youth and family focused services; and, research, adapt and implement Australian and international best-practice homelessness prevention/reduction initiatives to contribute to new models of practice relevant to rural and regional areas. • The project centred around an out-reach model that focused on providing a safe space with relevant structured activities coordinated by YOTS youth and family workers. Through community and service provider consultation, it was proposed that local services could coordinate strategies and activities and run them, where possible, from the centre, providing ease of access in a safe and supportive context. • Specific activities included: Implementing regular meetings with the stakeholders and community representatives; Developing a Terms of Reference for YOTS presence in the Walgett community; Undertaking a community consultation prior to finalising program activities; Implementing a range of recreational activities (sports, music, arts and crafts) early on in the activity; Implementing young family support initiatives; implementing a volunteering program, including volunteer support to young families through intergenerational volunteering; running a series of Culture and Healing Camps in partnership with local Elders and other services; Running a series of Music Camps; Providing alternative education support and referrals in partnership with local schools; Researching, identifying and adapting other best-practice models.

Evaluation of Indigenous justice programs.: final report / Project D, Safe Aboriginal Youth Patrol Programs in New South Wales and Northbridge Policy and Juvenile Aid Group in Western Australia

The Australian Government has provided funding to evaluate the effectiveness of Indigenous law and justice programs across five subject areas to identify the best approaches to tackling crime and justice issues and better inform government funding decisions in the future. This report presents the findings of subject area "D", which examined two different approaches to delivering community and night patrol services for young people: the Safe Aboriginal Youth Patrol programs in New South Wales, and the Northbridge Policy project (the Young People in Northbridge project), in Western Australia. Night patrols can address crime either directly or indirectly, by prevention work or by addressing the social causes of crime through community development.

Dynamic digital posters : making the most of collaborative learning spaces

Academic and professional staff at Queensland University of Technology (QUT) have been faced with the challenge of how to create engaging student experiences in collaborative learning spaces. In 2013 a new Bachelor of Science course was implemented focusing on inquiry-based, collaborative and active learning. Student groups in two of the first year units carried out a poster assessment task. This paper provides a preliminary evaluation of the assessment approach used, whereby students created dynamic digital posters to capitalise on the affordances of the learning space.

CoWs, Google and online possibilities : creating digital posters to enhance student engagement

A new Bachelor of Science (BSc) course was introduced at Queensland University of Technology (QUT) in 2013 and focused on inquiry-based, collaborative and active learning. Two of the first year units required that students carry out a group poster assessment task. This poster provides a preliminary evaluation from an academic staff perspective of the assessment approach used, whereby students created digital posters to utilise the affordances of new learning spaces. The digital posters approach was first introduced to a group of academic staff from the Science and Engineering Faculty (SEF) in 2012 during a professional development program to explicitly develop skills and shared understandings of teaching in collaborative learning spaces (Steel & Andrews, 2012). Considerations were given to the pedagogical requirements of a poster assessment task, the affordances of the learning space and an identification of possible benefits of using Google Sites to create digital posters. Positive feedback from this group (as highlighted in the quotes shown) and subsequent approval from unit coordinators for two of the new first year BSc units meant that the approach was adopted for Semester 1, 2013 with approximately 360 students in each unit.

Indigenous Youth Justice Programs Evaluation

Diversion from the youth justice system is a critical goal for addressing the overrepresentation of Indigenous young people in the criminal justice system. In this report, four programs that were already being implemented by states and territories and identified by them under the National Indigenous Law & Justice Framework as promising practice in diversion are examined. The programs were evaluated, as part of a broader initiative, to determine whether and on what basis they represent good practice (ie are supported by evidence). State and territory governments nominated the programs for evaluation.

Teaching persuasive texts : building a language of evaluation through hedging and moderated intensification

We identify two persuasive writing techniques – hedging and intensification – that pose difficulty for students in the middle years. We use examples of student writing from 3000 work samples collected as part of a larger Australian Research Council Linkage Project, URLearning (2009–2013). To realise the effective power of rhetorical persuasion, students need to be explicitly taught a range of hedging techniques to use to their advantage, and an expanded lexicon that does not rely on intensifiers. Practical teaching tips are provided for teachers.

Model as you do : engaging an S-BPM vendor on process modelling in 3D virtual worlds

Accurate process model elicitation continues to be a time consuming task, requiring skill on the part of the interviewer to extract explicit and tacit process information from the interviewee. Many errors occur in this elicitation stage that would be avoided by better activity recall, more consistent specification methods and greater engagement in the elicitation process by interviewees. Metasonic GmbH has developed a process elicitation tool for their process suite. As part of a research engagement with Metasonic, staff from QUT, Australia have developed a 3D virtual world approach to the same problem, viz. eliciting process models from stakeholders in an intuitive manner. This book chapter tells the story of how QUT staff developed a 3D Virtual World tool for process elicitation, took the outcomes of their research project to Metasonic for evaluation, and finally, Metasonic’s response to the initial proof of concept.

Navigating pathways for academic staff development : implications for institutions and academic ranks

BACKGROUND As engineering schools adopt outcomes - focused learning approaches in response to government expectations and industry requirements of graduates capable of learning and applying knowledge in different contexts, university academics must be capable of developing and delivering programs that meet these requirements. Those academics are increasingly facing challenges in progressing their research and also acquiring different skill sets to meet the learning and teaching requirements. PURPOSE The goal of this study was to identify the types of development and support structures in place for academic staff, especially early career ones, and examine how the type of institution and the rank or role of the staff member affects these structures. DESIGN/METHOD We conducted semi - structured interviews with 21 individuals in a range of positions pertaining to teaching and learning in engineering education. Open coding was used to identify main themes from the guiding questions raised in the interviews and refined to address themes relevant to the development of institutional staff . The interview data was then analysed based on the type of institution and the rank/ role of the participant. RESULTS While development programs that focus on improving teaching and learning are available, the approach on using these types of programs differed based on staff perspective. Fewer academics, regardless of rank/role, had knowledge of support structures related to other areas of scholarship, e.g. disciplinary research, educational research, learning the institutional culture. The type of institution also impacted how they weighted and encouraged multiple forms of scholarship. We found that academic staff holding higher ranking positions, e.g. dean or associate dean, were not only concerned with the success of their respective programs, but also in how to promote other academic staff participation throughout the process. CONCLUSIONS The findings from this stud y extend the premise that developing effective academic staff ultimately leads to more effective institutions and successful graduates and accomplishing this requires staff buy - in at multiple stages of instructional and program development. Staff and administration developing approaches for educational innovation together (Besterfield - Sacre et al., 2014) and getting buy - in from all academic staff to invest in engineering education development will ultimately lead to more successful engineering graduates.

Economic evaluation of adolescent reproductive health education interventions in Chilinh, Vietnam

This research was an economic analysis of two novel health education interventions compared to existing practice for reproductive health among young people in northern Vietnam. The research showed that implementing an educational intervention including school-based and health facility-based components was cost effective for males and females. The findings will assist decision makers in efficient allocation of scarce resources for adolescent health promotion in Vietnam and similar socio-economic contexts in Asia.

Design, construction and evaluation of systems to predict risk in obstetrics

We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy. We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy.

An evaluation of the neocognitron

We describe a sequence of experiments investigating the strengths and limitations of Fukushima's neocognitron as a handwritten digit classifier. Using the results of these experiments as a foundation, we propose and evaluate improvements to Fukushima's original network in an effort to obtain higher recognition performance. The neocognitron's performance is shown to be strongly dependent on the choice of selectivity parameters and we present two methods to adjust these variables. Performance of the network under the more effective of the two new selectivity adjustment techniques suggests that the network fails to exploit the features that distinguish different classes of input data. To avoid this shortcoming, the network's final layer cells were replaced by a nonlinear classifier (a multilayer perceptron) to create a hybrid architecture. Tests of Fukushima's original system and the novel systems proposed in this paper suggest that it may be difficult for the neocognitron to achieve the performance of existing digit classifiers due to its reliance upon the supervisor's choice of selectivity parameters and training data. These findings pertain to Fukushima's implementation of the system and should not be seen as diminishing the practical significance of the concept of hierarchical feature extraction embodied in the neocognitron. © 1997 IEEE.

A critical evaluation of written discharge advice for people with mild traumatic brain injury : what should we be looking for?

Objective: To formally evaluate the written discharge advice for people with mild traumatic brain injury (mTBI). Methods: Eleven publications met the inclusion criteria: (1) intended for adults; (2) ≤two A4 pages; (3) published in English; (4) freely accessible; and (5) currently used (or suitable for use) in Australian hospital emergency departments or similar settings. Two independent raters evaluated the content and style of each publication against established standards. The readability of the publication, the diagnostic term(s) contained in it and a modified Patient Literature Usefulness Index (mPLUI) were also evaluated. Results: The mean content score was 19.18 ± 8.53 (maximum = 31) and the mean style score was 6.8 ± 1.34 (maximum = 8). The mean Flesch-Kincaid reading ease score was 66.42 ± 4.3. The mean mPLUI score was 65.86 ± 14.97 (maximum = 100). Higher scores on these metrics indicate more desirable properties. Over 80% of the publications used mixed diagnostic terminology. One publication scored optimally on two of the four metrics and highly on the others. Discussion: The content, style, readability and usefulness of written mTBI discharge advice was highly variable. The provision of written information to patients with mTBI is advised, but this variability in materials highlights the need for evaluation before distribution. Areas are identified to guide the improvement of written mTBI discharge advice.