When non-significance may be significant : lessons learned from a study of the development, implementation and evaluation of a fleet risk assessment tool

This paper reports on the development and implementation of a self-report risk assessment tool that was developed in an attempt to increase the efficacy of crash prediction within Australian fleet settings. This study forms a part of a broader program of research into work related road safety and identification of driving risk. The first phase of the study involved a series of focus groups being conducted with 217 professional drivers which revealed that the following factors were proposed to influence driving performance: Fatigue, Knowledge of risk, Mood, Impatience and frustration, Speed limits, Experience, Other road users, Passengers, Health, and Culture. The second phase of the study involved piloting the newly developed 38 item Driving Risk Assessment Scale - Work Version (DRAS-WV) with 546 professional drivers. Factor analytic techniques identified a 9 factor solution that was comprised of speeding, aggression, time pressure, distraction, casualness, awareness, maintenance, fatigue and minor damage. Speeding and aggressive driving manoeuvres were identified to be the most frequent aberrant driving behaviours engaged in by the sample. However, a series of logistic regression analyses undertaken to determine the DRAS-WV scale’s ability to predict self-reported crashes revealed limited predictive efficacy e.g., 10% of crashes. This paper outlines proposed reasons for this limited predictive ability of the DRAS-WV as well as provides suggestions regarding the future of research that aims to develop methods to identify “at risk” drivers.

Enhancing opportunities to feed forward ; developing problem solving skills using web based scenarios

Universities often struggle to satisfy students’ need for feedback. This is an area where student satisfaction with courses of study can be low. Yet it is clear that one of the properties of good teaching is giving the highest quality feedback on student work. The term ‘feedback’ though is most commonly associated with summative assessment given by a teacher after work is completed. The student can often be a passive participant in the process. This paper looks at the implementation of a web based interactive scenario completed by students prior to summative assessment. It requires students to participate actively to develop and improve their legal problem solving skills. Traditional delivery of legal education focuses on print and an instructor who conveys the meaning of the written word to students. Today, mixed modes of teaching are often preferred and they can provide enhanced opportunities for feeding forward with greater emphasis on what students do. Web based activities allow for flexible delivery; they are accessible off campus, at a time that suits the student and may be completed by students at their own pace. This paper reports on an online interactive activity which provides valuable formative feedback necessary to allow for successful completion of a final problem solving assignment. It focuses on how the online activity feeds forward and contributes to the development of legal problem solving skills. Introduction to Law is a unit designed and introduced for completion by undergraduate students from faculties other than law but is focused most particularly on students enrolled in the Bachelor of Entertainment Industries degree, a joint initiative of the faculties of Creative Industries, Business and Law at the Queensland University of Technology in Australia. The final (and major) assessment for the unit is an assignment requiring students to explain the legal consequences of particular scenarios. A number of cost effective web based interactive scenarios have been developed to support the unit’s classroom activities. The tool commences with instruction on problem solving method. Students then view the stimulus which is a narrative produced in the form of a music video clip. A series of questions are posed which guide students through the process and they can compare their responses with sample answers provided. The activity clarifies the problem solving method and expectations for the summative assessment and allows students to practise the skill. The paper reports on the approach to teaching and learning taken in the unit including the design process and implementation of the activity. It includes an evaluation of the activity with respect to its effectiveness as a tool to feed forward and reflects on the implications for the teaching of law in higher education.

When digital native meets analogue reality : a case study of ICT skills in first year university students

Recently claims have been made that all universities will in coming decades merge to become just a few mega-institutions offering online degrees to the world. This assumes a degree of literacy with ICT (information and communication technology) amongst potential students, who are often regarded as 'digital natives'. Far from being digital natives, many students have considerable trouble using ICT beyond the ubiquitous Facebook. While some students are computer literate, a substantial proportion lack the skills to prosper under their own devices in an online tertiary education environment. For these students a blended learning experience is needed to develop skills to effectively interact in the virtual environment. This paper presents a case study that specifically examined the ICT capabilities of first-year university students enrolled in the School of Civil Engineering and the Built Environment at Queensland University of Technology (QUT). Empirical data are presented and curriculum strategies articulated to develop ICT skills in university undergraduates.

Curriculum design and higher order skills : challenging assumptions

This paper is based on an Australian Learning & Teaching Council (ALTC) funded evaluation in 13 universities across Australia and New Zealand of the use of Engineers Without Borders (EWB) projects in first-year engineering courses. All of the partner institutions have implemented this innovation differently and comparison of these implementations affords us the opportunity to assemble "a body of carefully gathered data that provides evidence of which approaches work for which students in which learning environments". This study used a mixed-methods data collection approach and a realist analysis. Data was collected by program logic analysis with course co-ordinators, observation of classes, focus groups with students, exit survey of students and interviews with staff as well as scrutiny of relevant course and curriculum documents. Course designers and co-ordinators gave us a range of reasons for using the projects, most of which alluded to their presumed capacity to deliver experience in and learning of higher order thinking skills in areas such as sustainability, ethics, teamwork and communication. For some students, however, the nature of the projects decreased their interest in issues such as ethical development, sustainability and how to work in teams. We also found that the projects provoked different responses from students depending on the nature of the courses in which they were embedded (general introduction, design, communication, or problem-solving courses) and their mode of delivery (lecture, workshop or online).

Peer assisted learning, skills development and Generation Y : a case study of a first year undergraduate law unit

The majority of current first year university students belong to Generation Y. Consequently, research suggests that, in order to more effectively engage them, their particular learning preferences should be acknowledged in the organisation of their learning environments and in the support provided. These preferences are reflected in the Torts Student Peer Mentor Program, which, as part of the undergraduate law degree at the Queensland University of Technology, utilises active learning, structured sessions and teamwork to supplement student understanding of the substantive law of Torts with the development of life-long skills. This article outlines the Program, and its relevance to the learning styles and experiences of Generation Y first year law students transitioning to university, in order to investigate student perceptions of its effectiveness – both generally and, more specifically, in terms of the Program’s capacity to assist students to develop academic and work-related skills.

Pilot evaluation of an adolescent risk and injury prevention programme incorporating curriculum and school connectedness components

School connectedness is an important protective factor for adolescent risk-taking behaviour. This study examined a pilot version of the Skills for Preventing Injury in Youth (SPIY) programme, combining teacher professional development for increasing school connectedness (connectedness component) with a risk and injury prevention curriculum for early adolescents (curriculum component). A process evaluation was conducted on the connectedness component, involving assessments of programme reach, participant receptiveness and initial use, and a preliminary impact evaluation was conducted on the combined connectedness and curriculum programme. The connectedness component was well received by teacher participants, who saw benefits for both themselves and their students. Classroom observation also showed that teachers who received professional development made use of the programme strategies. Grade 8 students who participated in the SPIY programme were less likely to report violent behaviour at six-month follow-up than were control students, and trends also suggested reduced transport injuries. The results of this research support the use of the combined SPIY connectedness and curriculum components in a large-scale effectiveness trial to assess the impact of the programme on students’ connectedness, risk-taking and associated injuries.

Comparison of genomic DNA extraction techniques from whole blood samples : a time, cost and quality evaluation study

Genomic DNA obtained from patient whole blood samples is a key element for genomic research. Advantages and disadvantages, in terms of time-efficiency, cost-effectiveness and laboratory requirements, of procedures available to isolate nucleic acids need to be considered before choosing any particular method. These characteristics have not been fully evaluated for some laboratory techniques, such as the salting out method for DNA extraction, which has been excluded from comparison in different studies published to date. We compared three different protocols (a traditional salting out method, a modified salting out method and a commercially available kit method) to determine the most cost-effective and time-efficient method to extract DNA. We extracted genomic DNA from whole blood samples obtained from breast cancer patient volunteers and compared the results of the product obtained in terms of quantity (concentration of DNA extracted and DNA obtained per ml of blood used) and quality (260/280 ratio and polymerase chain reaction product amplification) of the obtained yield. On average, all three methods showed no statistically significant differences between the final result, but when we accounted for time and cost derived for each method, they showed very significant differences. The modified salting out method resulted in a seven- and twofold reduction in cost compared to the commercial kit and traditional salting out method, respectively and reduced time from 3 days to 1 hour compared to the traditional salting out method. This highlights a modified salting out method as a suitable choice to be used in laboratories and research centres, particularly when dealing with a large number of samples.

Fresh frozen cadavers as a tool to educate student paramedics in procedural skills

Objectives – To describe the development of an educational workshop to develop procedural skills in undergraduate Paramedic students using fresh frozen cadavers and to report the student’s assessment of the program. Methods – A six-hour anatomy based workshop was developed using fresh frozen cadavers to teach a range of airway and invasive procedural skills to second year undergraduate paramedic students. Embedded QUAN (qual) methodology will be utilised to evaluate the student’s satisfaction, perception and quality of teaching as compared to other existing clinical teaching techniques such as high fidelity simulation. Students will be asked to complete an anonymous validated survey (10 questions formulated on a 5 point Likert scale) and provide a qualitative feedback pre and post the six-hour workshop. Results – This is a prospective study planned for September 2013. Low-risk human research ethics are being sought. Teaching evaluation results from the inaugural 2012 workshop (undergraduate and postgraduate Paramedic students) and interim results for 2013 will be presented. Conclusions – Clinical teaching using fresh frozen cadavers thus far has predominately been used in the education of medical and surgical trainees. A number of studies have found them to be effective and in some cases superior to traditional high fidelity simulation teaching strategies. Fresh frozen cadavers are said to provide perfect anatomy, normal tissue consistency and a realistic operative training experience (Lloyd, Maxwell-Armstrong et al. 2011). The authors believe that this study will show that the use of fresh frozen cadavers offers a safe and effective mode to teach procedural skills to student paramedics that will help bridge the skills gap and increase confidence prior to students undertaking such interventions on living patients. A modified training program may be formulated for general practitioners undertaking Emergency Medicine Advanced Rural Skills.

Formative evaluation for improving collaborative planning : a case study at the regional scale

In an earlier paper (Cameron & Johnson 2004) we introduced the idea of formative evaluation (or evaluation for development), the purpose of which is to provide information for improving planning programs and activities. This type of evaluation differs from the two other types: outcome evaluation which aims to judge the success or otherwise of a program; and evaluation for knowledge which seeks to contribute to theoretical work on planning processes and activities. In the earlier paper we also outlined the first stage of formative evaluation in the SEQ 2021 regional planning exercise showing how the process of planning for community engagement was modified in light of the evaluation findings. This current paper details the second stage of formative evaluation in which the collaborative planning component of SEQ 2021 was evaluated, as such it further demonstrates how formative evaluation can be used to improve planning programs. The evaluation findings also provide insights into strategies for more effective collaborative planning. We begin with an overview of collaborative approaches to regional planning, including the SEQ 2021 regional planning program. We then outline formal and informal evaluations of various collaborative regional planning exercises, including the predecessor of SEQ 2021 - SEQ 2001. This sets the scene for discussion of the approach used to evaluate the collaborative component of SEQ 2021. After outlining the main findings from the evaluation and the ways these findings were used to refine the collaborative planning process we conclude with a series of recommendations, relevant not only to SEQ 2021 but to other collaborative planning exercises

Get real : the development, implementation and evaluation of an intervention for primary aged children with High Functioning Autism

Children with Autism Spectrum Disorder experience difficulty in communication and in understanding the social world which can have negative consequences for their relationships, in managing emotions, and generally dealing with the challenges of everyday life. This thesis examines the effectiveness of the Active and Reflective components of the Get REAL program through the assessment of the detailed coding of video-recorded observations and longitudinal quantitative analysis. The aim of Get REAL is to increase the social, emotional, and cognitive learning of children with High Functioning Autism (HFA). Get REAL is a group program designed specifically for use in inclusive primary school settings. The Get REAL program was designed in response to the mixed success of generalisation of learning to new contexts of existing social skills programs. The theoretical foundation of Get REAL is based upon pedagogical theory and learning theory to facilitate transfer of learning, combined with experiential, individualised, evaluative and organisational approaches. This thesis is by publication and consists of four refereed journal papers; 1 accepted for publication and 3 that are under review. Paper 1 describes the development and theoretical basis of the Get REAL program and provides detail of the program structure and learning cycle. The focus of Paper 1 reflects the first question of interest in the thesis which is about the extent to which learning derived from participation in the program can be generalised to other contexts. Participants are 16 children with HFA ranging in age from 8-13 years. Results provided support for the generalisability of learning from Get REAL to home and school evidenced by parent and teacher data collected pre and post participation in Get REAL. Following establishment of the generalisation of learning from Get REAL, Papers 2 and 3 focus on the Active and Reflective components of the program in order to examine how individual and group learning takes place. Participants (N = 12) in the program are video-taped during the Active and Reflective Sessions. Using identical coding protocols of video data, improvements in prosocial behaviour and diminishing of inappropriate behaviours were apparent with the exception of perspective taking. Data also revealed that 2 of the participants had atypical trajectories. An in-depth case study analysis was then conducted with these 2 participants in Paper 4. Data included reports from health care and education professionals within the school and externally (e.g., paediatrician) and identified the multi-faceted nature of care needed for children with comorbid diagnoses and extremely challenging family circumstances as a complex task to effect change. Results of this research support the effectiveness of the Get REAL program in promoting pro social behaviours such as improvements in engaging with others and emotional regulation, and in diminishing unwanted behaviours such as conduct problems. Further, the gains made by the participating children were found to be generalisable beyond Get REAL to home and other school settings. The research contained in the thesis adds to current knowledge about how learning can take place for children with HFA. Results show that an experiential learning framework with a focus on social cognition, together with explicit teaching, scaffolded with video feedback, are key ingredients for the generalisation of social learning to broader contexts.

Development and evaluation of a program to improve clinician and patient communication during a telehealth consultation : C.R.I.S.P. Telehealth

Introduction: The delivery of health care in the 21st century will look like no other in the past. The fast paced technological advances that are being made will need to transition from the information age into clinical practice. The phenomenon of e-Health is the over-arching form of information technology and telehealth is one arm of that phenomenon. The uptake of telehealth both in Australia and overseas, has changed the face of health service delivery to many rural and remote communities for the better, removing what is known as the tyranny of distance. Many studies have evaluated the satisfaction and cost-benefit analysis of telehealth across the organisational aspects as well as the various adaptations of clinical pathways and this is the predominant focus of most studies published to date. However, whilst comments have been made by many researchers about the need to improve and attend to the communication and relationship building aspects of telehealth no studies have examined this further. The aim of this study was to identify the patient and clinician experiences, concerns, behaviours and perceptions of the telehealth interaction and develop a training tool to assist these clinicians to improve their interaction skills. Methods: A mixed methods design combining quantitative (survey analysis and data coding) and qualitative (interview analysis) approaches was adopted. This study utilised four phases to firstly qualitatively explore the needs of clients (patients) and clinicians within a telehealth consultation then designed, developed, piloted and quantitatively and qualitatively evaluated the telehealth communication training program. Qualitative data was collected and analysed during Phase 1 of this study to describe and define the missing 'communication and rapport building' aspects within telehealth. This data was then utilised to develop a self-paced communication training program that enhanced clinicians existing skills, which comprised of Phase 2 of this study to develop the interactive program. Phase 3 included evaluating the training program with 26 clinicians and results were recorded pre and post training, whilst phase 4 was the pilot for future recommendations of this training program using a patient group within a Queensland Health setting at two rural hospitals. Results: Comparisons of pre and post training data on 1) Effective communication styles, 2) Involvement in communication training package, 3) satisfaction pre and post training, and 4) health outcomes pre and post training indicated that there were differences between pre and post training in relation to effective communication style, increased satisfaction and no difference in health outcomes between pre and post training for this patient group. The post training results revealed over half of the participants (N= 17, 65%) were more responsive to non-verbal cues and were better able to reflect and respond to looks of anxiousness and confusion from a 'patient' within a telehealth consultation. It was also found that during post training evaluations, clinicians had enhanced their therapeutic communication with greater detail to their own body postures, eye contact and presentation. There was greater time spent looking at the 'patient' with an increase of 35 second intervals of direct eye contact and less time spent looking down at paperwork which decreased by 20 seconds. Overall 73% of the clinicians were satisfied with the training program and 61% strongly agreed that they recognised areas of their communication that needed improving during a telehealth consultation. For the patient group there was significant difference post training in rapport with a mean score from 42 (SD = 28, n = 27) to 48 (SD = 5.9, n = 24). For communication comfort of the patient group there was a significant difference between the pre and post training scores t(10) = 27.9, p = .002, which meant that overall the patients felt less inhibited whilst talking to the clinicians and more understood. Conclusion: The aim of this study was to explore the characteristics of good patient-clinician communication and unmet training needs for telehealth consultations. The study developed a training program that was specific for telehealth consultations and not dependent on a 'trainer' to deliver the content. In light of the existing literature this is a first of its kind and a valuable contribution to the research on this topic. It was found that the training program was effective in improving the clinician's communication style and increased the satisfaction of patient's within an e-health environment. This study has identified some historical myths that telehealth cannot be part of empathic patient centred care due to its technology tag.

A pre-post evaluation of an ambulatory nutrition support service for malnourished patients post hospital discharge: a pilot study

Introduction Malnutrition is common among hospitalised patients, with poor follow-up of nutrition support post-discharge. Published studies on the efficacy of ambulatory nutrition support (ANS) for malnourished patients post-discharge are scarce. The aims of this study were to evaluate the rate of dietetics follow-up of malnourished patients post-discharge, before (2008) and after (2010) implementation of a new ANS service, and to evaluate nutritional outcomes post-implementation. Materials and Methods Consecutive samples of 261 (2008) and 163 (2010) adult inpatients referred to dietetics and assessed as malnourished using Subjective Global Assessment (SGA) were enrolled. All subjects received inpatient nutrition intervention and dietetic outpatient clinic follow-up appointments. For the 2010 cohort, ANS was initiated to provide telephone follow-up and home visits for patients who failed to attend the outpatient clinic. Subjective Global Assessment, body weight, quality of life (EQ-5D VAS) and handgrip strength were measured at baseline and five months post-discharge. Paired t-test was used to compare pre- and post-intervention results. Results In 2008, only 15% of patients returned for follow-up with a dietitian within four months post-discharge. After implementation of ANS in 2010, the follow-up rate was 100%. Mean weight improved from 44.0 ± 8.5kg to 46.3 ± 9.6kg, EQ-5D VAS from 61.2 ± 19.8 to 71.6 ± 17.4 and handgrip strength from 15.1 ± 7.1 kg force to 17.5 ± 8.5 kg force; p<0.001 for all. Seventy-four percent of patients improved in SGA score. Conclusion Ambulatory nutrition support resulted in significant improvements in follow-up rate, nutritional status and quality of life of malnourished patients post-discharge.

Development and formative evaluation of a motorcycle rider training intervention to address risk taking

The need to address on-road motorcycle safety in Australia is important due to the disproportionately high percentage of riders and pillions killed and injured each year. One approach to preventing motorcycle-related injury is through training and education. However, motorcycle rider training lacks empirical support as an effective road safety countermeasure to reduce crash involvement. Previous reviews have highlighted that risk-taking is a contributing factor in many motorcycle crashes, rather than merely a lack of vehicle-control skills (Haworth & Mulvihill, 2005; Jonah, Dawson & Bragg, 1982; Watson et al, 1996). Hence, though the basic vehicle-handling skills and knowledge of road rules that are taught in most traditional motorcycle licence training programs may be seen as an essential condition of safe riding, they do not appear to be sufficient in terms of crash reduction. With this in mind there is considerable scope for the improvement of program focus and content for rider training and education. This program of research examined an existing traditional pre-licence motorcycle rider training program and formatively evaluated the addition of a new classroom-based module to address risky riding; the Three Steps to Safer Riding program. The pilot program was delivered in the real world context of the Q-Ride motorcycle licensing system in the state of Queensland, Australia. Three studies were conducted as part of the program of research: Study 1, a qualitative investigation of delivery practices and student learning needs in an existing rider training course; Study 2, an investigation of the extent to which an existing motorcycle rider training course addressed risky riding attitudes and motives; and Study 3, a formative evaluation of the new program. A literature review as well as the investigation of learning needs for motorcyclists in Study 1 aimed to inform the initial planning and development of the Three Steps to Safer Riding program. Findings from Study 1 suggested that the training delivery protocols used by the industry partner training organisation were consistent with a learner-centred approach and largely met the learning needs of trainee riders. However, it also found that information from the course needs to be reinforced by on-road experiences for some riders once licensed and that personal meaning for training information was not fully gained until some riding experience had been obtained. While this research informed the planning and development of the new program, a project team of academics and industry experts were responsible for the formulation of the final program. Study 2 and Study 3 were conducted for the purpose of formative evaluation and program refinement. Study 2 served primarily as a trial to test research protocols and data collection methods with the industry partner organisation and, importantly, also served to gather comparison data for the pilot program which was implemented with the same rider training organisation. Findings from Study 2 suggested that the existing training program of the partner organisation generally had a positive (albeit small) effect on safety in terms of influencing attitudes to risk taking, the propensity for thrill seeking, and intentions to engage in future risky riding. However, maintenance of these effects over time and the effects on riding behaviour remain unclear due to a low response rate upon follow-up 24 months after licensing. Study 3 was a formative evaluation of the new pilot program to establish program effects and possible areas for improvement. Study 3a examined the short term effects of the intervention pilot on psychosocial factors underpinning risky riding compared to the effects of the standard traditional training program (examined in Study 2). It showed that the course which included the Three Steps to Safer Riding program elicited significantly greater positive attitude change towards road safety than the existing standard licensing course. This effect was found immediately following training, and mean scores for attitudes towards safety were also maintained at the 12 month follow-up. The pilot program also had an immediate effect on other key variables such as risky riding intentions and the propensity for thrill seeking, although not significantly greater than the traditional standard training. A low response rate at the 12 month follow-up unfortunately prevented any firm conclusions being drawn regarding the impact of the pilot program on self-reported risky riding once licensed. Study 3a further showed that the use of intermediate outcomes such as self-reported attitudes and intentions for evaluation purposes provides insights into the mechanisms underpinning risky riding that can be changed by education and training. A multifaceted process evaluation conducted in Study 3b confirmed that the intervention pilot was largely delivered as designed, with course participants also rating most aspects of training delivery highly. The complete program of research contributed to the overall body of knowledge relating to motorcycle rider training, with some potential implications for policy in the area of motorcycle rider licensing. A key finding of the research was that psychosocial influences on risky riding can be shaped by structured education that focuses on awareness raising at a personal level and provides strategies to manage future riding situations. However, the formative evaluation was mainly designed to identify areas of improvement for the Three Steps to Safer Riding program and found several areas of potential refinement to improve future efficacy of the program. This included aspects of program content, program delivery, resource development, and measurement tools. The planned future follow-up of program participants' official crash and traffic offence records over time may lend further support for the application of the program within licensing systems. The findings reported in this thesis offer an initial indication that the Three Steps to Safer Riding is a useful resource to accompany skills-based training programs.