Data analysis skills for teachers

Although accountability in the form of high stakes testing is in favour in the contemporary Australian educational context, this practice remains a highly contested source of debate. Proponents for high stakes tests claim that higher standards in teaching and learning result from their implementation, whereas others believe that this type of testing regime is not required and may even in fact be counterproductive. Regardless of what side of the debate you sit on, the reality is that at present, high stakes testing appears to be here to stay. It could therefore be argued it is essential that teachers understand accountability and possess the specific skills to interpret and use test data beneficially.

Enhancing opportunities to feed forward ; developing problem solving skills using web based scenarios

Universities often struggle to satisfy students’ need for feedback. This is an area where student satisfaction with courses of study can be low. Yet it is clear that one of the properties of good teaching is giving the highest quality feedback on student work. The term ‘feedback’ though is most commonly associated with summative assessment given by a teacher after work is completed. The student can often be a passive participant in the process. This paper looks at the implementation of a web based interactive scenario completed by students prior to summative assessment. It requires students to participate actively to develop and improve their legal problem solving skills. Traditional delivery of legal education focuses on print and an instructor who conveys the meaning of the written word to students. Today, mixed modes of teaching are often preferred and they can provide enhanced opportunities for feeding forward with greater emphasis on what students do. Web based activities allow for flexible delivery; they are accessible off campus, at a time that suits the student and may be completed by students at their own pace. This paper reports on an online interactive activity which provides valuable formative feedback necessary to allow for successful completion of a final problem solving assignment. It focuses on how the online activity feeds forward and contributes to the development of legal problem solving skills. Introduction to Law is a unit designed and introduced for completion by undergraduate students from faculties other than law but is focused most particularly on students enrolled in the Bachelor of Entertainment Industries degree, a joint initiative of the faculties of Creative Industries, Business and Law at the Queensland University of Technology in Australia. The final (and major) assessment for the unit is an assignment requiring students to explain the legal consequences of particular scenarios. A number of cost effective web based interactive scenarios have been developed to support the unit’s classroom activities. The tool commences with instruction on problem solving method. Students then view the stimulus which is a narrative produced in the form of a music video clip. A series of questions are posed which guide students through the process and they can compare their responses with sample answers provided. The activity clarifies the problem solving method and expectations for the summative assessment and allows students to practise the skill. The paper reports on the approach to teaching and learning taken in the unit including the design process and implementation of the activity. It includes an evaluation of the activity with respect to its effectiveness as a tool to feed forward and reflects on the implications for the teaching of law in higher education.

Using the Donabedian framework to examine the quality and safety of nursing service innovation

Aims and objectives To evaluate the safety and quality of nurse practitioner service using the audit framework of Structure,Process and Outcome. Background Health service and workforce reform are on the agenda of governments and other service providers seeking to contain healthcare costs whilst providing safe and effective health care to communities. The nurse practitioner service is one health workforce innovation that has been adopted globally to improve timely access to clinical care, but there is scant literature reporting evaluation of the quality of this service innovation. Design. A mixed-methods design within the Donabedian evaluation framework was used. Methods The Donabedian framework was used to evaluate the Structure, Process and Outcome of nurse practitioner service. A range of data collection approaches was used, including stakeholder survey (n=36), in-depth interviews (11 patients and 13 nurse practitioners) and health records data on service processes. Results The study identified that adequate and detailed preparation of Structure and Process is essential for the successful implementation of a service innovation. The multidisciplinary team was accepting of the addition of nurse practitioner service, and nurse practitioner clinical care was shown to be effective, satisfactory and safe from the perspective of the clinician stakeholders and patients. Conclusions This study demonstrated that the Donabedian framework of Structure, Process and Outcome evaluation is a valuable and validated approach to examine the safety and quality of a service innovation. Furthermore, in this study, specific Structure elements were shown to influence the quality of service processes further validating the framework and the interdependence of the Structure, Process and Outcome components. Relevance to clinical practice Understanding the structure and process requirements for establishing nursing service innovation lays the foundation for safe, effective and patient-centred clinical care.

Development and interrater reliability testing of a telephone interview training programme for Australian nurse interviewers

Summary Background The final phase of a three phase study analysing the implementation and impact of the nurse practitioner role in Australia (the Australian Nurse Practitioner Project or AUSPRAC) was undertaken in 2009, requiring nurse telephone interviewers to gather information about health outcomes directly from patients and their treating nurse practitioners. A team of several registered nurses was recruited and trained as telephone interviewers. The aim of this paper is to report on development and evaluation of the training process for telephone interviewers. Methods The training process involved planning the content and methods to be used in the training session; delivering the session; testing skills and understanding of interviewers post-training; collecting and analysing data to determine the degree to which the training process was successful in meeting objectives and post-training follow-up. All aspects of the training process were informed by established educational principles. Results Interrater reliability between interviewers was high for well-validated sections of the survey instrument resulting in 100% agreement between interviewers. Other sections with unvalidated questions showed lower agreement (between 75% and 90%). Overall the agreement between interviewers was 92%. Each interviewer was also measured against a specifically developed master script or gold standard and for this each interviewer achieved a percentage of correct answers of 94.7% or better. This equated to a Kappa value of 0.92 or better. Conclusion The telephone interviewer training process was very effective and achieved high interrater reliability. We argue that the high reliability was due to the use of well validated instruments and the carefully planned programme based on established educational principles. There is limited published literature on how to successfully operationalise educational principles and tailor them for specific research studies; this report addresses this knowledge gap.

An integrated educational model for continuing nurse education

Aim This paper reports on the development and evaluation of an integrated clinical learning model to inform ongoing education for surgical nurses. The research aim was to evaluate the effectiveness of implementing a Respiratory Skills Update (ReSKU) education program, in the context of organisational utility, on improving surgical nurses' practice in the area of respiratory assessment. Background Continuous development and integration of technological innovations and research in the healthcare environment mandate the need for continuing education for nurses. Despite an increased worldwide emphasis on this, there is scant empirical evidence of program effectiveness. Methods A quasi experimental pre test, post test non–equivalent control group design evaluated the impact of the ReSKU program on surgical nurses' clinical practice. The 2008 study was conducted in a 400 bed regional referral public hospital and was consistent with contemporary educational approaches using multi-modal, interactive teaching strategies. Findings The study demonstrated statistically significant differences between groups regarding reported use of respiratory skills, three months after ReSKU program attendance. Between group data analysis indicated that the intervention group's reported beliefs and attitudes pertaining to subscale descriptors showed statistically significant differences in three of the six subscales. Conclusion The construct of critical thinking in the clinical context, combined with clinical reasoning and purposeful reflection, was a powerful educational strategy to enhance competency and capability in clinicians.

Curriculum design and higher order skills : challenging assumptions

This paper is based on an Australian Learning & Teaching Council (ALTC) funded evaluation in 13 universities across Australia and New Zealand of the use of Engineers Without Borders (EWB) projects in first-year engineering courses. All of the partner institutions have implemented this innovation differently and comparison of these implementations affords us the opportunity to assemble "a body of carefully gathered data that provides evidence of which approaches work for which students in which learning environments". This study used a mixed-methods data collection approach and a realist analysis. Data was collected by program logic analysis with course co-ordinators, observation of classes, focus groups with students, exit survey of students and interviews with staff as well as scrutiny of relevant course and curriculum documents. Course designers and co-ordinators gave us a range of reasons for using the projects, most of which alluded to their presumed capacity to deliver experience in and learning of higher order thinking skills in areas such as sustainability, ethics, teamwork and communication. For some students, however, the nature of the projects decreased their interest in issues such as ethical development, sustainability and how to work in teams. We also found that the projects provoked different responses from students depending on the nature of the courses in which they were embedded (general introduction, design, communication, or problem-solving courses) and their mode of delivery (lecture, workshop or online).

Tracking new directions in the evaluation of postgraduate supervision

This article focuses on the evaluation of thesis supervision and highlights the vast range of problem areas presently documented as significant areas of concern for both graduate students and their supervisors. Additionally, the authors report on a study completed in 1995, which surveyed all Australian universities about current evaluative practices in postgraduate supervision. The conclusion of this study was that the conduct of such evaluations appears to be minimal and is primarily designed to obtain an indicator of the general “health” of a university's postgraduate supervision rather than to foster improved supervisory practices. As part of the same study, the authors conducted University faculty-based student and supervisor focus groups. Key issues emerging from these focus groups were: (1) the importance of relational aspects of supervision as the student communicates over the long term with one or more supervisors; (2) the importance of systematic feedback, monitoring, and evaluation to the supervisory process; and (3) the lack of strategies to facilitate this evaluative feedback process. On the basis of these findings, the authors designed evaluative strategies to facilitate regular ongoing feedback between students and supervisors.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

Skill development and social inclusion via peer assisted learning in first-year higher education : an evaluation of student experiences across two law units

Because professions seek graduates who can 'collaborate, share skills and knowledge, and communicate' (Kruck and Reif, 2001, p 37), it is important that university graduates are not equipped solely with the content knowledge of their discipline, but also with prospective employment skills. Furthermore, when students 'interact more in positive ways with their teachers and peers, they gain more in terms of essential skills and competencies, such as critical thinking, problem~solving [and] effective communication' (NSSE, 2000, p 2)./n this way, peer assisted fellowing has the potential to enhance students' professional development, and provide the social inclusion and engagement necessary for effective learning. This session describes two peer assisted learning models embedded within first year QUT Faculty of Law units. Through a partnership between teaching staff, student mentors and mentees, the models aim to facilitate student socialisation whilst supplementing understanding of substantive law with the development of academic and work·related skills. Mentor and mentee perceptions, and program implications, are considered.

Using ‘think aloud’ as a strategy for improving clinical reasoning in high fidelity case-based simulation for undergraduate nursing students.

Theme Paper for Curriculum innovation and enhancement theme AIM: This paper reports on a research project that trialled an educational strategy implemented in an undergraduate nursing curriculum. The project aimed to explore the effectiveness of ‘think aloud’ as a strategy for improving clinical reasoning for students in simulated clinical settings. BACKGROUND: Nurses are required to apply and utilise critical thinking skills to enable clinical reasoning and problem solving in the clinical setting (Lasater, 2007). Nursing students are expected to develop and display clinical reasoning skills in practice, but may struggle articulating reasons behind decisions about patient care. The ‘think aloud’ approach is an innovative learning/teaching method which can create an environment suitable for developing clinical reasoning skills in students (Banning, 2008, Lee and Ryan-Wenger, 1997). This project used the ‘think aloud’ strategy within a simulation context to provide a safe learning environment in which third year students were assisted to uncover cognitive approaches to assist in making effective patient care decisions, and improve their confidence, clinical reasoning and active critical reflection about their practice. MEHODS: In semester 2 2011 at QUT, third year nursing students undertook high fidelity simulation (some for the first time), commencing in September of 2011. There were two cohorts for strategy implementation (group 1= used think aloud as a strategy within the simulation, group 2= no specific strategy outside of nursing assessment frameworks used by all students) in relation to problem solving patient needs. The think aloud strategy was described to students in their pre-simulation briefing and allowed time for clarification of this strategy. All other aspects of the simulations remained the same, (resources, suggested nursing assessment frameworks, simulation session duration, size of simulation teams, preparatory materials). Ethics approval has been obtained for this project. RESULTS: Results of a qualitative analysis (in progress- will be completed by March 2012) of student and facilitator reports on students’ ability to meet the learning objectives of solving patient problems using clinical reasoning and experience with the ‘think aloud’ method will be presented. A comparison of clinical reasoning learning outcomes between the two groups will determine the effect on clinical reasoning for students responding to patient problems. CONCLUSIONS: In an environment of increasingly constrained clinical placement opportunities, exploration of alternate strategies to improve critical thinking skills and develop clinical reasoning and problem solving for nursing students is imperative in preparing nurses to respond to changing patient needs.

Pilot evaluation of an adolescent risk and injury prevention programme incorporating curriculum and school connectedness components

School connectedness is an important protective factor for adolescent risk-taking behaviour. This study examined a pilot version of the Skills for Preventing Injury in Youth (SPIY) programme, combining teacher professional development for increasing school connectedness (connectedness component) with a risk and injury prevention curriculum for early adolescents (curriculum component). A process evaluation was conducted on the connectedness component, involving assessments of programme reach, participant receptiveness and initial use, and a preliminary impact evaluation was conducted on the combined connectedness and curriculum programme. The connectedness component was well received by teacher participants, who saw benefits for both themselves and their students. Classroom observation also showed that teachers who received professional development made use of the programme strategies. Grade 8 students who participated in the SPIY programme were less likely to report violent behaviour at six-month follow-up than were control students, and trends also suggested reduced transport injuries. The results of this research support the use of the combined SPIY connectedness and curriculum components in a large-scale effectiveness trial to assess the impact of the programme on students’ connectedness, risk-taking and associated injuries.