Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.

Successful outcomes in vocational education and training courses : how pedagogy and expectations influence achievement

This paper reports on a four year Australian Research Council funded Linkage Project titled Skilling Indigenous Queensland, conducted in regional areas of Queensland, Australia from 2009 to 2013. The project sought to investigate vocational education, training (VET) and teaching, Indigenous learners’ needs, employer cultural and expectations and community culture and expectations to identify best practice in numeracy teaching for Indigenous VET learners. Specifically it focused on ways to enhance the teaching and learning of courses and the associated mathematics in such courses to benefit learners and increase their future opportunities of employment. To date thirty-nine teachers/trainers/teacher aides and two hundred and thirty-one students consented to participate in the project. Nine VET courses were nominated to be the focus on the study. This paper focuses on questionnaire and interview responses from four trainers, two teacher aides and six students. In recent years a considerable amount of funding has been allocated to increasing Indigenous Peoples’ participation in education and employment. This increased funding is predicated on the assumption that it will make a difference and contribute to closing the education gap between Indigenous and non-Indigenous Australians (Council of Australia Governments, 2009). The central tenet is that access to education for Indigenous People will create substantial social and economic benefits for regional and remote Indigenous People. The project’s aim is to address some of the issues associated with the gap. To achieve the aims, the project adopted a mixed methods design aimed at benefitting research participants and included: participatory collaborative action research (Kemmis & McTaggart, 1988) and, community research (Smith, 1999). Participatory collaborative action research refers to a is a “collective, self-reflective enquiry undertaken by participants in social situations in order to improve the rationality and justice of their own social and educational practices” (Kemmis et al., 1988, p. 5). Community research is described as an approach that “conveys a much more intimate, human and self-defined space” (p. 127). Community research relies on and validates the community’s own definitions. As the project is informed by the social at a community level, it is described as “community action research or emancipatory research” (Smith, 1999, p. 127). It seeks to demonstrate benefit to the community, making positive differences in the lives of Indigenous People and communities. The data collection techniques included survey questionnaires, video recording of teaching and learning processes, teacher reflective video analysis of teaching, observations, semi-structured interviews and student numeracy testing. As a result of these processes, the findings indicate that VET course teachers work hard to adopt contextualising strategies to their teaching, however this process is not always straight forward because of the perceptions of how mathematics has been taught and learned historically. Further teachers, trainers and students have high expectations of one another with the view to successful outcomes from the courses.

Towards a model for student selection in clinical psychology

Background Selection of candidates for clinical psychology programmes is arguably the most important decision made in determining the clinical psychology workforce. However, there are few models to inform the development of selection tools to support selection procedures. The study, using a factor analytic structure, has operationalised the model predicting applicants' capabilities. Method Eighty-eight clinical applicants for entry into a postgraduate clinical psychology programme were assessed on a series of tasks measuring eight capabilities: guided reflection, communication skills, ethical decision making, writing, conceptual reasoning, empathy, and awareness of mind and self-observation. Results Factor analysis revealed three capabilities: labelled “awareness” accounting for 35.71% of variance; “reflection” accounting for 20.56%; and “reasoning” accounting for 18.24% of variance. Fourth year grade point average (GPA) did not correlate with performance on any of the selection capabilities other than a weak correlation with performance on the ethics capability. Conclusions Eight selection capabilities are identified for the selection of candidates independent of GPA. While the model is tentative, it is hoped that the findings will stimulate the development and validation of assessment procedures with good predictive validity which will benefit the training of clinical psychologists and, ultimately, effective service delivery.

Fresh frozen cadavers as a tool to educate student paramedics in procedural skills

Objectives – To describe the development of an educational workshop to develop procedural skills in undergraduate Paramedic students using fresh frozen cadavers and to report the student’s assessment of the program. Methods – A six-hour anatomy based workshop was developed using fresh frozen cadavers to teach a range of airway and invasive procedural skills to second year undergraduate paramedic students. Embedded QUAN (qual) methodology will be utilised to evaluate the student’s satisfaction, perception and quality of teaching as compared to other existing clinical teaching techniques such as high fidelity simulation. Students will be asked to complete an anonymous validated survey (10 questions formulated on a 5 point Likert scale) and provide a qualitative feedback pre and post the six-hour workshop. Results – This is a prospective study planned for September 2013. Low-risk human research ethics are being sought. Teaching evaluation results from the inaugural 2012 workshop (undergraduate and postgraduate Paramedic students) and interim results for 2013 will be presented. Conclusions – Clinical teaching using fresh frozen cadavers thus far has predominately been used in the education of medical and surgical trainees. A number of studies have found them to be effective and in some cases superior to traditional high fidelity simulation teaching strategies. Fresh frozen cadavers are said to provide perfect anatomy, normal tissue consistency and a realistic operative training experience (Lloyd, Maxwell-Armstrong et al. 2011). The authors believe that this study will show that the use of fresh frozen cadavers offers a safe and effective mode to teach procedural skills to student paramedics that will help bridge the skills gap and increase confidence prior to students undertaking such interventions on living patients. A modified training program may be formulated for general practitioners undertaking Emergency Medicine Advanced Rural Skills.

Development and formative evaluation of a motorcycle rider training intervention to address risk taking

The need to address on-road motorcycle safety in Australia is important due to the disproportionately high percentage of riders and pillions killed and injured each year. One approach to preventing motorcycle-related injury is through training and education. However, motorcycle rider training lacks empirical support as an effective road safety countermeasure to reduce crash involvement. Previous reviews have highlighted that risk-taking is a contributing factor in many motorcycle crashes, rather than merely a lack of vehicle-control skills (Haworth & Mulvihill, 2005; Jonah, Dawson & Bragg, 1982; Watson et al, 1996). Hence, though the basic vehicle-handling skills and knowledge of road rules that are taught in most traditional motorcycle licence training programs may be seen as an essential condition of safe riding, they do not appear to be sufficient in terms of crash reduction. With this in mind there is considerable scope for the improvement of program focus and content for rider training and education. This program of research examined an existing traditional pre-licence motorcycle rider training program and formatively evaluated the addition of a new classroom-based module to address risky riding; the Three Steps to Safer Riding program. The pilot program was delivered in the real world context of the Q-Ride motorcycle licensing system in the state of Queensland, Australia. Three studies were conducted as part of the program of research: Study 1, a qualitative investigation of delivery practices and student learning needs in an existing rider training course; Study 2, an investigation of the extent to which an existing motorcycle rider training course addressed risky riding attitudes and motives; and Study 3, a formative evaluation of the new program. A literature review as well as the investigation of learning needs for motorcyclists in Study 1 aimed to inform the initial planning and development of the Three Steps to Safer Riding program. Findings from Study 1 suggested that the training delivery protocols used by the industry partner training organisation were consistent with a learner-centred approach and largely met the learning needs of trainee riders. However, it also found that information from the course needs to be reinforced by on-road experiences for some riders once licensed and that personal meaning for training information was not fully gained until some riding experience had been obtained. While this research informed the planning and development of the new program, a project team of academics and industry experts were responsible for the formulation of the final program. Study 2 and Study 3 were conducted for the purpose of formative evaluation and program refinement. Study 2 served primarily as a trial to test research protocols and data collection methods with the industry partner organisation and, importantly, also served to gather comparison data for the pilot program which was implemented with the same rider training organisation. Findings from Study 2 suggested that the existing training program of the partner organisation generally had a positive (albeit small) effect on safety in terms of influencing attitudes to risk taking, the propensity for thrill seeking, and intentions to engage in future risky riding. However, maintenance of these effects over time and the effects on riding behaviour remain unclear due to a low response rate upon follow-up 24 months after licensing. Study 3 was a formative evaluation of the new pilot program to establish program effects and possible areas for improvement. Study 3a examined the short term effects of the intervention pilot on psychosocial factors underpinning risky riding compared to the effects of the standard traditional training program (examined in Study 2). It showed that the course which included the Three Steps to Safer Riding program elicited significantly greater positive attitude change towards road safety than the existing standard licensing course. This effect was found immediately following training, and mean scores for attitudes towards safety were also maintained at the 12 month follow-up. The pilot program also had an immediate effect on other key variables such as risky riding intentions and the propensity for thrill seeking, although not significantly greater than the traditional standard training. A low response rate at the 12 month follow-up unfortunately prevented any firm conclusions being drawn regarding the impact of the pilot program on self-reported risky riding once licensed. Study 3a further showed that the use of intermediate outcomes such as self-reported attitudes and intentions for evaluation purposes provides insights into the mechanisms underpinning risky riding that can be changed by education and training. A multifaceted process evaluation conducted in Study 3b confirmed that the intervention pilot was largely delivered as designed, with course participants also rating most aspects of training delivery highly. The complete program of research contributed to the overall body of knowledge relating to motorcycle rider training, with some potential implications for policy in the area of motorcycle rider licensing. A key finding of the research was that psychosocial influences on risky riding can be shaped by structured education that focuses on awareness raising at a personal level and provides strategies to manage future riding situations. However, the formative evaluation was mainly designed to identify areas of improvement for the Three Steps to Safer Riding program and found several areas of potential refinement to improve future efficacy of the program. This included aspects of program content, program delivery, resource development, and measurement tools. The planned future follow-up of program participants' official crash and traffic offence records over time may lend further support for the application of the program within licensing systems. The findings reported in this thesis offer an initial indication that the Three Steps to Safer Riding is a useful resource to accompany skills-based training programs.

Scaffolding techniques : a teacher training for cooperative learning in Thailand primary education

Using cooperative learning in classrooms promotes academic achievement, communication skills, problem-solving, social skills and student motivation. Yet it is reported that cooperative learning as a Western educational concept may be ineffective in Asian cultural contexts. The study aims to investigate the utilisation of scaffolding techniques for cooperative learning in Thailand primary mathematics classes. A teacher training program was designed to foster Thai primary school teachers’ cooperative learning implementation. Two teachers participated in this experimental program for one and a half weeks and then implemented cooperative learning strategies in their mathematics classes for six weeks. The data collected from teacher interviews and classroom observations indicates that the difficulty or failure of implementing cooperative learning in Thailand education may not be directly derived from cultural differences. Instead, it does indicate that Thai culture can be constructively merged with cooperative learning through a teacher training program and practices of scaffolding techniques.

Evaluating the impact of high risk foot training on undergraduate podiatry students

Background Diabetes is the leading cause of high risk foot (HRF) complications, admissions and lower limb amputation. Best practice training of podiatrists is known to have a beneficial impact on such outcomes; however, there has been a paucity of studies into undergraduate diabetes podiatry training. The primary aim of this paper was to investigate the changes in final year podiatry students’ confidence, knowledge and clinical practice in the management of HRF complications. Methods This was a prospective longitudinal study of final year podiatry students (n=25) at the Queensland University of Technology. All participants throughout 2011 undertook an intervention of a series of “hands on” HRF workshops, on-campus clinics and external clinical rotations. Outcome measures included customised confidence and knowledge surveys in HRF management across four time points. A timed simulated case scenario was used to evaluate changes in clinical practice at two time points. Friedman and Wilcoxon Signed Rank Tests were used to calculate differences between time points Results Overall improvements between the first and last time points were demonstrated in 20/21 confidence items (p<0.001), 12/27 clinical practice items (p<0.05) and 3/12 knowledge items (p<0.001). Although 8/12 knowledge items recorded high baseline scores of over 80%. Conclusions Overall, it appears student confidence and clinical practice improved with the introduction of designated HRF activities, whilst knowledge remained high. This suggests “hands on” practice and not didactic lectures improve students’ clinical confidence and practice. Results from the 2012 student cohort will also be presented at this conference.

A 3D virtual medical imaging suite : bridging the academic-clinical training boundary

Aims The Medical Imaging Training Immersive Environment (MITIE) system is a recently developed virtual reality (VR) platform that allows students to practice a range of medical imaging techniques. The aim of this pilot study was to harvest user feedback about the educational value of the application and inform future pedagogical development. This presentation explores the use of this technology for skills training and blurring the boundaries between academic learning and clinical skills training. Background MITIE is a 3D VR environment that allows students to manipulate a patient and radiographic equipment in order to produce a VR-generated image for comparison with a gold standard. As with VR initiatives in other health disciplines (1-6) the software mimics clinical practice as much as possible and uses 3D technology to enhance immersion and realism. The software was developed by the Medical Imaging Course Team at a provider University with funding from a Health Workforce Australia “Simulated Learning Environments” grant. Methods Over 80 students undertaking the Bachelor of Medical Imaging Course were randomised to receive practical experience with either MITIE or radiographic equipment in the medical radiation laboratory. Student feedback about the educational value of the software was collected and performance with an assessed setup was measured for both groups for comparison. Ethical approval for the project was provided by the university ethics panel. Results This presentation provides qualitative analysis of student perceptions relating to satisfaction, usability and educational value as well as comparative quantitative performance data. Students reported high levels of satisfaction and both feedback and assessment results confirmed the application’s significance as a pre-clinical training tool. There was a clear emerging theme that MITIE could be a useful learning tool that students could access to consolidate their clinical learning, either during their academic timetables or their clinical placement. Conclusion Student feedback and performance data indicate that MITIE has a valuable role to play in the clinical skills training for medical imaging students both in the academic and the clinical environment. Future work will establish a framework for an appropriate supporting pedagogy that can cross the boundary between the two environments. This project was possible due to funding made available by Health Workforce Australia.

Successful outcomes in vocational education and training courses and mathematics: how pedagogy and expectations influence achievement

This paper reports on a four year Australian Research Council funded Linkage Project titled Skilling Indigenous Queensland, conducted in regional areas of Queensland, Australia from 2009 to 2013. The project sought to investigate Vocational Education and Training (VET) and teaching, Indigenous learners’ needs, employer culture and expectations and community culture and expectations to identify best practice in numeracy teaching for Indigenous VET learners. Specifically it focused on ways to enhance the teaching and learning of courses and the associated mathematics in such courses to benefit learners and increase their future opportunities of employment. To date thirty - nine teachers/trainers/teacher aides and two hundred and thirty - one students consented to participate in the project. Nine VET courses offered in schools and Technical and Further Education Institutes (TAFE) were nominated to be the focus on the study. This paper focuses on student questionnaire responses and interview responses from teachers/trainers one high school principal and five students as a result of these processes, the findings indicated that VET course teachers work hard to adopt contextualising strategies to their teaching; however this process is not always straight forward because of the perceptions of how mathematics has been taught and learned by trainers and teachers. Further teachers, trainers and students have high expectations of one another with the view to successful outcomes from the courses.

Innovations in dietetic training in foodservice management : can early exposure to practice enhance final year competency development?

Student engagement in the delivery of theoretical course materials is a current challenge in the tertiary sector including for dietetic training. In 2011 with the creation of a new nutritionist position for Queensland Meals on Wheels (QMOW), a service learning approach to support this organisation was used with third year dietetic students undertaking two days of structured activities at various QMOW sites in South East Queensland, aligned with coursework in Foodservice Management (FSM). This cohort of students was then followed in their final year post successful completion of five weeks professional practice in FSM to see if this experience supported readiness for placement and competency development. Evaluation was undertaken of eligible students (n = 50) via paper based survey (response rate 94%) with all participating in targeted focus groups. Findings showed that students acknowledged the QMOW experience (on reflection 14 months later) providing opportunity for participation and/or observation in 5 of 12 FSM areas taught in third year, including food safety, meal production, assembly, delivery and dishwashing. Over half the students identified good exposure to FSM competency areas during the QMOW experience, with 83% satisfied with their competency exposure and subsequent practice during final year placements. A consistent theme emerged from focus groups supporting inclusion of practical opportunities with the theoretical component of the FSM subject to highlight relevance to learning. These findings highlight the importance of such teaching initiatives to met student learning preferences, linking theory with practice and supporting competency development in the final year of training programs.

Pushing cultural competence boundaries in nutrition and dietetics training

A health workforce ready for safe practice is a government priority, and particularly critical to support indigenous communities closing ‘the gap’. Increased pressure exists on dietetic training programs for quality placements, with fewer opportunities for immersion in Aboriginal and Torres Strait Islander communities to demonstrate cultural competence. In 2012, Queensland University of Technology established a partnership with Apunipima Cape York Health Council with 56 weeks of dietetic placement for 8 students provided to achieve these aims. Clinical practice in Community Public Health Nutrition (CPHN) was structured in a standard 6 week placement, with Individual Case Management (ICM) and Foodservice Management (FSM) integrated across 8 weeks (4 each), with an additional 2 weeks ICM prior in a metropolitan indigenous health service. Students transitioned from urban to rural then remote sites, with new web-based technologies used for support. Strong learning opportunities were provided, with CPHN projects in antenatal and child health, FSM on standardisation of procedures in a 22 bed health facility, and ICM exposing students to a variety of cases via hospital in/outpatients, general clinics and remote community outreach. Supervisor focus group evaluation was positive, with CPHN and FSM enhancing capacity of service. Student focus group evaluation revealed placements exceeded expectations, with rating high, and strong confidence in cultural competence described. Students debriefed final and third year cohorts on their experiences, with increased awareness and enthusiasm for work with indigenous communities indicated by groups. With the success of this partnership, placements are continuing 2013, and new boundaries in dietetic training established.

Asian international student attrition among Australian universities : are wellness programs the answer?

Education is one of Australia’s largest service-based exports. International students comprise approximately 24% of enrolments at Australian universities (Sawir, Marginson, Deumert, Nyland, & Ramia, 2008); and approximately 80% of these students are from the Asian region (Australian Federal Government, 2012). The financial cost of international student attrition for universities is significant. The Australian Federal Government Department of Education, Science and Training reports the attrition rates for first-year international undergraduate students ranged between 4% and 22.5% across all Australian Universities (2013). Academic, psychological, and sociocultural adjustments to a new environment can be challenging for international students. This process manifests from various stressors such as communication difficulties, adjustment to a new teaching style, new cultural norms and pressure on academic performance. These stressors result in an often overwhelming attempt to integrate and function effectively, and can consequently affect a student’s ability to meet academic requirements. The relationship between a student’s ability to successfully complete a higher education program is consistently related to a range of academic and non-academic factors. The role of specific Australian higher education institutions is vital in facilitating the continued education of Asian International students. Initiatives targeting an enhancement of modifiable lifestyle factors may have the potential to enhance a student’s ability to effectively and successfully transition into a lifestyle that facilitates their ability to adjust to the requirements of Australian universities. One possibility is the prospect of providing wellness programming, coaching and education targeting lifestyle behaviours for acculturation.

Doctoral research training : encouraging timely completion – case study reflections of one university on how it supports a vibrant research training culture

This is a case study of a young university striving to generate and sustain a vibrant Research Training culture. The university’s research training framework is informed by a belief in a project management approach to achieving successful research candidature. This has led to the definition and reporting of key milestones during candidature. In turn, these milestones have generated a range of training programs to support Higher Degree Research (HDR) students to meet these milestones in a timely fashion. Each milestone focuses on a specific set of skills blended with supporting the development of different parts of the doctoral thesis. Data on student progress and completion has provided evidence in highlighting the role that the milestones and training are playing in supporting timely completion. A university-wide reporting cycle generated data on the range of workshops and training provided to Higher Degree Research students and supervisors. The report provided details of thesis topic and format, as well as participation in research training events and participant evaluation of those events. Analysis of the data led to recommendations and comments on the strengths and weaknesses of the current research training program. Discussion considered strategies and drivers for enhancements into the future. In particular, the paper reflects on the significant potential role of centrally curated knowledge systems to support HDR student and supervisor access, and engagement and success. The research training program was developed using blended learning as a model. It covered face-to-face workshops as well as online modules. These were supplemented by web portals that offered a range of services to inform and educate students and supervisors and included opportunities for students to interact with each other. Topics ranged from the research life cycle, writing and publication, ethics, managing research data, managing copyright, and project management to use of software and the University’s Code of Conduct for Research. The challenges discussed included: How to reach off campus students and those studying in external modes? How best to promote events to potential participants? How long and what format is best for face-to-face sessions? What online resources best supplement face-to-face offerings? Is there a place for peer-based learning and what form should this take? These questions are raised by a relatively young university seeking to build and sustain a vibrant research culture. The rapid growth in enrolments in recent years has challenged previous one-to-one models of support. This review of research training is timely in seeking strategies to address changing research training support capacity and student needs. Part of the discussion will focus on supervisory training, noting that good supervision is the one remaining place where one-to-one support is provided. Ensuring that supervisors are appropriately equipped to address student expectations is considered in the context of the research training provisions. The paper concludes with reflection on the challenges faced, and recommended ways forward as the number of research students grows into the future.

The development and evaluation of a medical imaging training immersive environment

Introduction A novel realistic 3D virtual reality (VR) application has been developed to allow medical imaging students at Queensland University of Technology to practice radiographic techniques independently outside the usual radiography laboratory. Methods A flexible agile development methodology was used to create the software rapidly and effectively. A 3D gaming environment and realistic models were used to engender presence in the software while tutor-determined gold standards enabled students to compare their performance and learn in a problem-based learning pedagogy. Results Students reported high levels of satisfaction and perceived value and the software enabled up to 40 concurrent users to prepare for clinical practice. Student feedback also indicated that they found 3D to be of limited value in the desktop version compared to the usual 2D approach. A randomised comparison between groups receiving software-based and traditional practice measured performance in a formative role play with real equipment. The results of this work indicated superior performance with the equipment for the VR trained students (P = 0.0366) and confirmed the value of VR for enhancing 3D equipment-based problem-solving skills. Conclusions Students practising projection techniques virtually performed better at role play assessments than students practising in a traditional radiography laboratory only. The application particularly helped with 3D equipment configuration, suggesting that teaching 3D problem solving is an ideal use of such medical equipment simulators. Ongoing development work aims to establish the role of VR software in preparing students for clinical practice with a range of medical imaging equipment.

The QUT Training Model for Peer Leader Capacity Building

QUT has enacted a university-wide Peer Program’s Strategy which aims to improve student success and graduate outcomes. A component of this strategy is a training model providing relevant, quality-assured and timely training for all students who take on leadership roles. The training model is designed to meet the needs of the growing scale and variety of peer programs, and to recognise the multiple roles and programs in which students may be involved during their peer leader journey. The model builds peer leader capacity by offering centralised, beginning and ongoing training modules, delivered by in-house providers, covering topics which prepare students to perform their role safely, inclusively, accountably and skilfully. The model also provides efficiencies by differentiating between ‘core competency' and ‘program-specific’ modules, thus avoiding training duplication across multiple programs, and enabling training to be individually and flexibly formatted to suit the specific and unique needs of each program.