Multiple mini-interview : how long is long enough?

Objectives The multiple mini-interview (MMI) overcomes the limitations of the traditional panel interview by multiple sampling to provide improved objectivity and reliability. Reliability of the MMI is affected by number of stations; however, there are few data reporting the influence of interview duration on MMI outcome and reliability. We aimed to determine whether MMI stations can be shortened without affecting applicant rankings or compromising test reliability.
Methods A total of 175 applicants were interviewed and assessed at 10 8-minute stations. Applicants were scored once after 8 minutes at five control stations and twice after 5 minutes and 8 minutes at five experimental stations. Scores at 5 and 8 minutes were compared using t-tests and correlation coefficients. Rankings of applicants based on 5- and 8-minute scores were compared using Spearman's rank order coefficient. The reliability of the MMI was examined for 5- and 8-minute scores using generalisability theory.
Results Mean scores at 5 minutes were lower than mean scores at 8 minutes. Cumulative scores at 5 minutes were also lower. There were highly significant correlations between 5- and 8-minute scores at all experimental stations (0.82–0.91; P < 0.01) and between the cumulative scores at 5 and 8 minutes (0.92; P < 0.01). There was a strong correlation between applicant rankings based on cumulative 5- and 8-minute scores (Spearman's rank order coefficient 0.92). Reliability was not affected.
Conclusions Reducing the duration of MMI stations from 8 to 5 minutes conserves resources with minimal effect on applicant ranking and test reliability.

Assessment of simulated clinical skills and distance students: can we do it better?

Australian universities have traditionally been able to supplement clinical education, for undergraduate nursing courses, delivered on placement with weekly clinical teaching in the simulated environment. The Objective Structured Clinical Assessment (OSCA) tool has been used in this simulated environment to assess clinical skills. Recently, however, online delivery of undergraduate nursing courses has become more common. The move from an internal mode of teaching to an online external mode is seen worldwide and poses challenges to staff and students as well as changing the teaching and learning culture of institutions (Philip and Wozniak, 2009). This cultural shift and the resulting diminishing timeframe for students to acquire and practice simulated clinical skills imply that it may become necessary to rethink assessment forms such as the OSCA assessment. This study examines whether or not the OSCA tool developed by Bujack et al. (1991a) is the best tool to be used in this new context, where online teaching is supplemented by very short, annual, intensive periods of study. Skills acquisition theories dictate that time is required to produce an ideal skills acquisition environment (Quinn, 2000) but the time constraints placed on students in such intensive periods of study could influence skills acquisition. This cross-sectional qualitative study used semi-structured interviews and focus groups to collect data. 65% of the nursing faculty participated in the study. The teaching of the Bachelor of Nursing (BN) occurred on two campuses and staff from both areas participated. This group of nurse academics was employed across the range of academic levels (from lecturer to professor) at the University. Data analysis followed a generic thematic analysis framework. Findings in this study show that there are a variety of attitudes and underpinning beliefs amongst staff in relation to the OSCAs. Doubts were raised in regard to the suitability of the use of the OSCA tool in this setting. It also became apparent during this study that the OSCA tool possibly serves purposes other than an assessment tool.

Nurse academics perceptions of the efficacy of the OSCA tool

The use of Objective Structured Clinical Examination/Objective Structured Clinical Assessment (OSCE/OSCA) has been well documented. How assessors currently view the process, and if the OSCA tool still fulfils the assessment requirements, is unclear. In this study, the beliefs and expectations of assessors towards the assessment tool used in an undergraduate nursing degree to assess clinical skills was investigated. A cross-sectional study used semi-structured interviews and focus groups with 16 lecturers in nursing from a rural Australian university. This represents 65% of the total nurse academic staff employed there. The key issues that the academic staff raised reflect those from previous studies, such as the use of OSCA as formative assessment or a quality check process before the clinical practice. The OSCAs were seen as a good assessment tool, which gave students the opportunity to receive feedback on their performance in relation to clinical skills. The drawbacks identified in relation to the use of OSCAs were that the OSCA was seen as stressful to students. This drawback was thought to be further compounded if there was a lack of congruence regarding essential criteria between assessors. If not adequately addressed these drawbacks will erode the potential the OSCA tool has to foster uniformity, which was one of the main reasons for its implementation. .

A trial of e-simulation of sudden patient deterioration (FIRST2ACT WEB) on student learning

BACKGROUND: High-fidelity simulation pedagogy is of increasing importance in health professional education; however, face-to-face simulation programs are resource intensive and impractical to implement across large numbers of students. OBJECTIVES: To investigate undergraduate nursing students' theoretical and applied learning in response to the e-simulation program-FIRST2ACT WEBTM, and explore predictors of virtual clinical performance. DESIGN AND SETTING: Multi-center trial of FIRST2ACT WEBTM accessible to students in five Australian universities and colleges, across 8 campuses. PARTICIPANTS: A population of 489 final-year nursing students in programs of study leading to license to practice. METHODS: Participants proceeded through three phases: (i) pre-simulation-briefing and assessment of clinical knowledge and experience; (ii) e-simulation-three interactive e-simulation clinical scenarios which included video recordings of patients with deteriorating conditions, interactive clinical tasks, pop up responses to tasks, and timed performance; and (iii) post-simulation feedback and evaluation. Descriptive statistics were followed by bivariate analysis to detect any associations, which were further tested using standard regression analysis. RESULTS: Of 409 students who commenced the program (83% response rate), 367 undergraduate nursing students completed the web-based program in its entirety, yielding a completion rate of 89.7%; 38.1% of students achieved passing clinical performance across three scenarios, and the proportion achieving passing clinical knowledge increased from 78.15% pre-simulation to 91.6% post-simulation. Knowledge was the main independent predictor of clinical performance in responding to a virtual deteriorating patient R(2)=0.090, F(7, 352)=4.962, p<0.001. DISCUSSION: The use of web-based technology allows simulation activities to be accessible to a large number of participants and completion rates indicate that 'Net Generation' nursing students were highly engaged with this mode of learning. CONCLUSION: The web-based e-simulation program FIRST2ACTTM effectively enhanced knowledge, virtual clinical performance, and self-assessed knowledge, skills, confidence, and competence in final-year nursing students.

An evaluation of the effect of an educational intervention for Australian social workers on competence in delivering brief cognitive behavioural strategies : a randomised controlled trial

Background Broad community access to high quality evidence-based primary mental health care is an ongoing challenge around the world. In Australia one approach has been to broaden access to care by funding psychologists and other allied health care professionals to deliver brief psychological treatments to general practitioners' patients. To date, there has been a scarcity of studies assessing the efficacy of social worker delivered psychological strategies. This study aims to build the evidence base by evaluating the impact of a brief educational intervention on social workers' competence in delivering cognitive behavioural strategies (strategies derived from cognitive behavioural therapy). Methods A randomised controlled trial design was undertaken with baseline and one-week follow-up measurement of both objective and self-perceived competence. Simulated consultations with standardised depressed patients were recorded on videotape and objective competence was assessed by blinded reviewers using the Cognitive Therapy Scale. Questionnaires completed by participants were used to measure self-perceived competence. The training intervention was a 15 hour face-to-face course involving presentations, video example consultations, written materials and rehearsal of skills in pairs. Results 40 Melbourne-based (Australia) social workers enrolled and were randomised and 9 of these withdrew from the study before the pre training simulated consultation. 30 of the remaining 31 social workers (97%) completed all phases of the intervention and evaluation protocol (16 from intervention and 14 from control group). The intervention group showed significantly greater improvements than the control group in objective competence (mean improvement of 14.2 (7.38-21.02) on the 66 point Cognitive Therapy Scale) and in subjective confidence (mean improvement of 1.28 (0.84-1.72) on a 5 point Likert scale). On average, the intervention group improved from below to above the base competency threshold on the Cognitive Therapy Scale whilst the control group remained below. Conclusions Social workers can attain significant improvements in competency in delivering cognitive behavioural strategies from undertaking brief face to face training. This is relevant in the context of health reforms that involve social worker delivery of evidence based psychological care. Further research is required to assess how these improvements in competence translate into performance in practice and clinical outcomes for patients.

Valuing and desiring purposes of education to transcend miseducative measurement practices

The separating and isolating tendencies of measuring practices can lead educators to lose sight of the aims and purposes of education. These end purposes can be used to guide and ensure that the activities of educators are educational, and therefore, Biesta recommends there is a need for educators to reconnect with them. This article. explores this notion of a ‘reconnection’ and argues that if educators are to challenge any potentially miseducative measuring practices, then this reconnection must require educators to value and desire particular end purposes. Desires are recognised to be existential in character and are identified as being necessary for initiating actions. It is argued here that this aspect of desires is important for understanding the significance of a ‘reconnection’ because without it the purposes of education may remain only as abstract philosophical ideals. To make a difference and to challenge the isolating and miseducative tendencies of some measuring practices, educators must come to value particular purposes of education, and in addition, they must exercise the courage to enact them. This can be made possible because educators strongly care for and desire the actualisation of the purposes to which they are connected.