Validity evidence and reliability of a Simulated Patient Feedback Instrument

In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Methods Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. Results All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. Conclusions The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

Approaches to Ensure Quality Standards for Standardized/ Simulated Patient Performance in High Stakes Objective Structured Clinical Exams (OSCEs)

Introduction To meet the quality standards for high-stakes OSCEs, it is necessary to ensure high quality standardized performance of the SPs involved.[1] One of the ways this can be assured is through the assessment of the quality of SPs` performance in training and during the assessment. There is some literature concerning validated instruments that have been used to assess SP performance in formative contexts but very little related to high stakes contexts.[2], [3], [4]. Content and structure During this workshop different approaches to quality control for SPs` performance, developed in medicine, pharmacy and nursing OSCEs, will be introduced. Participants will have the opportunity to use these approaches in simulated interactions. Advantages and disadvantages of these approaches will be discussed. Anticipated outcomes By the end of this session, participants will be able to discuss the rationale for quality control of SPs` performance in high stakes OSCEs, outline key factors in creating strategies for quality control, identify various strategies for assuring quality control, and reflect on applications to their own practice. Who should attend The workshop is designed for those interested in quality assurance of SP performance in high stakes OSCEs. Level All levels are welcome. References Adamo G. 2003. Simulated and standardized patients in OSCEs: achievements and challenges:1992-2003. Med Teach. 25(3), 262- 270. Wind LA, Van Dalen J, Muijtjens AM, Rethans JJ. Assessing simulated patients in an educational setting: the MaSP (Maastricht Assessment of Simulated Patients). Med Educ 2004, 38(1):39-44. Bouter S, van Weel-Baumgarten E, Bolhuis S. Construction and validation of the Nijmegen Evaluation of the Simulated Patient (NESP): Assessing Simulated Patients' ability to role-play and provide feedback to students. Acad Med: Journal of the Association of American Medical Colleges 2012. May W, Fisher D, Souder D: Development of an instrument to measure the quality of standardized/simulated patient verbal feedback. Med Educ 2012, 2(1).

Quality assurance of simulated patient feedback in communication training for fourth-year medical students

Introduction In our program, simulated patients (SPs) give feedback to medical students in the course of communication skills training. To ensure effective training, quality control of the SPs’ feedback should be implemented. At other institutions, medical students evaluate the SPs’ feedback for quality control (Bouter et al., 2012). Thinking about implementing quality control for SPs’ feedback in our program, we wondered whether the evaluation by students would result in the same scores as evaluation by experts. Methods Consultations simulated by 4th-year medical students with SPs were video taped including the SP’s feedback to the students (n=85). At the end of the training sessions students rated the SPs’ performance using a rating instrument called Bernese Assessment for Role-play and Feedback (BARF) containing 11 items concerning feedback quality. Additionally the videos were evaluated by 3 trained experts using the BARF. Results The experts showed a high interrater agreement when rating identical feedbacks (ICCunjust=0.953). Comparing the rating of students and experts, high agreement was found with regard to the following items: 1. The SP invited the student to reflect on the consultation first, Amin (= minimal agreement) 97% 2. The SP asked the student what he/she liked about the consultation, Amin = 88%. 3. The SP started with positive feedback, Amin = 91%. 4. The SP was comparing the student with other students, Amin = 92%. In contrast the following items showed differences between the rating of experts and students: 1. The SP used precise situations for feedback, Amax (=maximal agreement) 55%, Students rated 67 of SPs’ feedbacks to be perfect with regard to this item (highest rating on a 5 point Likert scale), while only 29 feedbacks were rated this way by the experts. 2. The SP gave precise suggestions for improvement, Amax 75%, 62 of SPs’ feedbacks obtained the highest rating from students, while only 44 of SPs’ feedbacks achieved the highest rating in the view of the experts. 3. The SP speaks about his/her role in the third person, Amax 60%. Students rated 77 feedbacks with the highest score, while experts judged only 43 feedbacks this way. Conclusion Although evaluation by the students was in agreement with that of experts concerning some items, students rated the SPs’ feedback more often with the optimal score than experts did. Moreover it seems difficult for students to notice when SPs talk about the role in the first instead of the third person. Since precision and talking about the role in the third person are important quality criteria of feedback, this result should be taken into account when thinking about students’ evaluation of SPs’ feedback for quality control. Bouter, S., E. van Weel-Baumgarten, and S. Bolhuis. 2012. Construction and Validation of the Nijmegen Evaluation of the Simulated Patient (NESP): Assessing Simulated Patients’ Ability to Role-Play and Provide Feedback to Students. Academic Medicine: Journal of the Association of American Medical Colleges

Benefits of communication skills training after real patient exposure

The established communication skills training (CST) curriculum consists of continuous education, frequent practice with real patients, supervision and feedback from medical teachers. The limitation of this curriculum is that students have to directly apply the theoretical basis they are provided with to real patients. To improve the existing CST curriculum a CST module involving simulated patients was introduced in order to develop more complex communication skills.

A multilingual audiometer simulator software for training purposes

A set of algorithms, which allows a computer to determine the answers of simulated patients during pure tone and speech audiometry, is presented. Based on these algorithms, a computer program for training in audiometry was written and found to be useful for teaching purposes.

Consequences of insecurity in emergency telephone consultations: an experimental study in medical students.

QUESTION UNDER STUDY Handling emergency telephone consultations (ETCs) is a challenging and very important task for doctors. The aims of the study were to document insecurity in medical students during ETCs and to identify the reasons for that insecurity. We hypothesised that insecurity is associated with advising more urgent action (e.g. advice to call for an ambulance) in ETCs. METHODS We used ETCs with simulated patients (SPs), with each student randomly allocated two of four possible cases. After the training, 137 students reported on any insecurity that they had in the various ETC phases. We analysed the reasons for insecurity using descriptive statistics. The association between the students' advice that urgent action was needed and their insecurity was analysed with Spearman rank correlation. RESULTS Overall, 95% of the students felt insecure in at least one phase of their ETC. History taking was the phase in which students felt most insecure (63.1%), followed by the phase of analysing the information given by the patient (44.9%). Perceived insecurity was associated with more urgent advice in one case scenario (abdominal pain; correlation r = 0.46; p <0.01). The other two cases (child with fever; chest pain) also had a positive, but not statistically significant, correlation trend (p <0.12; p <0.08). CONCLUSIONS Insecurity is highly prevalent among medical students in their ETC decision-making. ETC training in medical schools, with a focus on structured history taking and formulating discriminating questions, might help decrease insecurity in ETCs. Medical education should also teach management of insecurity.

Crossover comparison of the laryngeal mask supreme and the i-gel in simulated difficult airway scenario in anesthetized patients

BACKGROUND: The single-use supraglottic airway devices LMA-Supreme (LMA-S; Laryngeal Mask Company, Henley-on-Thames, United Kingdom) and i-gel (Intersurgical Ltd, Wokingham, Berkshire, United Kingdom) have a second tube for gastric tube insertion. Only the LMA-S has an inflatable cuff. They have the same clinical indications and might be useful for difficult airway management. This prospective, crossover, randomized controlled trial was performed in a simulated difficult airway scenario using an extrication collar limiting mouth opening and neck movement. METHODS: Sixty patients were included. Both devices were placed in random order in each patient. Primary outcome was overall success rate. Other measurements were time to successful ventilation, airway leak pressure, fiberoptic glottic view, and adverse events. RESULTS: Success rate for the LMA-S was 95% versus 93% for the i-gel (P = 1.000). LMA-S needed shorter insertion time (34 +/- 12 s vs. 42 +/- 23 s, P = 0.024). Tidal volumes and airway leak pressure were similar (LMA-S 26 +/- 8 cm H20; i-gel 27 +/- 9 cm H20; P = 0.441). Fiberoptic view through the i-gel showed less epiglottic downfolding. Overall agreement in insertion outcome was 54 (successes) and 1 (failure) or 55 (92%) of 60 patients. The difference in success rate was 1.7% (95% CI -11.3% to 7.6%). CONCLUSIONS: Both airway devices had similar insertion success and clinical performance in the simulated difficult airway situation. The authors found less epiglottic downfolding and better fiberoptic view but longer insertion time with the i-gel. Our study shows that both devices are feasible for emergency airway management in patients with reduced neck movement and limited mouth opening.

Line bisection in simulated homonymous hemianopia

Hemianopic patients make a systematic error in line bisection, showing a contra-lesional bias towards their blind side, which is the opposite of that in hemineglect patients. This error has been attributed variously to the visual field defect, to long-term strategic adaptation, or to independent effects of damage to extrastriate cortex. To determine if hemianopic bisection error can occur without the latter two factors, we studied line bisection in healthy subjects with simulated homonymous hemianopia using a gaze-contingent display, with different line-lengths, and with or without markers at both ends of the lines. Simulated homonymous hemianopia did induce a contra-lesional bisection error and this was associated with increased fixations towards the blind field. This error was found with end-marked lines and was greater with very long lines. In a second experiment we showed that eccentric fixation alone produces a similar bisection error and eliminates the effect of line-end markers. We conclude that a homonymous hemianopic field defect alone is sufficient to induce both a contra-lesional line bisection error and previously described alterations in fixation distribution, and does not require long-term adaptation or extrastriate damage.

Rapid adaptation of visual search in simulated hemianopia

Patients with homonymous hemianopia have altered visual search patterns, but it is unclear how rapidly this develops and whether it reflects a strategic adaptation to altered perception or plastic changes to tissue damage. To study the temporal dynamics of adaptation alone, we used a gaze-contingent display to simulate left or right hemianopia in 10 healthy individuals as they performed 25 visual search trials. Visual search was slower and less accurate in hemianopic than in full-field viewing. With full-field viewing, there were improvements in search speed, fixation density, and number of fixations over the first 9 trials, then stable performance. With hemianopic viewing, there was a rapid shift of fixation into the blind field over the first 5-7 trials, followed by continuing gradual improvements in completion time, number of fixations, and fixation density over all 25 trials. We conclude that in the first minutes after onset of hemianopia, there is a biphasic pattern of adaptation to altered perception: an early rapid qualitative change that shifts visual search into the blind side, followed by more gradual gains in the efficiency of using this new strategy, a pattern that has parallels in other studies of motor learning.

CT patterns of fungal pulmonary infections of the lung: Comparison of standard-dose and simulated low-dose CT

To assess the effect of radiation dose reduction on the appearance and visual quantification of specific CT patterns of fungal infection in immuno-compromised patients.

Radiation exposure of patients who undergo CT of the trunk

PURPOSE: To determine the radiation dose delivered to organs during standard computed tomographic (CT) examination of the trunk. MATERIALS AND METHODS: In vivo locations and sizes of specific body organs were determined from CT images of patients who underwent examinations. The corresponding CT investigations were then simulated on an anthropomorphic phantom. The resulting doses were measured at 70 different sites inside the phantom by using thermoluminescent dosimeters. On the basis of measurements of free-in-air air kerma at the rotation axis of the CT gantry, conversion factors were calculated so that measurements could be used with different models of CT equipment. RESULTS: Starting from the dose values recorded, the mean organ doses were determined for 21 organs. The skin received 22-36 mGy; the lungs, less than 1-18 mGy; the kidneys, 7-24 mGy; and the ovaries, less than 1-19 mGy, depending on the type of CT examination performed. CONCLUSION: These values are high compared with other x-ray examinations and should be minimized as much as possible. The number of tomographic sections obtained should be kept as low as possible according to diagnostic need.

Tracing of patients lost to follow-up and HIV transmission: Mathematical modelling study based on two large ART programmes in Malawi

OBJECTIVES: Treatment as prevention depends on retaining HIV-infected patients in care. We investigated the effect on HIV transmission of bringing patients lost to follow up (LTFU) back into care. DESIGN: Mathematical model. METHODS: Stochastic mathematical model of cohorts of 1000 HIV-infected patients on antiretroviral therapy (ART), based on data from two clinics in Lilongwe, Malawi. We calculated cohort viral load (CVL; sum of individual mean viral loads each year) and used a mathematical relationship between viral load and transmission probability to estimate the number of new HIV infections. We simulated four scenarios: 'no LTFU' (all patients stay in care); 'no tracing' (patients LTFU are not traced); 'immediate tracing' (after missed clinic appointment); and, 'delayed tracing' (after six months). RESULTS: About 440 of 1000 patients were LTFU over five years. CVL (million copies/ml per 1000 patients) were 3.7 (95% prediction interval [PrI] 2.9-4.9) for no LTFU, 8.6 (95% PrI 7.3-10.0) for no tracing, 7.7 (95% PrI 6.2-9.1) for immediate, and 8.0 (95% PrI 6.7-9.5) for delayed tracing. Comparing no LTFU with no tracing the number of new infections increased from 33 (95% PrI 29-38) to 54 (95% PrI 47-60) per 1000 patients. Immediate tracing prevented 3.6 (95% PrI -3.3-12.8) and delayed tracing 2.5 (95% PrI -5.8-11.1) new infections per 1000. Immediate tracing was more efficient than delayed tracing: 116 and to 142 tracing efforts, respectively, were needed to prevent one new infection. CONCLUSION: Tracing of patients LTFU enhances the preventive effect of ART, but the number of transmissions prevented is small.

Impact of a stress coping strategy on perceived stress levels and performance during a simulated cardiopulmonary resuscitation: a randomized controlled trial

Background Cardiopulmonary resuscitation (CPR) causes significant stress, which may cause deficiencies in attention and increase distractibility. This may lead to misjudgements of priorities and delays in CPR performance, which may further increase mental stress (vicious cycle). This study assessed the impact of a task-focusing strategy on perceived stress levels and performance during a simulated CPR scenario. Methods This prospective, randomized-controlled trial was conducted at the simulator-center of the University Hospital Basel, Switzerland. A total of 124 volunteer medical students were randomized to receive a 10 minute instruction to cope with stress by loudly posing two taskfocusing questions (“what is the patient’s condition?”, “what immediate action is needed?”) when feeling overwhelmed by stress (intervention group) or a control group. The primary outcome was the perceived levels of stress and feeling overwhelmed (stress/overload); secondary outcomes were hands-on time, time to start CPR and number of leadership statements. Results Participants in the intervention group reported significantly less stress/overload levels compared to the control group (mean difference: -0.6 (95%CI −1.3, -0.1), p=0.04). Higher stress/overload was associated with less hands-on time. Leadership statements did not differ between groups, but the number of leadership statements did relate to performance. Hands-on time was longer in the intervention- group, but the difference was not statistically different (difference 5.5 (95%CI −3.1, 14.2), p=0.2); there were no differences in time to start CPR (difference −1.4 (95%CI −8.4, 5.7), p=0.71). Conclusions A brief stress-coping strategy moderately decreased perceived stress without significantly affecting performance in a simulated CPR. Further studies should investigate more intense interventions for reducing stress.

Systematic literature review of digital three-dimensional superimposition techniques to create virtual dental patients

PURPOSE Digital developments have led to the opportunity to compose simulated patient models based on three-dimensional (3D) skeletal, facial, and dental imaging. The aim of this systematic review is to provide an update on the current knowledge, to report on the technical progress in the field of 3D virtual patient science, and to identify further research needs to accomplish clinical translation. MATERIALS AND METHODS Searches were performed electronically (MEDLINE and OVID) and manually up to March 2014 for studies of 3D fusion imaging to create a virtual dental patient. Inclusion criteria were limited to human studies reporting on the technical protocol for superimposition of at least two different 3D data sets and medical field of interest. RESULTS Of the 403 titles originally retrieved, 51 abstracts and, subsequently, 21 full texts were selected for review. Of the 21 full texts, 18 studies were included in the systematic review. Most of the investigations were designed as feasibility studies. Three different types of 3D data were identified for simulation: facial skeleton, extraoral soft tissue, and dentition. A total of 112 patients were investigated in the development of 3D virtual models. CONCLUSION Superimposition of data on the facial skeleton, soft tissue, and/or dentition is a feasible technique to create a virtual patient under static conditions. Three-dimensional image fusion is of interest and importance in all fields of dental medicine. Future research should focus on the real-time replication of a human head, including dynamic movements, capturing data in a single step.