Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey

Background Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options. Methods Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled. Results About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach. Conclusions Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.

Approaches to Ensure Quality Standards for Standardized/ Simulated Patient Performance in High Stakes Objective Structured Clinical Exams (OSCEs)

Introduction To meet the quality standards for high-stakes OSCEs, it is necessary to ensure high quality standardized performance of the SPs involved.[1] One of the ways this can be assured is through the assessment of the quality of SPs` performance in training and during the assessment. There is some literature concerning validated instruments that have been used to assess SP performance in formative contexts but very little related to high stakes contexts.[2], [3], [4]. Content and structure During this workshop different approaches to quality control for SPs` performance, developed in medicine, pharmacy and nursing OSCEs, will be introduced. Participants will have the opportunity to use these approaches in simulated interactions. Advantages and disadvantages of these approaches will be discussed. Anticipated outcomes By the end of this session, participants will be able to discuss the rationale for quality control of SPs` performance in high stakes OSCEs, outline key factors in creating strategies for quality control, identify various strategies for assuring quality control, and reflect on applications to their own practice. Who should attend The workshop is designed for those interested in quality assurance of SP performance in high stakes OSCEs. Level All levels are welcome. References Adamo G. 2003. Simulated and standardized patients in OSCEs: achievements and challenges:1992-2003. Med Teach. 25(3), 262- 270. Wind LA, Van Dalen J, Muijtjens AM, Rethans JJ. Assessing simulated patients in an educational setting: the MaSP (Maastricht Assessment of Simulated Patients). Med Educ 2004, 38(1):39-44. Bouter S, van Weel-Baumgarten E, Bolhuis S. Construction and validation of the Nijmegen Evaluation of the Simulated Patient (NESP): Assessing Simulated Patients' ability to role-play and provide feedback to students. Acad Med: Journal of the Association of American Medical Colleges 2012. May W, Fisher D, Souder D: Development of an instrument to measure the quality of standardized/simulated patient verbal feedback. Med Educ 2012, 2(1).

Quality Control for SP Performance Regarding Patient Portrayal in a High-Stakes OSCE

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SP’s performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Methods SP trainers from all five Swiss medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised with the partners twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. Both, SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). FAIR_OSCE The list of items to assess the quality of the simulation by SPs was primarily developed and used to provide formative feedback to the SPs in order to help them to improve their performance. It was therefore named “Feedbackstruckture for the Assessment of Interactive Role play in Objective Structured Clinical Exams (FAIR_OSCE). It was also used to assess the quality of patient portrayal during the exam. The results were calculated for each of the five faculties individually. Formative evaluation was given to the five faculties with individual feedback without revealing results of other faculties other than overall results. Results High quality of patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. In example the rate of completely correct reaction in medical tests increased from 88% to 95%. 95% completely correct reactions together with 4% sufficient reactions add up to 99% of the reactions meeting the requirements of the exam. SP educators using the instrument reported an augmentation of SPs performance induced by the use of the instrument. Disadvantages mentioned were high concentration needed to explicitly observe all criteria and cumbersome handling of the paper-based forms. Conclusion We were able to document a very high quality of SP performance in our exam. The data also indicate that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed augmentation of performance. The development of an iPad based application for the form is planned to address the cumbersome handling of the paper.

FAIR_OSCE – Feedback structure for assessment of interactive roleplay in Objective Structured Clinical Exams

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SPs’ performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Project description SP trainers from five medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). Outcome High quality of SPs’ patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. For example, the rate of completely correct reaction in medical tests increased from 88% to 95%. Together with 4% of sufficient performances these 95% add up to 99% of the reactions in medical tests meeting the standards of the exam. SP educators using the instrument reported an augmentation of SPs’ performance induced by the use of the instrument. Disadvantages mentioned were the high concentration needed to observe all criteria and the cumbersome handling of the paper-based forms. Discussion We were able to document a very high quality of SP performance in our exam. The data also indicates that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed enhancement of performance. The development of an iPad-based application for the form is planned to address the cumbersome handling of the paper.

Testing the GlaaS algorithm for dose measurements on low- and high-energy photon beams using an amorphous silicon portal imager

The GLAaS algorithm for pretreatment intensity modulation radiation therapy absolute dose verification based on the use of amorphous silicon detectors, as described in Nicolini et al. [G. Nicolini, A. Fogliata, E. Vanetti, A. Clivio, and L. Cozzi, Med. Phys. 33, 2839-2851 (2006)], was tested under a variety of experimental conditions to investigate its robustness, the possibility of using it in different clinics and its performance. GLAaS was therefore tested on a low-energy Varian Clinac (6 MV) equipped with an amorphous silicon Portal Vision PV-aS500 with electronic readout IAS2 and on a high-energy Clinac (6 and 15 MV) equipped with a PV-aS1000 and IAS3 electronics. Tests were performed for three calibration conditions: A: adding buildup on the top of the cassette such that SDD-SSD = d(max) and comparing measurements with corresponding doses computed at d(max), B: without adding any buildup on the top of the cassette and considering only the intrinsic water-equivalent thickness of the electronic portal imaging devices device (0.8 cm), and C: without adding any buildup on the top of the cassette but comparing measurements against doses computed at d(max). This procedure is similar to that usually applied when in vivo dosimetry is performed with solid state diodes without sufficient buildup material. Quantitatively, the gamma index (gamma), as described by Low et al. [D. A. Low, W. B. Harms, S. Mutic, and J. A. Purdy, Med. Phys. 25, 656-660 (1998)], was assessed. The gamma index was computed for a distance to agreement (DTA) of 3 mm. The dose difference deltaD was considered as 2%, 3%, and 4%. As a measure of the quality of results, the fraction of field area with gamma larger than 1 (%FA) was scored. Results over a set of 50 test samples (including fields from head and neck, breast, prostate, anal canal, and brain cases) and from the long-term routine usage, demonstrated the robustness and stability of GLAaS. In general, the mean values of %FA remain below 3% for deltaD equal or larger than 3%, while they are slightly larger for deltaD = 2% with %FA in the range from 3% to 8%. Since its introduction in routine practice, 1453 fields have been verified with GLAaS at the authors' institute (6 MV beam). Using a DTA of 3 mm and a deltaD of 4% the authors obtained %FA = 0.9 +/- 1.1 for the entire data set while, stratifying according to the dose calculation algorithm, they observed: %FA = 0.7 +/- 0.9 for fields computed with the analytical anisotropic algorithm and %FA = 2.4 +/- 1.3 for pencil-beam based fields with a statistically significant difference between the two groups. If data are stratified according to field splitting, they observed %FA = 0.8 +/- 1.0 for split fields and 1.0 +/- 1.2 for nonsplit fields without any significant difference.

Stable expression of Escherichia coli beta-glucuronidase A (GusA) in Giardia lamblia: application to high-throughput drug susceptibility testing

OBJECTIVES: In order to create a suitable model for high-throughput drug screening, a Giardia lamblia WB C6 strain expressing Escherichia coli glucuronidase A (GusA) was created and tested with respect to susceptibility to the anti-giardial drugs nitazoxanide and metronidazole. METHODS: GusA, a well-established reporter gene in other systems, was cloned into the vector pPacVInteg allowing stable expression in G. lamblia under control of the promoter from the glutamate dehydrogenase (gdh) gene. The resulting transgenic strain was compared with the wild-type strain in a vitality assay, characterized with respect to susceptibility to nitazoxanide, metronidazole and -- as assessed in a 96-well plate format -- to a panel of 15 other compounds to be tested for anti-giardial activity. RESULTS: GusA was stably expressed in G. lamblia. Using a simple glucuronidase assay protocol, drug efficacy tests yielded results similar to those from cell counting. CONCLUSIONS: G. lamblia WB C6 GusA is a suitable tool for high-throughput anti-giardial drug screening.

High-school students' need for cognition, self-control capacity, and school achievement: testing a mediation hypothesis

In the present article, we examine the hypothesis that high-school students' motivation to engage in cognitive endeavors (i.e., their need for cognition; NFC) is positively related to their dispositional self-control capacity. Furthermore, we test the prediction that the relation between NFC and school achievement is mediated by self-control capacity. A questionnaire study with grade ten high-school students (N = 604) revealed the expected relations between NFC, self-control capacity, and school achievement. Sobel tests showed that self-control capacity mediated the relation between NFC and school grades as well as grade retention.

Nanotoxicology: a perspective and discussion of whether or not in vitro testing is a valid alternative

Despite the many proposed advantages related to nanotechnology, there are increasing concerns as to the potential adverse human health and environmental effects that the production of, and subsequent exposure to nanoparticles (NPs) might pose. In regard to human health, these concerns are founded upon the plethora of knowledge gained from research relating to the effects observed following exposure to environmental air pollution. It is known that increased exposure to environmental air pollution can cause reduced respiratory health, as well as exacerbate pre-existing conditions such as cardiovascular disease and chronic obstructive pulmonary disease. Such disease states have also been associated with exposure to the NP component contained within environmental air pollution, raising concerns as to the effects of NP exposure. It is not only exposure to accidentally produced NPs however, which should be approached with caution. Over the past decades, NPs have been specifically engineered for a wide range of consumer, industrial and technological applications. Due to the inevitable exposure of NPs to humans, owing to their use in such applications, it is therefore imperative that an understanding of how NPs interact with the human body is gained. In vivo research poses a beneficial model for gaining immediate and direct knowledge of human exposure to such xenobiotics. This research outlook however, has numerous limitations. Increased research using in vitro models has therefore been performed, as these models provide an inexpensive and high-throughput alternative to in vivo research strategies. Despite such advantages, there are also various restrictions in regard to in vitro research. Therefore, the aim of this review, in addition to providing a short perspective upon the field of nanotoxicology, is to discuss (1) the advantages and disadvantages of in vitro research and (2) how in vitro research may provide essential information pertaining to the human health risks posed by NP exposure.

Combinations of prognostic tools for identification of high-risk normotensive patients with acute symptomatic pulmonary embolism

In haemodynamically stable patients with acute symptomatic pulmonary embolism (PE), studies have not evaluated the usefulness of combining the measurement of cardiac troponin, transthoracic echocardiogram (TTE), and lower extremity complete compression ultrasound (CCUS) testing for predicting the risk of PE-related death.

Quantitative repeated open application testing with a rinse-off product in methyldibromo glutaronitrile-sensitive patients: results of the IVDK

While the use of methyldibromo glutaronitrile (MDBGN) in leave-on products is clearly associated with high sensitization or elicitation risk, such a clear-cut relation could be questioned with regard to rinse-off products.

Cardiac troponin testing and the simplified Pulmonary Embolism Severity Index. The SWIss Venous ThromboEmbolism Registry (SWIVTER)

A low simplified Pulmonary Embolism Severity Index (sPESI), defined as age ≤80 years and absence of systemic hypotension, tachycardia, hypoxia, cancer, heart failure, and lung disease, identifies low-risk patients with acute pulmonary embolism (PE). It is unknown whether cardiac troponin testing improves the prediction of clinical outcomes if the sPESI is not low. In the prospective Swiss Venous Thromboembolism Registry, 369 patients with acute PE and a troponin test (conventional troponin T or I, highly sensitive troponin T) were enrolled from 18 hospitals. A positive test result was defined as a troponin level above the manufacturers assay threshold. Among the 106 (29%) patients with low sPESI, the rate of mortality or PE recurrence at 30 days was 1.0%. Among the 263 (71%) patients with high sPESI, 177 (67%) were troponin-negative and 86 (33%) troponin-positive; the rate of mortality or PE recurrence at 30 days was 4.6% vs. 12.8% (p=0.015), respectively. Overall, risk assessment with a troponin test (hazard ratio [HR] 3.39, 95% confidence interval [CI] 1.38-8.37; p=0.008) maintained its prognostic value for mortality or PE recurrence when adjusted for sPESI (HR 5.80, 95%CI 0.76-44.10; p=0.09). The combination of sPESI with a troponin test resulted in a greater area under the receiver-operating characteristic curve (HR 0.72, 95% CI 0.63-0.81) than sPESI alone (HR 0.63, 95% CI 0.57-0.68) (p=0.023). In conclusion, although cardiac troponin testing may not be required in patients with a low sPESI, it adds prognostic value for early death and recurrence for patients with a high sPESI.

Relevance of in vitro tests of adhesive and composite dental materials, a review in 3 parts. Part 1: Approval requirements and standardized testing of composite materials according to ISO specifications

The first part of this three-part review on the relevance of laboratory testing of composites and adhesives deals with approval requirements for composite materials. We compare the in vivo and in vitro literature data and discuss the relevance of in vitro analyses. The standardized ISO protocols are presented, with a focus on the evaluation of physical parameters. These tests all have a standardized protocol that describes the entire test set-up. The tests analyse flexural strength, depth of cure, susceptibility to ambient light, color stability, water sorption and solubility, and radiopacity. Some tests have a clinical correlation. A high flexural strength, for instance, decreases the risk of fractures of the marginal ridge in posterior restorations and incisal edge build-ups of restored anterior teeth. Other tests do not have a clinical correlation or the threshold values are too low, which results in an approval of materials that show inferior clinical properties (e.g., radiopacity). It is advantageous to know the test set-ups and the ideal threshold values to correctly interpret the material data. Overall, however, laboratory assessment alone cannot ensure the clinical success of a product.

High specificity of line-immunoassay based algorithms for recent HIV-1 infection independent of viral subtype and stage of disease

Background Serologic testing algorithms for recent HIV seroconversion (STARHS) provide important information for HIV surveillance. We have shown that a patient's antibody reaction in a confirmatory line immunoassay (INNO-LIATM HIV I/II Score, Innogenetics) provides information on the duration of infection. Here, we sought to further investigate the diagnostic specificity of various Inno-Lia algorithms and to identify factors affecting it. Methods Plasma samples of 714 selected patients of the Swiss HIV Cohort Study infected for longer than 12 months and representing all viral clades and stages of chronic HIV-1 infection were tested blindly by Inno-Lia and classified as either incident (up to 12 m) or older infection by 24 different algorithms. Of the total, 524 patients received HAART, 308 had HIV-1 RNA below 50 copies/mL, and 620 were infected by a HIV-1 non-B clade. Using logistic regression analysis we evaluated factors that might affect the specificity of these algorithms. Results HIV-1 RNA <50 copies/mL was associated with significantly lower reactivity to all five HIV-1 antigens of the Inno-Lia and impaired specificity of most algorithms. Among 412 patients either untreated or with HIV-1 RNA ≥50 copies/mL despite HAART, the median specificity of the algorithms was 96.5% (range 92.0-100%). The only factor that significantly promoted false-incident results in this group was age, with false-incident results increasing by a few percent per additional year. HIV-1 clade, HIV-1 RNA, CD4 percentage, sex, disease stage, and testing modalities exhibited no significance. Results were similar among 190 untreated patients. Conclusions The specificity of most Inno-Lia algorithms was high and not affected by HIV-1 variability, advanced disease and other factors promoting false-recent results in other STARHS. Specificity should be good in any group of untreated HIV-1 patients.