A General Latent Class Model for Performance Evaluation of Diagnostic Tests in the Absence of a Gold Standard: An Application to Chagas Disease

We propose a new general Bayesian latent class model for evaluation of the performance of multiple diagnostic tests in situations in which no gold standard test exists based on a computationally intensive approach. The modeling represents an interesting and suitable alternative to models with complex structures that involve the general case of several conditionally independent diagnostic tests, covariates, and strata with different disease prevalences. The technique of stratifying the population according to different disease prevalence rates does not add further marked complexity to the modeling, but it makes the model more flexible and interpretable. To illustrate the general model proposed, we evaluate the performance of six diagnostic screening tests for Chagas disease considering some epidemiological variables. Serology at the time of donation (negative, positive, inconclusive) was considered as a factor of stratification in the model. The general model with stratification of the population performed better in comparison with its concurrents without stratification. The group formed by the testing laboratory Biomanguinhos FIOCRUZ-kit (c-ELISA and rec-ELISA) is the best option in the confirmation process by presenting false-negative rate of 0.0002% from the serial scheme. We are 100% sure that the donor is healthy when these two tests have negative results and he is chagasic when they have positive results.

Ensino de anestesiologia durante a graduação por meio de uma liga acadêmica: qual o impacto no aprendizado dos alunos?

Justificativa e objetivos: As ligas acadêmicas vêm se firmando como instrumentos de ensino médico e inserção dos estudantes de medicina na prática das especialidades, inclusive na anestesiologia. Como o papel das ligas no processo de desenvolvimento de competências e aprendizado de seus alunos não é bem conhecido, avaliou-se o aprendizado dos integrantes de uma liga acadêmica de anestesiologia após um ano de participação nas suas atividades. MÉTODO: Os alunos de uma liga acadêmica de anestesiologia foram acompanhados de março a dezembro de 2010 e avaliados por meio de testes cognitivos objetivos de múltipla escolha aplicados antes do início das atividades e após sua conclusão. A frequência nas atividades e o perfil epidemiológico dos alunos foram correlacionados aos resultados dos testes. RESULTADOS: Foram analisados 20 acadêmicos do 3º ao 6º ano, com média de idade de 22,8 (21-26) anos. A frequência média nas atividades propostas foi 10,4/13 (80%). A média de acertos no primeiro teste foi 8,1/17 (47,6%), sendo que os alunos do 3º ano apresentam notas menores (p < 0,02) em relação aos demais. No teste pós-liga, a média de acertos foi 11,9/17 (70%), mostrando melhora no desempenho dos alunos (p < 0,05), e não houve diferença entre as notas dos diferentes anos da graduação. Foi encontrada relação forte entre frequência nas atividades e melhora nas notas (r = 0,719; p < 0,001). CONCLUSÕES: Os alunos que participaram da liga apresentaram melhora nos testes de avaliação de conhecimento, sugerindo que a liga é útil instrumento de ensino e promove ganho de aprendizado em anestesiologia. As atividades desenvolvidas nas ligas podem ter papel positivo na formação acadêmica dos graduandos, destacando-se, neste artigo, a anestesiologia.

Schriftliche Prüfungen: Vorteile von Multiple True-False Fragen gegenüber Typ-A Fragen

Fragestellung/Einleitung: Prüfungen sind essentieller Bestandteil in der ärztlichen Ausbildung. Sie liefern wertvolle Informationen über den Entwicklungsprozess der Studierenden und wirken lernbegleitend und lernmodulierend [1], [2]. Bei schriftlichen Prüfungen dominieren derzeit Multiple Choice Fragen, die in verschiedenen Typen verwendet werden. Zumeist werden Typ-A Fragen genutzt, bei denen genau eine Antwort richtig ist. Multiple True-False (MTF) Fragen hingegen lassen mehrere richtige Antworten zu: es muss für jede Antwortmöglichkeit entschieden werden, ob diese richtig oder falsch ist. Durch die Mehrfachantwort scheinen MTF Fragen bestimmte klinische Sachverhalte besser widerspiegeln zu können. Auch bezüglich Reliabilität und dem Informationsgewinn pro Testzeit scheinen MTF Fragen den Typ-A Fragen überlegen zu sein [3]. Dennoch werden MTF Fragen bislang selten genutzt und es gibt wenig Literatur zu diesem Fragenformat. In dieser Studie soll untersucht werden, inwiefern die Verwendung von MTF Fragen die Nutzbarkeit (Utility) nach van der Vleuten (Reliabilität, Validität, Kostenaufwand, Effekt auf den Lernprozess und Akzeptanz der Teilnehmer) [4] schriftlicher Prüfungen erhöhen kann. Um die Testreliabilität zu steigern, sowie den Kostenaufwand für Prüfungen zu senken, möchten wir das optimale Bewertungssystem (Scoring) für MTF Fragen ermitteln. Methoden: Wir analysieren die Daten summativer Prüfungen der Medizinischen Fakultät der Universität Bern. Unsere Daten beinhalten Prüfungen vom ersten bis zum sechsten Studienjahr, sowie eine Facharztprüfung. Alle Prüfungen umfassen sowohl MTF als auch Typ-A Fragen. Für diese Prüfungen vergleichen wir die Viertel-, Halb- und Ganzpunktbewertung für MTF Fragen. Bei der Viertelpunktbewertung bekommen Kandidaten für jede richtige Teilantwort ¼ Punkt. Bei der Halbpunktbewertung wird ½ Punkt vergeben, wenn mehr als die Hälfte der Antwortmöglichkeiten richtig ist, einen ganzen Punkt erhalten die Kandidaten wenn alle Antworten richtig beantwortet wurden. Bei der Ganzpunktbewertung erhalten Kandidaten lediglich einen Punkt wenn die komplette Frage richtig beantwortet wurde. Diese unterschiedlichen Bewertungsschemata werden hinsichtlich Fragencharakteristika wie Trennschärfe und Schwierigkeit sowie hinsichtlich Testcharakteristika wie der Reliabilität einander gegenübergestellt. Die Ergebnisse werden ausserdem mit denen für Typ A Fragen verglichen. Ergebnisse: Vorläufige Ergebnisse deuten darauf hin, dass eine Halbpunktbewertung optimal zu sein scheint. Eine Halbpunktbewertung führt zu mittleren Item-Schwierigkeiten und daraus resultierend zu hohen Trennschärfen. Dies trägt zu einer hohen Testreliabilität bei. Diskussion/Schlussfolgerung: MTF Fragen scheinen in Verbindung mit einem optimalen Bewertungssystem, zu höheren Testreliabilitäten im Vergleich zu Typ A Fragen zu führen. In Abhängigkeit des zu prüfenden Inhalts könnten MTF Fragen einen wertvolle Ergänzung zu Typ-A Fragen darstellen. Durch die geeignete Kombination von MTF und Typ A Fragen könnte die Nutzbarkeit (Utility) schriftlicher Prüfungen verbessert werden.

Food choice in Siganus fuscescens: influence of macrophyte nutrient content and availability

Using multiple-choice feeding experiments, the selection of six species of macrophytes by the herbivorous rabbitfish Siganus fuscescens was examined. The rabbitfish showed distinct food choice in the laboratory; however, selection of macrophytes by S. fuscescens was not related to their absolute nutrient content (nitrogen, carbon, energy and ash free dry mass). Nutrient assimilation estimates showed that the macrophytes which were most preferred were those that S. fuscescens assimilated best. In S. fuscescens, the macrophytes that were preferred passed through the gut significantly faster than the less preferred species. Gut transit time had a significant effect on the absolute value of a food item in terms of net nutrient gain per unit time. This study showed that food value could be inferred from the absolute nutrient content of the macrophytes. Thus both the ability to assimilate nutrients as well as the absolute nutrient content of macrophytes must be quantified when assessing food value. (C) 2004 The Fisheries society of the British Isles.

Prenatal augmented auditory stimulation and visual system development: Effects of timing on intersensory organization

This study investigated the effects of augmented prenatal auditory stimulation on postnatal visual responsivity and neural organization in bobwhite quail (Colinus virginianus). I delivered conspecific embryonic vocalizations before, during, or after the development of a multisensory, midbrain audiovisual area, the optic tectum. Postnatal simultaneous choice tests revealed that hatchlings receiving augmented auditory stimulation during optic tectum development as embryos failed to show species-typical visual preferences for a conspecific maternal hen 72 hours after hatching. Auditory simultaneous choice tests showed no hatchlings had deficits in auditory function in any of the groups, indicating deficits were specific to visual function. ZENK protein expression confirmed differences in the amount of neural plasticity in multiple neuroanatomical regions of birds receiving stimulation during optic tecturn development, compared to unmanipulated birds. The results of these experiments support the notion that the timing of augmented prenatal auditory stimulation relative to optic tectum development can impact postnatal perceptual organization in an enduring way.^

The effects of student -centered and traditional models of teaching on reading skills

A pre-test, post-test, quasi-experimental design was used to examine the effects of student-centered and traditional models of reading instruction on outcomes of literal comprehension and critical thinking skills. The sample for this study consisted of 101 adult students enrolled in a high-level developmental reading course at a large, urban community college in the Southeastern United States. The experimental group consisted of 48 students, and the control group consisted of 53 students. Students in the experimental group were limited in the time spent reading a course text of basic skills, with instructors using supplemental materials such as poems, news articles, and novels. Discussions, the reading-writing connection, and student choice in material selection were also part of the student-centered curriculum. Students in the control group relied heavily on a course text and vocabulary text for reading material, with great focus placed on basic skills. Activities consisted primarily of multiple-choice questioning and quizzes. The instrument used to collect pre-test data was Descriptive Tests of Language Skills in Reading Comprehension; post-test data were taken from the Florida College Basic Skills Exit Test. A MANCOVA was used as the statistical method to determine if either model of instruction led to significantly higher gains in literal comprehension skills or critical thinking skills. A paired samples t-test was also used to compare pre-test and post-test means. The results of the MANCOVA indicated no significant difference between instructional models on scores of literal comprehension and critical thinking. Neither was there any significant difference in scores between subgroups of age (under 25 and 25 and older) and language background (native English speaker and second-language learner). The results of the t-test indicated, however, that students taught under both instructional models made significant gains in on both literal comprehension and critical thinking skills from pre-test to post-test.

Implicit eyewitness memory

After a crime has occurred, one of the most pressing objectives for investigators is to identify and interview any eyewitness that can provide information about the crime. Depending on his or her training, the investigative interviewer will use (to varying degrees) mostly yes/no questions, some cued and multiple-choice questions, with few open-ended questions. When the witness cannot generate any more details about the crime, one assumes the eyewitness' memory for the critical event has been exhausted. However, given what we know about memory, is this a safe assumption? In line with the extant literature on human cognition, if one assumes (a) an eyewitness has more available memories of the crime than he or she has accessible and (b) only explicit probes have been used to elicit information, then one can argue this eyewitness may still be able to provide additional information via implicit memory tests. In accordance with these notions, the present study had two goals: demonstrate that (1) eyewitnesses can reveal memory implicitly for a detail-rich event and (2) particularly for brief crimes, eyewitnesses can reveal memory for event details implicitly that were inaccessible when probed for explicitly. Undergraduates (N = 227) participated in a psychological experiment in exchange for research credit. Participants were presented with one of three stimulus videos (brief crime vs. long crime vs. irrelevant video). Then, participants either completed a series of implicit memory tasks or worked on a puzzle for 5 minutes. Lastly, participants were interviewed explicitly about the previous video via free recall and recognition tasks. Findings indicated that participants who viewed the brief crime provided significantly more crime-related details implicitly than those who viewed the long crime. The data also showed participants who viewed the long crime provided marginally more accurate details during free recall than participants who viewed the brief crime. Furthermore, participants who completed the implicit memory tasks provided significantly less accurate information during the explicit interview than participants who were not given implicit memory tasks. This study was the first to investigate implicit memory for eyewitnesses of a crime. To determine its applied value, additional empirical work is required.

The Effect of Mobile Technology as an Active Student Response System on the Acquisition of U.S. History Content of Secondary Students with Specific Learning Disabilities

Students with specific learning disabilities (SLD) typically learn less history content than their peers without disabilities and show fewer learning gains. Even when they are provided with the same instructional strategies, many students with SLD struggle to grasp complex historical concepts and content area vocabulary. Many strategies involving technology have been used in the past to enhance learning for students with SLD in history classrooms. However, very few studies have explored the effectiveness of emerging mobile technology in K-12 history classrooms. This study investigated the effects of mobile devices (iPads) as an active student response (ASR) system on the acquisition of U.S. history content of middle school students with SLD. An alternating treatments single subject design was used to compare the effects of two interventions. There were two conditions and a series of pretest probesin this study. The conditions were: (a) direct instruction and studying from handwritten notes using the interactive notebook strategy and (b) direct instruction and studying using the Quizlet App on the iPad. There were three dependent variables in this study: (a) percent correct on tests, (b) rate of correct responses per minute, and (c) rate of errors per minute. A comparative analysis suggested that both interventions (studying from interactive notes and studying using Quizlet on the iPad) had varying degrees of effectiveness in increasing the learning gains of students with SLD. In most cases, both interventions were equally effective. During both interventions, all of the participants increased their percentage correct and increased their rate of correct responses. Most of the participants decreased their rate of errors. The results of this study suggest that teachers of students with SLD should consider a post lesson review in the form of mobile devices as an ASR system or studying from handwritten notes paired with existing evidence-based practices to facilitate students’ knowledge in U.S. history. Future research should focus on the use of other interactive applications on various mobile operating platforms, on other social studies subjects, and should explore various testing formats such as oral question-answer and multiple choice.

Computer interaction : its effect on attitude and performance in older adults

This study examined the interaction of age, attitude, and performance within the context of an interactive computer testing experience. Subjects were 13 males and 47 females between the ages of 55 and 82, with a minimum of a high school education. Initial attitudes toward computers, as measured by the Cybernetics Attitude Scale (CAS), demonstrated overall equivalence between these older subjects and previously tested younger subjects. Post-intervention scores on the CAS indicated that attitudes toward computers were unaffected by either a "fun" or a "challenging" computer interaction experience. The differential effects of a computerized vs. a paperand- pencil presentation format of a 20-item, multiple choice vocabulary test were examined. Results indicated no significant differences in the performance of subjects in the two conditions, and no interaction effect between attitude and performance. These findings suggest that the attitudes of older adults towards computers do not affect their computerized testing performance, at least for short term testing of verbal abilities. A further implication is that, under the conditions presented here, older subjects appear to be unaffected by mode of testing. The impact of recent advanced in technology on older adults is discussed.

A simulação como método de ensino de habilidades no curso de medicina: relato de uma experiência com punção lombar

The teaching of the lumbar puncture (LP) technique with simulator is not well systematized in the curricula of medical schools. Studies show that training in the simulator provides learning technical skills, acquisition and retention of knowledge, improve self-confidence of the learner and enables the transfer to clinical practice. We intend this study to introduce simulated training in LP in medical course at the Universidade Federal do Rio Grande do Norte evaluating the experience taking into account quantitative aspects (performance on standardized tests) and qualitative (perception of the students regarding the method and the teaching process learning). The study was conducted in two phases. In the first phase practical training in PL was introduced in the 3rd year of medical school. Seventy-seven students were trained in small groups, guided by a checklist developed in the model Objective Structured Assessment of Technical Skill (OSATS), at this moment they knew they were not under performance evaluation. They were also asked whether they had prior chances to make an LP in patients. At the end of the first phase the students evaluated training in the following areas: teaching technique, simulator realism, time available per group, number of participants per group and relevance to medical practice. In the second phase, two years later, 18 students trained in first stage performed a new LP on the mannequin simulator, and its performance was evaluated through the same checklist of training in order to verify the technical retention. In addition, they answered a multiple choice test about practical aspects of the LP technique. Each participant received individual feedback on their performance at the end of their participation in the study. In the first phase of the study we found that only 4% of students had performed a lumbar puncture in patients until the 3rd year. The training of LP technique with simulator mannequin was considered relevant and the teaching methods was thoroughly evaluated. In the second phase, all participants were successful in implementing the lumbar puncture on the mannequin simulator, compliance with the most steps in a reasonable time, suggesting that would be able to perform the procedure in a patient.

Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements

The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.

Receptive Vocabulary of CLIL and Non-CLIL Primary and Secondary School Learners

CLIL instruction has been reported to be beneficial for foreign language vocabulary learning since CLIL students show higher vocabulary profiles than students of their same age in traditional EFL contexts. However, to our knowledge, the receptive vocabulary knowledge of CLIL and non-CLIL learners at the end of primary and secondary education has not been examined yet. Hence, this study aims at comparing the receptive vocabulary size 79 CLIL primary learners with the receptive vocabulary knowledge of 331 non-CLIL learners at the end of primary and secondary school. Sex-based differences were also analysed. The 2k Vocabulary Levels Test (VLT) was used for the purposes of the study. Results revealed that learners’ receptive vocabulary sizes lie within the most frequent 1000 words, non-CLIL secondary school students throw better results than primary students but the differences between the secondary group and the CLIL group are not statistically significant. As for sex-based differences, we found no significant differences among the groups. These findings led us to believe that the CLIL approach offers a benefit for vocabulary acquisition since CLIL learners have been exposed to the foreign language for a shorter period of time and the results are quite similar to their non-CLIL secondary school partners.

Assessing the Role of Vocabulary Breadth and Depth in Reading Comprehension: A Quantitative Study of Finnish EFL Learners

In this MA thesis, Finnish learners of English were studied in order to examine the relationship between second language vocabulary size, vocabulary depth, and reading comprehension. In addition, given the well-established connection between vocabulary size and reading comprehension, the second aim of the study was to see whether assessing vocabulary depth could add another dimension in predicting and explaining reading comprehension proficiency. Two groups were studied: the first group consisted of 39 Finnish upper secondary school students (the TOKA group) whereas the second group consisted of 19 university students of English at the University of Turku (the YLI group). Thus, comparisons were made between the results of a less advanced and a very advanced group of English learners, which was the third aim of the study. The participants in both groups filled in a background information form and took three tests: a multiple-choice reading comprehension test, a multiple-choice vocabulary size test, and a test designed to elicit information on learners’ depth of vocabulary knowledge of certain English words. The data were analysed using statistical methods. The results of the study show that the scores on the three tests were positively correlated in both study groups as well as in the two groups together. However, the correlations were higher in the TOKA group and in the two groups in total than in the YLI group. When examining the variance in reading comprehension test scores explained by vocabulary size and vocabulary depth, the figures of explained variance were again higher in the TOKA group and in the two groups in total than in the YLI group. When it comes to the results of the YLI group, vocabulary depth did not indeed seem to add any explained variance into the explanation of reading comprehension test scores. Based on the results of the study, it seems that vocabulary size and depth have a less significant role in the reading comprehension skills of more advanced learners of English. When looking at the less advanced TOKA group, on the other hand, vocabulary size and depth seem to be clear indicators of reading proficiency. In addition, the test results of the YLI group were clearly more uniform than those of the TOKA group. The variance in the test results of the TOKA group was large.

Clickers versus hand-raising in the physics college classroom : do clickers make a difference?

Abstract: Students who are actively involved in the learning process tend to develop deeper knowledge than those in traditional lecture classrooms (Beatty, 2007; Crouch & Mazur, 2001; Hake, 1998; Richardson, 2003). An instructional strategy that promotes active involvement is Peer Instruction. This strategy encourages student engagement by asking them to respond to conceptual multiple-choice questions intermittently throughout the lecture. These questions can be responded to by using an electronic hand-held device commonly known as a clicker that enables students’ responses to be displayed on a screen. When clickers are not available, a show of hands or other means can be used. The literature suggests that the impact on student learning is the same, whether the teacher uses clickers or simply asks students to raise their hand or use flashcards when responding to the questions (Lasry, 2007). This critical analysis argues that using clickers to respond to these in-class conceptual multiple-choice questions as opposed to using a show of hands leads to deeper conceptual understanding, better performance on tests, and greater overall enjoyment during class.||Résumé: Les étudiants qui sont activement impliqués dans le processus d'apprentissage ont tendance à développer des connaissances plus approfondies que lors de cours traditionnels (Beatty, 2007; Crouch & Mazur, 2001; Hake, 1998; Richardson, 2003). Une stratégie d'enseignement qui favorise la participation active est l’apprentissage par les pairs. Cette stratégie d’enseignement encourage l'engagement des élèves en leur demandant de répondre à des questions à choix multiples conceptuelles à plusieurs reprises durant le déroulement du cours. Ces questions peuvent être répondues à l'aide d'un appareil portatif électronique (un « clicker ») qui permet d’afficher de façon anonyme les réponses des élèves sur un écran. Si les clickers ne sont pas disponibles, les étudiants peuvent aussi répondre aux questions en levant la main. La littérature suggère que la méthode utilisée n’a pas d’impact sur l'apprentissage des élèves, que l'enseignant utilise des clickers, des flashcards ou qu’il demande simplement aux élèves de lever la main pour répondre aux questions (Lasry, 2007). Cette analyse critique fait valoir que l'utilisation de clickers pour répondre à ces questions à choix multiples conceptuelles en classe, plutôt que de faire lever la main aux étudiants, résulte en une compréhension conceptuelle plus approfondie, une meilleure performance aux examens et plus de plaisir pendant les cours.

The effect of mobile technology as an active student response system on the acquisition of U.S. history content of secondary students with specific learning disabilities

Students with specific learning disabilities (SLD) typically learn less history content than their peers without disabilities and show fewer learning gains. Even when they are provided with the same instructional strategies, many students with SLD struggle to grasp complex historical concepts and content area vocabulary. Many strategies involving technology have been used in the past to enhance learning for students with SLD in history classrooms. However, very few studies have explored the effectiveness of emerging mobile technology in K-12 history classrooms. ^ This study investigated the effects of mobile devices (iPads) as an active student response (ASR) system on the acquisition of U.S. history content of middle school students with SLD. An alternating treatments single subject design was used to compare the effects of two interventions. There were two conditions and a series of pretest probesin this study. The conditions were: (a) direct instruction and studying from handwritten notes using the interactive notebook strategy and (b) direct instruction and studying using the Quizlet App on the iPad. There were three dependent variables in this study: (a) percent correct on tests, (b) rate of correct responses per minute, and (c) rate of errors per minute. ^ A comparative analysis suggested that both interventions (studying from interactive notes and studying using Quizlet on the iPad) had varying degrees of effectiveness in increasing the learning gains of students with SLD. In most cases, both interventions were equally effective. During both interventions, all of the participants increased their percentage correct and increased their rate of correct responses. Most of the participants decreased their rate of errors. ^ The results of this study suggest that teachers of students with SLD should consider a post lesson review in the form of mobile devices as an ASR system or studying from handwritten notes paired with existing evidence-based practices to facilitate students’ knowledge in U.S. history. Future research should focus on the use of other interactive applications on various mobile operating platforms, on other social studies subjects, and should explore various testing formats such as oral question-answer and multiple choice. ^