Qualifications, examinations and assessment: views and perspectives of students in the 14–19 phase on policy and practice

This paper brings to the forefront students’ views on one of the most significant aspects of education in the 14-19 phase, specifically the qualifications, examinations and assessment they experience. In this respect the paper foregrounds students as ‘policy actors’, they are significant players in the mediation of national qualification systems rather than just subjects in their implementation. Data from a national dataset of focus groups with 243 students from the 14-19 phase is presented. Key themes are highlighted relating to young people’s experiences of qualifications, examinations and assessment at this stage of education in a context of continuous initiatives and change as well as the impact of students being on the receiving end of qualification reform in situ which can be confusing, unsettling and ultimately detrimental to future success.

Estimation of fetal and effective dose for CT examinations

Doses from CT examinations are difficult to estimate. However, they are requested more frequently due to the increase in CT examinations. In particular, fetal dose estimations are frequently required for patients who have discovered, subsequent to the examination, that they were pregnant when the examination was conducted. A computer model has been developed to facilitate such dose calculations. This model combines empirical beam data with anatomical information. The model has been verified using thermoluminescent dosemeter (TLD) readings of internal and surface dose from both phantoms and patients, including intrauterine doses for patients undergoing afterloading gynaecological intracavitary treatment. Although only limited experimental data were available, the results indicate that the model accurately predicts uterine doses within acceptable errors. This approach has been validated for fetal dose estimation. The model was also used in a comparison with the nationally available CT dose data from the National Radiological Protection Board (NRPB). The two models were found to be in agreement for fetal dose estimations.

The student experience in Objective Structured Clinical Examinations (OSCEs): phenomenological analysis

Background and purpose
The dominant psychometric discourse of OSCEs may lead to unexpected problems, such as a checklist-based student performance1 which under emphasises the clinical relationship with student and standardised patient (SP). Such encounters can be dehumanising for SPs2 and have implications for what students learn about relational skills through the assessment process. In this study we explore medical students’ experiences of undertaking OSCEs using a phenomenological frame.
Methodology
Interpretative phenomenological analysis is a form of qualitative methodology which has strong resonance with existentialism and focuses on the lived experience without significant reference to external political or discursive
forces.
Six 4th year undergraduate medical students from Queen’s University Belfast were recruited in December 2013. Maximum variation sampling was used. Students were interviewed by a researcher in the week prior to the
OSCE and then again in the week following the OSCE in Jan 2014. Interviews were minimally structured in order to be open to respondents, rather than adhering to a fixed topic guide, but focussed on participants’ experiences, thoughts and feelings about taking part in OSCEs. Interviews were audio-recorded and
transcribed. Students were also asked to complete a short diary entry in the days prior to the OSCEs and another immediately following. Diary entries were written, emailed or audio-recorded at student’s preference.
Results
Transcripts are currently being analysed by interpretative phenomenological analysis. Preliminary analysis has demonstrated the significance of students’ relationships within the OSCE triad (student, SP and examiner); the effect of the immediate examination environment; realism versus roleplay; students’ perceptions of the purpose of assessment; and coping mechanisms.
Full results will be available by the time of the conference.
Conclusion and Discussion
Understanding the student experience in OSCEs is a crucial step in understanding the complex construction of relationships within the OSCE triad. The focus in OSCEs is typically on standardisation and reliability, but in exploring social interactions we may refocus attention on their inherent potential for learning and effects on both students and patients.
References
1. Hodges B. Medical education and the maintenance of incompetence. Med Teach 2006;28(8):690-6
2. Johnston JL, Lundy G, McCullough M, Gormley GJ. The view from over there: reframing the OSCE through the experience of standardised patient
raters. Med Educ 2013;47(9):899-909

Predictability in high-stakes examinations: students' perspectives on a perennial assessment dilemma

Key debates within educational assessment continuously encourage us to reflect on the design, delivery and implementation of examination systems as well as their relevance to students. In more recent times, such reflections have also required a rethinking of who is authoritative about assessment issues and whose views we seek in order to better understand these perennial assessment dilemmas. This paper considers one such dilemma, predictability in high-stakes assessment, and presents students’ perspectives on this issue. The context is the Irish Leaving Certificate (LC) taken by upper secondary students (aged between 16 and 18) in order (mainly) to enter tertiary-level education. The data come from 13 group interviews with 81 students across a range of schools in Ireland. Listening to students about complex, high-stakes examining problems has a limited history within the educational assessment literature. The findings from the study address this shortcoming and depict how students’ insightful reflections can improve our understanding of these dilemmas. Further, students are more than able to reflect on their own situations with regard to high stakes examining contexts and have important contributions to make to our fuller understanding of those elements that will promote high quality and fair assessment.

Preparing assessments and examinations. Common errors and good practice.

Assessment forms an important part of the student learning experience and students place a high value on the quality of feedback that they receive from academic staff on where they might improve on their examinations or assignments. However while feedback is important the quality of the actual assessment itself before students undertake an examination or commence writing an assignment is also important. It is imperative that students are clear in their understanding of what is expected of them in order to achieve a particular grade and that there is lack of ambiguity in examinations or assignments. Biggs (2003) highlighted the importance of clarity in what students are expected to be able to do at the end of a unit of study, and that intended learning outcomes should be clearly aligned to the assessment and communicated to students so that they can structure their learning activities to optimize their assessment performance. However as Rust (2002) highlighted there are often inconsistencies in assessment practices ranging from a mis-match of assessment and learning outcomes to the inclusion of additional learning criteria and lack of clarity in the instructions. Such inconsistencies and unacceptable errors in examination papers can undermine student confidence in the assessment process
In order to try and minimise such inconsistencies an internal assessment group was set up October 2013 within the School of Nursing and Midwifery at Queens University Belfast, consisting of representative academic staff from across the range of undergraduate and post graduate courses in nursing and midwifery. The assessment group was to be a point of reference for all school examinations with a particular remit to develop an assessment strategy for all nursing and midwifery programmes and to ensure that all assessments comply with current best practice and with Nursing and Midwifery Council (NMC) requirements.
Aim
This paper aims to highlight some examples of good practice and common errors that were found in assignments and examinations that were submitted to the assessment group for review.
References

Biggs. J. (2003) Teaching for Quality Learning at University – What the Student Does 2nd Edition SRHE / Open University Press, Buckingham.
Rust, C.( 2002) The impact of assessment on student learning, Active Learning in Higher education Vol3(2):145-158

The Impact of Multimedia Education on Uptake of Comprehensive Eye Examinations in Rural China: A Randomized, Controlled Trial

PURPOSE: To study the effect of multimedia education on acceptance of comprehensive eye examinations (CEEs), critical for detecting glaucoma and diabetic eye disease, among rural Chinese patients using a randomized, controlled design.
METHODS: Patients aged ≥40 years were recruited from 52 routine clinic sessions (26 intervention, 26 control) conducted at seven rural hospitals in Guangdong, China. Subjects answered demographic questionnaires, were tested on knowledge about CEEs and chronic eye disease, and were told the cost of examination (range US$0-8). At intervention sessions, subjects were cluster-randomized to view a 10-minute video on the value of CEEs and retested. Control subjects were not retested. Trial outcomes were acceptance of CEEs (primary outcome) and final knowledge scores (secondary outcome).
RESULTS: At baseline, >70% (p = 0.70) of both intervention (n = 241, 61.2 ± 12.3 years) and control (n = 218, 58.4 ± 11.7 years) subjects answered no knowledge questions correctly, but mean scores on the test (maximum 5 points) increased by 1.39 (standard deviation 0.12) points (p < 0.001) after viewing the video. Intervention (73.0%) and control (72.9%) subjects did not differ in acceptance of CEEs (p > 0.50). In mixed-effect logistic regression models, acceptance of CEEs was associated with availability of free CEEs (odds ratio 18.3, 95% confidence interval 1.32-253.0), but not group assignment or knowledge score. Acceptance was 97.5% (79/81) when free exams were offered.
CONCLUSIONS: Education increased knowledge about but not acceptance of CEEs, which was generally high. Making CEEs free could further increase acceptance.

Evaluation of extended training for general practice in Northern Ireland: qualitative study.

Objective To evaluate participants' perceptions of the impact on them of an additional six months' training beyond the standard 12 month general practice vocational training scheme. Design Qualitative study using focus groups. Setting General practice vocational training in Northern Ireland. Participants 13 general practitioner registrars, six of whom participated in the additional six months' training, and four trainers involved in the additional six months' training. Main outcome measures: Participants' views about their experiences in 18 month and 12 month courses. Results Participants reported that the 12 month course was generally positive but was too pressurised and focused on examinations, and also that it had a negative impact on self care. The nature of the learning and assessment was reported to have left participants feeling averse to further continuing education and lacking in confidence. In contrast, the extended six month component was reported to have restimulated learning by focusing more on patient care and promoting self directed learning. It developed confidence, promoted teamwork, and gave experience of two practice contexts, and was reported as valuable by both ex-registrars and trainers. However, both the 12 and 18 month courses left participants feeling underprepared for practice management and self care. Conclusions 12 months' training in general practice does not provide doctors with the necessary competencies and confidence to enter independent practice. The extended period was reported to promote greater professional development, critical evaluation skills, and orientation to lifelong learning but does not fill all the gaps.

Teaching and cognitive outcomes in A-levels and Advanced GNVQ's: Case studies from Science and Business Studies Classrooms

The purpose of the paper is to demonstrate how a research diary methodology, designed to analyse A-level and GNVQ classrooms, can be a powerful tool for examining pedagogy and quality of learning at the level of case study. Two subject areas, science and business studies, are presented as cases. Twelve teachers and thirty-four students were studied over a four-week period in May 1997 and contrasts were drawn between lessons from three A-level physics teachers/three Advanced GNVQ science teachers and two A-level business/economics teachers/four Advanced GNVQ business teachers. Lessons were analysed within a cognitive framework which distinguishes between conceptual and procedural learning and emphasizes the importance of metacognition and epistemological beliefs. Two dimensions of lessons were identified: pedagogical activities (e.g. teacher-led explanation, teacher-led guidance on a task, question/answer sessions, group discussions, working with IT) and cognitive outcomes (e.g. structuring and memorizing facts, understanding concepts and arguments, critical thinking, problem-solving, learning core skills, identifying values). Immediately after each lesson, teachers and students (three per class) completed structured research diaries with respect to the above dimensions. Data from the diaries reveal general and unique features of the lessons. Time-ofyear effects were evident (examinations pending in May), particularly in A-level classrooms. Students in business studies classes reported a wider range of learning activities and greater variety in cognitive outcomes than did students in science classes. Science students self-rating of their ability to manage and direct their own learning was generally low. The phenomenological aspects of the classrooms were consistently linked to teachers' lesson plans and what their teaching objectives were for those particular students at that particular time of the year.

Wide Variation in Adenoma Detection Rates at Screening Flexible Sigmoidoscopy

Background & Aims: Wide between-center variation in adenoma detection rates (ADRs) was observed in the U.K. Flexible Sigmoidoscopy Screening Trial (overall, 12.1%; range, 8.6%-15.9%; P <0.0001). The aim of this study was to determine whether the observed differences could be attributed to varying performance by endoscopists, to examine the effect of experience on performance, and to identify an attainable, standard ADR to which endoscopists could aspire. 

Methods: Thirteen medical endoscopists, one per trial center, each performed about 3000 examinations (200 per month) using the same equipment and protocol. Overall and monthly ADRs were compared using multivariable logistic regression. 

Results: Differences in ADRs were not explained by patient characteristics, incidence of colorectal cancer in the local population, or the endoscopists' medical specialty or previous experience. Average ADRs increased significantly with screening experience (up to 400 examinations). Endoscopists were classified as higher, intermediate, or lower adenoma detectors, and performance levels were maintained over time. Higher detectors had ADRs of 15% overall (men, 20%; women, 10%) and also detected more adenomas per case (higher/lower detectors, 21.7/10.4 adenomas per :100 examinations). 

Conclusions: The differences in ADRs were due to variation in performance of the endoscopists. Long-term follow-up will determine whether this variation is clinically important. We suggest that the standards in higher detecting centers should be achievable by all endoscopists screening unscreened populations aged older than 55 years. Endoscopists should aim to stay above the lower 95% confidence interval band for 200 examinations (10% overall; 5% in women, 15% in men).