How secure is a Newtonian paradigm for psychological and educational measurement?

From the early 1900s, some psychologists have attempted to establish their discipline as a quantitative science. In using quantitative methods to investigate their theories, they adopted their own special definition of measurement of attributes such as cognitive abilities, as though they were quantities of the type encountered in Newtonian science. Joel Michell has presented a carefully reasoned argument that psychological attributes lack additivity, and therefore cannot be quantities in the same way as the attributes of classical Newtonian physics. In the early decades of the 20th century, quantum theory superseded Newtonian mechanics as the best model of physical reality. This paper gives a brief, critical overview of the evolution of current measurement practices in psychology, and suggests the need for a transition from a Newtonian to a quantum theoretical paradigm for psychological measurement. Finally, a case study is presented that considers the implications of a quantum theoretical model for educational measurement. In particular, it is argued that, since the OECD’s Programme for International Student Assessment (PISA) is predicated on a Newtonian conception of measurement, this may constrain the extent to which it can make accurate comparisons of the achievements of different education systems.

A quantum measurement paradigm for educational predicates: Implications for validity in educational measurement

The outcomes of educational assessments undoubtedly have real implications for students, teachers, schools and education in the widest sense. Assessment results are, for example, used to award qualifications that determine future educational or vocational pathways of students. The results obtained by students in assessments are also used to gauge individual teacher quality, to hold schools to account for the standards achieved by their students, and to compare international education systems. Given the current high-stakes nature of educational assessment, it is imperative that the measurement practices involved have stable philosophical foundations. However, this paper casts doubt on the theoretical underpinnings of contemporary educational measurement models. Aspects of Wittgenstein’s later philosophy and Bohr’s philosophy of quantum theory are used to argue that a quantum theoretical rather than a Newtonian model is appropriate for educational measurement, and the associated implications for the concept of validity are elucidated. Whilst it is acknowledged that the transition to a quantum theoretical framework would not lead to the demise of educational assessment, it is argued that, where practical, current high-stakes assessments should be reformed to become as ‘low-stakes’ as possible. The paper also undermines some of the pro high-stakes testing rhetoric that has a tendency to afflict education.

Assessing clinical reasoning: A method to monitor its development in a PBL curriculum

The aim of this study was to develop and trial a method to monitor the evolution of clinical reasoning in a PBL curriculum that is suitable for use in a large medical school. Termed Clinical Reasoning Problems (CRPs), it is based on the notion that clinical reasoning is dependent on the identification and correct interpretation of certain critical clinical features. Each problem consists of a clinical scenario comprising presentation, history and physical examination. Based on this information, subjects are asked to nominate the two most likely diagnoses and to list the clinical features that they considered in formulating their diagnoses, indicating whether these features supported or opposed the nominated diagnoses. Students at different levels of medical training completed a set of 10 CRPs as well as the Diagnostic Thinking Inventory, a self-reporting questionnaire designed to assess reasoning style. Responses were scored against those of a reference group of general practitioners. Results indicate that the CRPs are an easily administered, reliable and valid assessment of clinical reasoning, able to successfully monitor its development throughout medical training. Consequently, they can be employed to assess clinical reasoning skill in individual students and to evaluate the success of undergraduate medical schools in providing effective tuition in clinical reasoning.

The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates

Objective: To evaluate the feasibility, reliability and acceptability of the mini clinical evaluation exercise (mini-CEX) for performance assessment among international medical graduates (IMGs). Design, setting and participants: Observational study of 209 patient encounters involving 28 IMGs and 35 examiners at three metropolitan teaching hospitals in New South Wales, Victoria and Queensland, September-December 2006. Main outcome measures: The reliability of the mini-CEX was estimated using generatisability (G) analysis, and its acceptability was evaluated by a written survey of the examiners and IMGs. Results: The G coefficient for eight encounters was 0.88, suggesting that the reliability of the mini-CEX was 0.90 for 10 encounters. Almost half of the IMGs (7/16) and most examiners (14/18) were satisfied with the mini-CEX as a learning tool. Most of the IMGs and examiners enjoyed the immediate feedback, which is a strong component of the tool. Conclusion: The mini-CEX is a reliable tool for performance assessment of IMGs, and is acceptable to and well received by both learners and supervisors.

In-training assessment - Its potential in enhancing clinical teaching

Context In-training assessment (ITA) has established its place alongside formative and summative assessment at both the undergraduate and postgraduate level. In this paper the authors aimed to identify those characteristics of ITA that could enhance clinical teaching. Methods A literature review and discussions by an expert working group at the Ninth Cambridge Conference identified the aspects of ITA that could enhance clinical teaching. Results The features of ITA identified included defining the specific benefits to the learner, teacher and institution, and highlighting the patient as the context for ITA and clinical teaching. The ‘mapping’ of a learner’s progress towards the clinical teaching objectives by using multiple assessments over time, by multiple observers in both a systematic and opportunistic way correlates with the incremental nature of reaching clinical competence. Conclusions The importance of ITA based on both direct and indirect evidence of what the learner actually does in the real clinical setting is emphasized. Particular attention is given to addressing concerns in the more controversial areas of assessor training, ratings and documentation for ITA. Areas for future research are also identified.

Student perception of a new integrated anatomy practical program: Does students' prior learning make a difference?

While there is evidence that science and non-science background students display small differences in performance in basic and clinical sciences, early in a 4-year, graduate entry medical program, this lessens with time. With respect to anatomy knowledge, there are no comparable data as to the impact previous anatomy experience has on the student perception of the anatomy practical learning environment. A study survey was designed to evaluate student perception of the anatomy practical program and its impact on student learning, for the initial cohort of a new medical school. The survey comprised 19 statements requiring a response using a 5-point Likert scale, in addition to a free text opportunity to provide opinion of the perceived educational value of the anatomy practical program. The response rate for a total cohort of 82 students was 89%. The anatomy practical program was highly valued by the students in aiding their learning of anatomy, as indicated by the high mean scores for all statements (range: 4.04-4.7). There was a significant difference between the students who had and had not studied a science course prior to entering medicine, with respect to statements that addressed aspects of the course related to its structure, organization, variety of resources, linkage to problem-based learning cases, and fairness of assessment. Nonscience students were more positive compared to those who had studied science before (P levels ranging from 0.004 to 0.035). Students less experienced in anatomy were more challenged in prioritizing core curricular knowledge. © 2011 Wiley-Liss, Inc.

Preparing health science students for interdisciplinary professional practice

In 2002, a number of lecturers from different clinical schools within the Faculty of Health Sciences at La Trobe University embarked on the development of a new interdisciplinary professional practice subject to be undertaken by all final-year undergraduate health science students. The subject was designed to better prepare students for their first professional appointment by introducing them to the concepts of interdisciplinary teamwork, the health care context, and the challenges and constraints that organizational contexts present. This report details the background of the project, the consultation and development that took place in the design of the subject, and implementation of the subject. The uniqueness of the project is explained by the number of disciplines involved, the online delivery, and the focus on a set of generic graduate attributes for health science students. It is hoped that students who have undertaken this subject will have a better understanding of the roles of other health professionals and the context in which they will be working by grappling with many real-life professional issues that they will face when they graduate and enter the workforce.

WriteSim TCExam--an open source text simulation environment for training novice researchers in scientific writing.

BACKGROUND: The ability to write clearly and effectively is of central importance to the scientific enterprise. Encouraged by the success of simulation environments in other biomedical sciences, we developed WriteSim TCExam, an open-source, Web-based, textual simulation environment for teaching effective writing techniques to novice researchers. We shortlisted and modified an existing open source application - TCExam to serve as a textual simulation environment. After testing usability internally in our team, we conducted formal field usability studies with novice researchers. These were followed by formal surveys with researchers fitting the role of administrators and users (novice researchers) RESULTS: The development process was guided by feedback from usability tests within our research team. Online surveys and formal studies, involving members of the Research on Research group and selected novice researchers, show that the application is user-friendly. Additionally it has been used to train 25 novice researchers in scientific writing to date and has generated encouraging results. CONCLUSION: WriteSim TCExam is the first Web-based, open-source textual simulation environment designed to complement traditional scientific writing instruction. While initial reviews by students and educators have been positive, a formal study is needed to measure its benefits in comparison to standard instructional methods.

How teachers of Geography and History of Secondary Education assess?

The purpose of this article is to analyse the assessment procedures and instruments used by teachers of Geography and History of Compulsory Secondary School (ESO) in the Region of Murcia (Spain). The data have been extracted implementing a survey technique proceeded by a descriptive analysis. The results show that teachers generally have a traditional conception of assessment, reflected in the fact that they think that assessment should not change when teaching strategies are changed or when they innovate. On the other hand, although they consider that is necessary to employ a variety of instruments to assess well and to prevent school failure, they still use exams as the most objective and essential instrument in the assessment, while they don’t apply continuous assessment, only tests in a continuous way. The implementation of similar research in other areas or in other subjects shows the existence of contrasts in teacher assessment practices.

The role of personality and self-efficacy in the selection and retention of successful nursing students:a longitudinal study

Aim: This paper is a report of a study to examine the role of personality and self-efficacy in predicting academic performance and attrition in nursing students.

Background: Despite a considerable amount of research investigating attrition in nursing students and new nurses, concerns remain. This particular issue highlights the need for a more effective selection process whereby those selected are more likely to complete their preregistration programme successfully, and remain employed as Registered Nurses.

Method: A longitudinal design was adopted. A questionnaire, which included measures of personality and occupational and academic self-efficacy, was administered to 384 students early in the first year of the study. At the end of the programme, final marks and attrition rates were obtained from university records for a total of 350 students. The data were collected from 1999 to 2002.

Findings: Individuals who scored higher on a psychoticism scale were more likely to withdraw from the programme. Occupational self-efficacy was revealed to be a statistically significant predictor of final mark obtained, in that those with higher self-efficacy beliefs were more likely to achieve better final marks. Extraversion was also shown to negatively predict academic performance in that those with higher extraversion scores were more likely to achieve lower marks.

Conclusion: More research is needed to explore the attributes of successful nursing students and the potential contribution of psychological profiling to a more effective selection process.