Assessment of medical courses in Brazil using student-completed questionnaires. Is it reliable?

INTRODUCTION: Debates about the quality of medical education have become more evident in the recent past, and as a result several different assessment methods have been refined for that purpose. The use of questionnaires filled out by medical students to assess the quality of lectures is one of the most common methods employed in our milieu. However, the reliability of this investigation method has not yet been systematically tested. The authors present the reliability of a specific form applied to the fourth grade medical students during the clinical psychiatry course. METHOD: Eighty-one fourth grade medical students were instructed to complete a form immediately after each clinical psychiatry lecture. Thirty-four students (42%) failed to turn in the forms after the final lecture. These students were given an identical form to assess the lectures in a retrospective fashion. The grades given by both groups of students for each performed lecture and the number of students who have graded an unperformed lecture were compared. Statistical significance for both groups was determined by means of the chi-square test (p< 0.05). RESULTS: Eighteen out of the 34 students who filled out the forms retrospectively (53%) rated the unperformed lecture, whereas only 5 out of the 47 students who filled out the forms during the course (11%) did so. This is statistically significant (p< 0.05). There was no statistical difference for the grades given to the lectures that were actually performed. DISCUSSION: The authors concluded the low reliability rate of the retrospective evaluation warrant a continuous assessment method during the course.

Assessing student individual performance within PBL teams: findings from the implementation of a new mechanism

This paper reports on the experience of the implementation of a new mechanism to assess individual student contribution within project work, where students work in teams to solve a large-scale open-ended interdisciplinary project. The study takes place at the University of Minho, with first year engineering students, enrolled in the Industrial Management and Engineering (Integrated Masters) degree. The aim of this paper is to describe the main principles and procedures underlying the assessment mechanism created and also provide some feedback from its first implementation, based on the students, lecturers and tutors perceptions. For data collection, a survey was sent to all course lecturers and tutors involved in the assessment process. Students also contributed with suggestions, both on a workshop held at the end of the project and also by answering a survey on the overall satisfaction with PBL experience. Findings show a positive level of acceptance of the new mechanism by the students and also by the lecturers and tutors. The study identified the need to clarify the criteria used by the lecturers and the exact role of the tutor, as well as the need for further improvement of its features and procedures. Some recommendations are also issued regarding technical aspects related to some of the steps of the procedures, as well as the need for greater support on the adjustment and final setting of the individual grades.

Assessment of Autonomic Function by Phase Rectification of RRInterval Histogram Analysis in Chagas Disease

Background:In chronic Chagas disease (ChD), impairment of cardiac autonomic function bears prognostic implications. Phase‑rectification of RR-interval series isolates the sympathetic, acceleration phase (AC) and parasympathetic, deceleration phase (DC) influences on cardiac autonomic modulation.Objective:This study investigated heart rate variability (HRV) as a function of RR-interval to assess autonomic function in healthy and ChD subjects.Methods:Control (n = 20) and ChD (n = 20) groups were studied. All underwent 60-min head-up tilt table test under ECG recording. Histogram of RR-interval series was calculated, with 100 ms class, ranging from 600–1100 ms. In each class, mean RR-intervals (MNN) and root-mean-squared difference (RMSNN) of consecutive normal RR-intervals that suited a particular class were calculated. Average of all RMSNN values in each class was analyzed as function of MNN, in the whole series (RMSNNT), and in AC (RMSNNAC) and DC (RMSNNDC) phases. Slopes of linear regression lines were compared between groups using Student t-test. Correlation coefficients were tested before comparisons. RMSNN was log-transformed. (α < 0.05).Results:Correlation coefficient was significant in all regressions (p < 0.05). In the control group, RMSNNT, RMSNNAC, and RMSNNDCsignificantly increased linearly with MNN (p < 0.05). In ChD, only RMSNNAC showed significant increase as a function of MNN, whereas RMSNNT and RMSNNDC did not.Conclusion:HRV increases in proportion with the RR-interval in healthy subjects. This behavior is lost in ChD, particularly in the DC phase, indicating cardiac vagal incompetence.

RQIA Review of the implementation of Promoting Quality Care (PQC) Good Practice Guidance on the Assessment and Management of Risk in Mental Health and Learning Disability Services (PDF1.07MB)

Overview Report October 2012

Assessment of left coronary artery beat-to-beat repositioning in order to propose an optimal acquisition window for coronary MRA

Introduction: Coronary magnetic resonance angiography (MRA) is a medical imaging technique that involves collecting data from consecutive heartbeats, always at the same time in the cardiac cycle, in order to minimize heart motion artifacts. This technique relies on the assumption that coronary arteries always follow the same trajectory from heartbeat to heartbeat. Until now, choosing the acquisition window in the cardiac cycle was based exclusively on the position of minimal coronary motion. The goal of this study was to test the hypothesis that there are time intervals during the cardiac cycle when coronary beat-to-beat repositioning is optimal. The repositioning uncertainty values in these time intervals were then compared with the intervals of low coronary motion in order to propose an optimal acquisition window for coronary MRA. Methods: Cine breath-hold x-ray angiograms with synchronous ECG were collected from 11 patients who underwent elective routine diagnostic coronarography. Twenty-three bifurcations of the left coronary artery were selected as markers to evaluate repositioning uncertainty and velocity during cardiac cycle. Each bifurcation was tracked by two observers, with the help of a user-assisted algorithm implemented in Matlab (The Mathworks, Natick, MA, USA) that compared the trajectories of the markers coming from consecutive heartbeats and computed the coronary repositioning uncertainty with steps of 50ms until 650ms after the R-wave. Repositioning uncertainty was defined as the diameter of the smallest circle encompassing the points to be compared at the same time after the R-wave. Student's t-tests with a false discovery rate (FDR, q=0.1) correction for multiple comparison were applied to see whether coronary repositioning and velocity vary statistically during cardiac cycle. Bland-Altman plots and linear regression were used to assess intra- and inter-observer agreement. Results: The analysis of left coronary artery beat-to-beat repositioning uncertainty shows a tendency to have better repositioning in mid systole (less than 0.84±0.58mm) and mid diastole (less than 0.89±0.6mm) than in the rest of the cardiac cycle (highest value at 50ms=1.35±0.64mm). According to Student's t-tests with FDR correction for multiple comparison (q=0.1), two intervals, in mid systole (150-200ms) and mid diastole (550-600ms), provide statistically better repositioning in comparison with the early systole and the early diastole. Coronary velocity analysis reveals that left coronary artery moves more slowly in end systole (14.35±11.35mm/s at 225ms) and mid diastole (11.78±11.62mm/s at 625ms) than in the rest of the cardiac cycle (highest value at 25ms: 55.96±22.34mm/s). This was confirmed by Student's t-tests with FDR correction for multiple comparison (q=0.1, FDR-corrected p-value=0.054): coronary velocity values at 225, 575 and 625ms are not much different between them but they are statistically inferior to all others. Bland-Altman plots and linear regression show that intra-observer agreement (y=0.97x+0.02 with R²=0.93 at 150ms) is better than inter-observer (y=0.8x+0.11 with R²=0.67 at 150ms). Discussion: The present study has demonstrated that there are two time intervals in the cardiac cycle, one in mid systole and one in mid diastole, where left coronary artery repositioning uncertainty reaches points of local minima. It has also been calculated that the velocity is the lowest in end systole and mid diastole. Since systole is less influenced by heart rate variability than diastole, it was finally proposed to test an acquisition window between 150 and 200ms after the R-wave.

A teaching/learning support tool for introductory programming courses

In This work we present a Web-based tool developed with the aim of reinforcing teaching and learning of introductory programming courses. This tool provides support for teaching and learning. From the teacher's perspective the system introduces important gains with respect to the classical teaching methodology. It reinforces lecture and laboratory sessions, makes it possible to give personalized attention to the student, assesses the degree of participation of the students and most importantly, performs a continuous assessment of the student's progress. From the student's perspective it provides a learning framework, consisting in a help environment and a correction environment, which facilitates their personal work. With this tool students are more motivated to do programming

Electronic voting system for computer supported collaborative learning

The EVS4CSCL project starts in the context of a Computer Supported Collaborative Learning environment (CSCL). Previous UOC projects created a CSCL generic platform (CLPL) to facilitate the development of CSCL applications. A discussion forum (DF) was the first application developed over the framework. This discussion forum was different from other products on the marketplace because of its focus on the learning process. The DF carried out the specification and elaboration phases from the discussion learning process but there was a lack in the consensus phase. The consensus phase in a learning environment is not something to be achieved but tested. Common tests are done by Electronic Voting System (EVS) tools, but consensus test is not an assessment test. We are not evaluating our students by their answers but by their discussion activity. Our educational EVS would be used as a discussion catalyst proposing a discussion about the results after an initial query or it would be used after a discussion period in order to manifest how the discussion changed the students mind (consensus). It should be also used by the teacher as a quick way to know where the student needs some reinforcement. That is important in a distance-learning environment where there is no direct contact between the teacher and the student and it is difficult to detect the learning lacks. In an educational environment, assessment it is a must and the EVS will provide direct assessment by peer usefulness evaluation, teacher marks on every query created and indirect assessment from statistics regarding the user activity.

Cross-validation of bioelectrical impedance analysis for the assessment of body composition in a representative sample of 6- to 13-year-old children.

BACKGROUND/OBJECTIVES: (1) To cross-validate tetra- (4-BIA) and octopolar (8-BIA) bioelectrical impedance analysis vs dual-energy X-ray absorptiometry (DXA) for the assessment of total and appendicular body composition and (2) to evaluate the accuracy of external 4-BIA algorithms for the prediction of total body composition, in a representative sample of Swiss children. SUBJECTS/METHODS: A representative sample of 333 Swiss children aged 6-13 years from the Kinder-Sportstudie (KISS) (ISRCTN15360785). Whole-body fat-free mass (FFM) and appendicular lean tissue mass were measured with DXA. Body resistance (R) was measured at 50 kHz with 4-BIA and segmental body resistance at 5, 50, 250 and 500 kHz with 8-BIA. The resistance index (RI) was calculated as height(2)/R. Selection of predictors (gender, age, weight, RI4 and RI8) for BIA algorithms was performed using bootstrapped stepwise linear regression on 1000 samples. We calculated 95% confidence intervals (CI) of regression coefficients and measures of model fit using bootstrap analysis. Limits of agreement were used as measures of interchangeability of BIA with DXA. RESULTS: 8-BIA was more accurate than 4-BIA for the assessment of FFM (root mean square error (RMSE)=0.90 (95% CI 0.82-0.98) vs 1.12 kg (1.01-1.24); limits of agreement 1.80 to -1.80 kg vs 2.24 to -2.24 kg). 8-BIA also gave accurate estimates of appendicular body composition, with RMSE < or = 0.10 kg for arms and < or = 0.24 kg for legs. All external 4-BIA algorithms performed poorly with substantial negative proportional bias (r> or = 0.48, P<0.001). CONCLUSIONS: In a representative sample of young Swiss children (1) 8-BIA was superior to 4-BIA for the prediction of FFM, (2) external 4-BIA algorithms gave biased predictions of FFM and (3) 8-BIA was an accurate predictor of segmental body composition.

Capitalism in the classroom : the impact of self-enhancement values and normative assessment on student motivation, goals and attitudes towards cheating

Abstract The neo-liberal capitalist ideology has come under heavy fire with anecdotal evidence indicating a link between these same values and unethical behavior. Academic institutions reflect social values and act as socializing agents for the young. Can this explain the high and increasing rates of cheating that currently prevail in education? Our first chapter examines the question of whether self-enhancement values of power and açhievement, the individual level equivalent of neo-liberal capitalist values, predict positive attitudes towards cheating. Furthermore, we explore the mediating role of motivational factors. Results of four studies reveal that self-enhancement value endorsement predicts the adoption of performance-approach goals, a relationship mediated by introjected regulation, namely desire for social approval and that self-enhancement value endorsement also predicts the condoning of cheating, a relationship mediated by performance-approach goal adoption. However, self-transcendence values prescribed by a normatively salient source have the potential to reduce the link between self-enhancement value endorsément and attitudes towards cheating. Normative assessment constitutes a key tool used by academic institutions to socialize young people to accept the competitive, meritocratic nature of a sociéty driven by a neo-liberal capitalist ideology. As such, the manifest function of grades is to motivate students to work hard and to buy into the competing ethos. Does normative assessment fulfill these functions? Our second chapter explores the reward-intrinsic motivation question in the context of grading, arguably a high-stakes reward. In two experiments, the relative capacity of graded high performance as compared to the task autonomy experienced in an ungraded task to predict post-task intrinsic motivation is assessed. Results show that whilst the graded task performance predicts post-task appreciation, it fails to predict ongoing motivation. However, perceived autonomy experienced in non-graded condition, predicts both post-task appreciation and ongoing motivation. Our third chapter asks whether normative assessment inspires the spirit of competition in students. Results of three experimental studies reveal that expectation of a grade for a task, compared to no grade, induces greater adoption of performance-avoidance, but not performance-approach, goals. Experiment 3 provides an explanatory mechanism for this, showing that reduced autonomous motivation experienced in previous graded tasks mediates the relationship between grading and adoption of performance avoidance goals in a subsequent task. The above results, when combined, provide evidence as to the deleterious effects of self enhancement values and the associated practice of normative assessment in school on student motivation, goals and ethics. We conclude by using value and motivation theory to explore solutions to this problem.

Adaptive integral assessment package for the A2UN@ project

This paper presents a first approach of Evaluation Engine Architecture (EEA) as proposal to support adaptive integral assessment, in the context of a virtual learning environment. The goal of our research is design an evaluation engine tool to assist in the whole assessment process within the A2UN@ project, linking that tool with the other key elements of a learning design (learning task, learning resources and learning support). The teachers would define the relation between knowledge, competencies, activities, resources and type of assessment. Providing this relation is possible obtain more accurate estimations of student's knowledge for adaptive evaluations and future recommendations. The process is supported by usage of educational standards and specifications and for an integral user modelling

Assessment and reuse of contents in the competence-based educational platform InterMediActor

This paper describes a failure alert system and a methodology for content reuse in a new instructional design system called InterMediActor (IMA). IMA provides an environment for instructional content design, production and reuse, and for students’ evaluation based in content specification through a hierarchical structure of competences. The student assessment process and information extraction process for content reuse are explained.

Assessment of the teaching-learning process in students of the health area: cardiopulmonary resuscitation maneuvers

OBJECTIVETo evaluate the skills and knowledge of undergraduate students in the health area on cardiopulmonary resuscitation maneuvers with the use of an automatic external defibrillator.METHODThe evaluation was performed in three different stages of the teaching-learning process. A theoretical and practical course was taught and the theoretical classes included demonstration. The evaluation was performed in three different stages of the teaching-learning process. Two instruments were applied to evaluate the skills (30-items checklist) and knowledge (40-questions written test). The sample comprised 84 students.RESULTSAfter the theoretical and practical course, an increase was observed in the number of correct answers in the 30-items checklist and 40-questions written test.CONCLUSIONAfter the theoretical class (including demonstration), only one of the 30-items checklist for skills achieved an index ≥ 90% of correct answers. On the other hand, an index of correct answers greater than 90% was achieved in 26 (86.7%) of the 30 items after a practical training simulation, evidencing the importance of this training in the defibrillation procedure.

Student experiences in a course blended learning

Fascination is a project for design students, developed jointly by the Technical University of Catalonia (UPC) and the University of Technology Thonburi (KMUTT), which works with two groups of students, one group of participants in Spain and another group in Thailand where, hands-on activities, a range of technologies are used to prepare students for the lessons, through learning activities and content. This research paper presents the test of both a general model and a tool for measuring the participants’ experiences in a course that uses a blended learning methodology, with the aim of collecting empirical evidence to justify the effort of applying this methodology, based on the participants’ satisfaction. The procedure used in the conceptualization of the survey, the generation of topics, the collection of data, and the validation of the scale of various items are described here. The information, provided by the 26 people surveyed about the course and the virtual environment that was used, was analyzed to measure their perceptions and explore possible relations. Finally the conclusions of the research and the future work are presented.