Development of a Real-time Formative Feedback Student Response System

This paper is focusing IT-supported real-time formative feedback in a classroom context. The development of a Student and Teacher Response System (STRS) is described. Since there are a number of obstacles for effective interaction in large classes, IT can be used to support the teachers aim to find out if students understand the lecture and accordingly adjust the content and design of the lecture. The system can be used for formative assessment before, during, and after a lecture. It is also possible for students to initiate interaction during lectures by posing questions anonymously. The main contributions of the paper are a) the description of the interactive real-time system and b) the development process behind it.

Student and teacher response system : development of an interactive anonymous real-timeformative feedback system

This paper is focusing IT-supported real-time formative feedback in a classroom context. The development of a Student and Teacher Response System (STRS) is described. Since there are a number of obstacles for effective interaction in large classes IT can be used to support the teachers aim to find out if students understand the lecture and accordingly adjust the content and design of the lecture. The system can be used for formative assessment before, during, and after a lecture. It is also possible for students to initiate interaction during lectures by posing questions anonymously. The main contributions of the paper are a) the description of the interactive real-time system and b) the development process behind it.

Interação em sala de aula de língua inglesa: o feedback como estratégia do desempenho assistido

This dissertation aims at characterizing the practices as well as the effects of a teacher s feedback in oral conversation interaction with students in an English Language classroom at a Primary School, 6th Grade in Açu/RN, Brazil. Therefore, this study is based on Vygotsky s (1975) and Bruner´s (1976) researches, which state that the learning process is constructed through interaction between a more experienced individual (teacher, parents and friends) and a learner who plays an active role, a re-constructor of knowledge. It is also based on Ur´s (2006) and Brookhart s (2008) studies (among other authors in Applied Linguistic) who defend that the feedback process needs to be evaluative and formative since it sets interfaces with both students autonomy and learning improvement. Our study is based on qualitative, quantitative and interpretive researches, whose natural environment (the classroom) is a direct source of data generated in this research through field observations/note-taking as well as through the transcriptions of five English classes audio taped. This study shows the following results: the teacher still seems to accept the patterns of interaction in the classroom that correspond to the IRE process (Initiation, Response, Evaluation) in behaviorist patterns: (1) he speaks and determines the turns of speech; (2) the teacher asks more questions and directs the activities most of the time; (3) the teacher´s feedback presents the following types: questioning, modeling, repeated response, praise, depreciation, positive/negative and sarcasm feedback, whose functions are to assess students' performance based on the rightness and wrongness of their responses. Thus, this implies to state that the feedback does not seem to help students improvement in terms of acquiring knowledge because of its normative effects/roles. Therefore, it is the teacher´s role to give evaluative and formative feedback to a student so that he/she should advance in the learning of the language and in the construction of knowledge

Flipped classroom and learning strategies

This paper seeks to answer the research question "How does the flipped classroom affect students’ learning strategies?" In e-learning research, several studies have focused on how students and teachers perceive the flipped classroom approach. In general, these studies have reported pleasing results. Nonetheless, few, if any, studies have attempted to find out the potential effects of the flipped classroom approach on how students learn. This study was based on two cases: 1) a business modelling course and 2) a research methodology course. In both cases, participating students were from information systems courses at Dalarna University in Sweden. Recorded lectures replaced regular lectures. The recorded lectures were followed by seminars that focused on the learning content of each lecture in various ways. Three weeks after the final seminar, we arranged for two focus group interviews to take place in each course, with 8 to 10 students participating in each group. We asked open questions on how the students thought they had been affected and more dedicated questions that were generated from a literature study on the effects of flipped classroom courses. These questions dealt with issues about mobility, the potential for repeating lectures, formative feedback, the role of seminars, responsibility, empowerment, lectures before seminars, and any problems encountered. Our results show that, in general, students thought differently about learning after the courses in relation to more traditional approaches, especially regarding the need to be more active. Most students enjoyed the mobility aspect and the accessibility of recorded lectures, although a few claimed it demanded a more disciplined attitude. Most students also expressed a feeling of increased activity and responsibility when participating in seminars. Some even felt empowered because they could influence seminar content. The length of and possibility to navigate in recorded lectures was also considered important. The arrangement of the seminar rooms should promote face-to-face discussions. Finally, the types of questions and tasks were found to affect the outcomes of the seminars. The overall conclusion with regard to students’ learning strategies is that to be an active, responsible, empowered, and critical student you have to be an informed student with possibilities and mandate to influence how, where and when to learn and be able to receive continuous feedback during the learning process. Flipped classroom can support such learning.

Quality Control for SP Performance Regarding Patient Portrayal in a High-Stakes OSCE

Introduction Since the quality of patient portrayal of standardized patients (SPs) during an Objective Structured Clinical Exam (OSCE) has a major impact on the reliability and validity of the exam, quality control should be initiated. Literature about quality control of SP’s performance focuses on feedback [1, 2] or completion of checklists [3, 4]. Since we did not find a published instrument meeting our needs for the assessment of patient portrayal, we developed such an instrument after being inspired by others [5] and used it in our high-stakes exam. Methods SP trainers from all five Swiss medical faculties collected and prioritized quality criteria for patient portrayal. Items were revised with the partners twice, based on experiences during OSCEs. The final instrument contains 14 criteria for acting (i.e. adequate verbal and non-verbal expression) and standardization (i.e. verbatim delivery of the first sentence). All partners used the instrument during a high-stakes OSCE. Both, SPs and trainers were introduced to the instrument. The tool was used in training (more than 100 observations) and during the exam (more than 250 observations). FAIR_OSCE The list of items to assess the quality of the simulation by SPs was primarily developed and used to provide formative feedback to the SPs in order to help them to improve their performance. It was therefore named “Feedbackstruckture for the Assessment of Interactive Role play in Objective Structured Clinical Exams (FAIR_OSCE). It was also used to assess the quality of patient portrayal during the exam. The results were calculated for each of the five faculties individually. Formative evaluation was given to the five faculties with individual feedback without revealing results of other faculties other than overall results. Results High quality of patient portrayal during the exam was documented. More than 90% of SP performances were rated to be completely correct or sufficient. An increase in quality of performance between training and exam was noted. In example the rate of completely correct reaction in medical tests increased from 88% to 95%. 95% completely correct reactions together with 4% sufficient reactions add up to 99% of the reactions meeting the requirements of the exam. SP educators using the instrument reported an augmentation of SPs performance induced by the use of the instrument. Disadvantages mentioned were high concentration needed to explicitly observe all criteria and cumbersome handling of the paper-based forms. Conclusion We were able to document a very high quality of SP performance in our exam. The data also indicate that our training is effective. We believe that the high concentration needed using the instrument is well invested, considering the observed augmentation of performance. The development of an iPad based application for the form is planned to address the cumbersome handling of the paper.

Bespoke Radiology e-Learning

Introduction: Foundation doctors are expected to assess and interpret plain x-ray studies of the chest/abdomen before a definitive report is issued by senior staff. The Royal College of Radiologists have published guidelines (RCR curriculum) on the scope of plain film findings medical students should be familiar with.1 Studies have shown that the x-ray interpretation without feedback does not significantly improve diagnostic ability. 2 Queen’s University, Belfast Trust Radiology and Experior Medical developed an online system to assess individual student ability to interpret X-ray findings. Over a series of assessments each student’s profile is built up, identifying strengths and weakness. The system can then create bespoke individual assessments re-evaluating previously identified weak areas and quantifying interpretative skill improvement. Aim: To determine how readily an online system is adopted by senior medical students, investigating if increasing exposure to x-ray interpretation combined with cyclical formative feedback enhances performance. Methods: The system was offered to all 270 final year medical students as an online resource. The system comprised a series of 20 weekly 30 minute assessments, containing normal and abnormal x-rays within the RCR curriculum. After each assessment students were given formative feedback, including their own result, annotated answers, peer group comparison and a breakdown of areas of strength and weakness. Focus groups of 4-5 students addressed student perspectives of the system, including ease of use, image resolution, system performance across different operating platforms, perceived value of formative feedback loops, breakdown of performance and the value of bespoke personalised assessments. Research Ethics Approval was granted for the study. Data analysis was via two-sided one-sample t-test; initial minimal recruitment was estimated as 60 students, to detect a mean 10% change in performance, with a standard deviation of 20%. Results and Discussion: Over 80% (n = XXX/270) of the student cohort engaged with the study. Student baseline average was 39%, increasing to 62% by the exit test. The steadily sustained improvement (57% relative performance in interpretative diagnostic accuracy) was despite increasing test difficulty. Student feedback via focus groups was universally positive throughout the examined domains. Conclusion: The online resource proved to be valuable, with high levels of student engagement, improving performance despite increasingly difficulty testing and positive learner experience with the system. References: 1. Undergraduate Radiology Curriculum, The Royal College of Ra, April 2012. Ref No. BFCR(12)4 The Royal College of Radiologists, April 2012 2. I Satia, S Bashagha, A Bibi, R Ahmed, S Mellor, F Zaman. Assessing the accuracy and certainty in interpretating chest x-rays in the medical division. Clin Med August 2013 Vol.13 no. 4 349-352

The Impact of Student Response Systems on the Learning Experience of Undergraduate Psychology Students

Student response systems (SRS) are hand-held devices or mobile phone polling systems which collate real-time, individual responses to on-screen questions. Previous research examining their role in higher education has highlighted both advantages and disadvantages of their use. This paper explores how different SRS influence the learning experience of psychology students across different levels of their programme. Across two studies, first year students’ experience of using Turningpoint clickers and second year students’ experience of using Poll Everywhere was investigated. Evaluations of both studies revealed that SRS has a number of positive impacts on learning, including enhanced engagement, active learning, peer interaction, and formative feedback. Technical and practical issues emerged as consistent barriers to the use of SRS. Discussion of these findings and the authors’ collective experiences of these technologies are used to provide insight into the way in which SRS can be effectively integrated within undergraduate psychology programmes.

Pachler N (2008) SCOPING A VISION FOR FORMATIVE E-ASSESSMENT - FEEDBACK Findings from the literature review

Pachler N (2008) SCOPING A VISION FOR FORMATIVE E-ASSESSMENT - FEEDBACK Findings from the literature review

The Comprehensive Assessment of Leadership for Learning: A Next Generation Formative Evaluation and Feedback System

Recent developments in federal policy have prompted the creation of state evaluation frameworks for principals and teachers that hold educators accountable for effective practices and student outcomes. These changes have created a demand for formative evaluation instruments that reflect current accountability pressures and can be used by schools to focus school improvement and leadership development efforts. The Comprehensive Assessment of Leadership for Learning (CALL) is a next generation, 360-degree on-line assessment and feedback system that reflect best practices in feedback design. Some unique characteristics of CALL include a focus on: leadership distributed throughout the school rather than as carried out by an individual leader; assessment of leadership tasks rather than perceptions of leadership practice; a focus on larger complex systems of middle and high school; and transparency of assessment design. This paper describes research contributing to the design and validation of the CALL survey instrument.

Formative assessment and motivation in higher education: a successful approach in mechanical engineer thermodynamics

Between 2000/01 and 2006/07, the approval rate of a Thermodynamics course in a Mechanical Engineer graduation was 25%. However, a careful analysis of the results showed that 41% of the students chosen not to attend or dropped out, missing the final examination. Thus, a continuous assessment methodology was developed, whose purpose was to reduce drop out, motivating students to attend this course, believing that what was observed was due, not to the incapacity to pass, but to the anticipation of the inevitability of failure by the students. If, on one hand, motivation is defined as a broad construct pertaining to the conditions and processes that account for the arousal, direction, magnitude, and maintenance of effort, on the other hand, assessment is one of the most powerful tools to change the will that students have to learn, motivating them to learn in a quicker and permanent way. Some of the practices that were implemented, included: promoting learning goal orientation rather than performance goal orientation; cultivating intrinsic interest in the subject and put less emphasis on grades but make grading criteria explicit; emphasizing teaching approaches that encourage collaboration among students and cater for a range of teaching styles; explaining the reasons for, and the implications of, tests; providing feedback to students about their performance in a form that is non-egoinvolving and non-judgemental and helping students to interpret it; broadening the range of information used in assessing the attainment of individual students. The continuous assessment methodology developed was applied in 2007/08 and 2008/09, having found an increase in the approval from 25% to 55% (30%), accompanied by a decrease of the drop out from 41% to 23,5% (17,5%). Flunking with a numerical grade lowered from 34,4% to 22,0% (12,4%). The perception by the students of the continuous assessment relevance was evaluated with a questionnaire. 70% of the students that failed the course respond that, nevertheless, didn’t repent having done the continuous assessment.

Feedback i oportunitats de millora: una avaluació orientada a l"aprenentatge actiu

This paper analyzes an innovative experience of formative assessment aimed at improving the teaching of Statistics, which could be easily extrapolated to other studies. We detail the implementation of the double correction, consisting of correcting students' work twice. With the first correction, carried out by classmates according to a rubric developed by the academic, possible errors or deficiencies are discovered, and students are provided with a feedback that allows them to correct and improve their work before being graded by the teacher; whereas in the second correction of the work, once upgraded, the professor evaluates and grades the work. As a result, there is a significant improvement in the quality of students" works, and an active learning from their own mistakes. Both contents and competencies are reinforced by the experience.

The Lived Experience of Nursing Students With Formative Assessment Formally Embedded in Clinical Courses: A Transcendental Phenomenological Study

The potential of formative assessment (FA) for informing learning in classroom-based nursing courses is clearly established in the literature; however, research on FA in clinical courses remains scarce. This inquiry explored the lived experience of nursing students using transcendental phenomenology and described the phenomenon of being assessed in clinical courses. The research question guiding the study was: How is the phenomenon of assessment experienced by nursing students when FA is formally embedded in clinical courses? Inherent in this question were the following issues: (a) the meaning of clinical experiences for nursing students, (b) the meaning of being assessed through FA, and (c) what it is like to be assessed when FA is formally embedded within clinical experiences. The noematic themes that illuminated the whatness of the participants’ experience were (a) enabled cognitive activity, (b) useful feedback, (c) freedom to be, (d) enhanced focus, (e) stress moderator, and (f) respectful mentorship. The noetic themes associated with how the phenomenon was experienced were related to bodyhood, temporality, spatiality, and relationship to others. The results suggest a fundamental paradigm shift from traditional nursing education to a more pervasive integration of FA in clinical courses so that students have time to learn before being graded on their practice. Furthermore, this inquiry and the literature consulted provide evidence that using cognitive science theory to inform and reform clinical nursing education is a timely option to address the repeated calls from nursing leaders to modernize nursing education. This inquiry contributes to reduce our reliance on assumptions derived from research on FA in nursing classrooms and provides evidence based on the reality of using formative assessment in clinical courses. Recommendations for future research are presented.

Investigating Formative Assessment: Exploring the Impact on the Self-Efficacy and Motivation of Mathematics Students

This is a study of the implementation and impact of formative assessment strategies on the motivation and self-efficacy of secondary school mathematics students. An explanatory sequential mixed methods design was implemented where quantitative and qualitative data were collected and analyzed sequentially in 2 different phases. The first phase involved quantitative data from student questionnaires and the second phase involved qualitative data from individual student and teacher interviews. The findings of the study suggest that formative assessment is implemented in practice in diverse ways and is a process where the strategies are interconnected. Teachers experience difficulty in incorporating peer and self-assessment and perceive a need for exemplars. Key factors described as influencing implementation include teaching philosophies, interpretation of ministry documents, teachers’ experiences, leadership in administration and department, teacher collaboration, misconceptions of teachers, and student understanding of formative assessment. Findings suggest that overall, formative assessment positively impacts student motivation and self-efficacy, because feedback is provided which offers encouragement and recognition by highlighting the progress that has been made and what steps need to be taken to improve. However, students are impacted differently with some considerations including how students perceive mistakes and if they fear judgement. Additionally, the impact of formative assessment is influenced by the connection between self-efficacy and motivation, namely how well a student is doing is a source of both concepts.