Assessing self- and peer-assessment: The students' views

Self- and peer-assessment are being used increasingly in higher education, to help assign grades to students' work and to help students to learn more effectively. However, in spite of this trend there is little in the published literature on how students view these methods. In this paper we present an analysis of the views of a large number of students (N = 233) who had just experienced self- and peer-feedback as part of one of their subjects. It is a rarely questioned commonplace in the literature that in order to gain benefit from peer and self-assessment schemes students first need training in the specific scheme being used; ideally they will play a role in devising the scheme. The intervention reported here, which involved a large (N = 233) group of students, included no such measures. The results show that students felt, nonetheless, that they benefited from the intervention. The results also present prima facie evidence that training or other measures to further involve the students in the peer and self-assessment scheme might be beneficial. Our analysis of students' views revealed eight general dimensions under which are grouped twenty higher order themes. The results both support and extend previous research and give a more detailed picture than previously available. The general dimensions found were: Difficult; Gained Better Understanding of Marking; Discomfort; Productive (including learning benefits and improved work); Problems with Implementation; Read Others' Work; Develop Empathy (with assessing staff); and, Motivation (especially motivation to impress peers). The practical implications of these findings are discussed.

Peer assessment in ULPGC on line CLIL course 2010-2013

[EN]Introduction to Content Language and Integrated Learning ( CLIL) is a 60 hour on line course offered by Universidad de Las Palmas de Gran Canaria within its extracurricular courses. During three years we have had 76 students whose final assessment has been to create a CLIL lesson which was evaluated by their peers. The tool for the peer assessment has been the Moodle platform; forums. The assessment was guided by a list of questions they had to apply when evaluating the lessons, but the replies from the CLIL lesson creators did not have any predetermined structure or guidelines to be followed. We have analysed the assessments and replies by grouping them according to content similarities.

Il trattamento ortodontico nei bambini con particolari necessità sanitarie (SHCN): una valutazione della durata e del risultato clinico utilizzando l'indice PAR (Peer Assessment Rating), la componente DHC (Dental Health Component) e la componente AC (Aesthetic Component) dell'indice IOTN (Orthodontic Treatment Need Index)

Obbiettivo: Valutazione delle eventuali differenze nel trattamento ortodontico di un gruppo di bambini con particolari necessità sanitarie (SHCN) rispetto ad un gruppo di bambini non diagnosticati con SHCN. Materiali e Metodi: Il gruppo campione (SHCN) è costituito da 50 bambini con SHCN. Il gruppo di controllo (NO SHCN) è costituito da 50 bambini non diagnosticati con SHCN pienamente corrispondenti per età, genere e tipo di apparecchio ortodontico utilizzato con i pazienti del gruppo di studio. I dati riguardanti i gruppi SHCN e NO SHCN sono stati analizzati in modo retrospettivo, valutando: - il punteggio pre- e post-trattamento e la riduzione finale dei valori dell'indice PAR (Peer Assessment Rating), della componente DHC (Dental Health Component) e della componente AC (Aesthetic Component) dell'indice IOTN (Orthodontic Treatment Need Index), - il numero di appuntamenti, - il numero di sedute semplici e complesse, - la durata complessiva del trattamento, - l'età all’inizio ed alla fine della terapia. Risultati: Non sono state rilevate differenze statisticamente significative tra i due gruppi per quanto concerne il numero di appuntamenti, la durata complessiva del trattamento, l'età all’inizio ed alla fine della terapia ortodontica (valori del p-value:0.682, 0.458, 0.535, 0.675). Sono state rilevate differenze statisticamente significative tra i due gruppi per quanto riguarda i punteggi dell’indice PAR, delle componenti DHC e AC dello IOTN pre- e post-trattamento, il numero di sedute semplici e complesse (valori del p-value:0.030, 0.000, 0.020, 0.023, 0.000, 0.000, 0.043, 0.037). Per quanto concerne la riduzione finale del valore dell’indice PAR, della componente DHC e di quella AC dello IOTN non sono state riscontrate differenze statisticamente significative tra i due gruppi (valori del p-value:0.060, 0.765, 0.825). Conclusioni: Lo studio incoraggia gli ortodontisti a trattare i bambini con SHCN nell'obiettivo di migliorarne la qualità di vita, pur evidenziando la necessità di un maggior numero di sedute complesse.

Formatives Selbst- und Peer-Assessment im Unterricht peripher venöser Venenpunktion

Das Lernen einer Fertigkeit durch Demonstration und anschließendes Üben wird „Modeling“ genannt. Es basiert darauf, die Diskrepanz zwischen dem Soll-Zustand (Demonstration) und dem Ist-Zustand (Üben) zu erkennen und zu beheben. Dafür ist die exakte Analyse der eigenen Fertigkeiten beim Üben unentbehrlich. Entsprechend ist auch bekannt, dass formative Evaluationen wesentlich zum erfolgreichen Lernen beitragen. Wir haben deshalb im Kurs für periphere Venenpunktion im 3. Studienjahr formatives Selbst- und Peer-Assessment eingeführt. Die Struktur des Assessment entspricht einem DOPS (dircect observation of procedural skills). DOPS stammt aus dem Arbeitsplatz-basieren Assessment und beinhaltet die Beurteilung folgender Kriterien: Vorbereitung/Nachsorge, technische Fertigkeit, Asepsis/Sicherheit, klinische Urteilsfähigkeit, Organisation/Effizienz, professionelles Verhalten, Gesamteindruck. Diese Kriterien wurden für den Unterricht konkretisiert (z.B. Vorbereitung mit Beschriftung der Röhrchen, etc.) und den Studierenden als Merkblätter ausgeteilt. Die Studierenden beurteilten ihre eigene Performance bzw. die eines Kommilitonen, gaben sich Feedback und legten individuelle Lernziele zur Verbesserung fest. Dieses Vorgehen hat den Vorteil, dass sowohl der Übende, als auch der beobachtende Kommilitone, die optimale Ausführung der jeweiligen Tätigkeit reflektieren, welches für beide eine Möglichkeit zum Lernen bietet . Bei der Evaluation des Kurses wurden die Handouts mit den Kriterien der DOPS von Teilnehmern von 9 der 10 Gruppen positiv erwähnt. Im Rahmen eines Debriefing mit den studentischen Tutoren wurde jedoch kritisch angemerkt, dass der Prozess der formativen Selbst- und Fremdevaluation den Studierenden im 3. Studienjahr nicht vertraut war. Es war für die Teilnehmer schwierig konkretes Feedback zu geben und individuelle Lernziele festzulegen. Für das kommende Jahr planen wir in Bezug auf den Kurs folgendes: Die Kriterien der korrekten Durchführung einer Fertigkeit zu formulieren wird von den Teilnehmern als hilfreich empfunden und soll deshalb beibehalten werden. Die Studierenden, die dieses Jahr an dem Kurs teilnehmen, haben bereits ein Feedbacktraining absolviert. Der Kurs kann deshalb neu an Vorkenntnisse anknüpfen. Darüber hinaus soll der Prozess der Festlegung der individuellen Lernziele in der Schulung der studentischen Tutoren des Kurses mehr Gewicht erhalten, damit die Tutoren die Teilnehmer hier gezielt unterstützen können.

Using multimedia and peer assessment to promote collaborative e-learning

Collaborative e-learning is increasingly appealing as a pedagogical approach that can positively affect student learning. We propose a didactical model that integrates multimedia with collaborative tools and peer assessment to foster collaborative e-learning. In this paper, we explain it and present the results of its application to the “International Seminars on Materials Science” online course. The proposed didactical model consists of five educational activities. In the first three, students review the multimedia resources proposed by the teacher in collaboration with their classmates. Then, in the last two activities, they create their own multimedia resources and assess those created by their classmates. These activities foster communication and collaboration among students and their ability to use and create multimedia resources. Our purpose is to encourage the creativity, motivation, and dynamism of the learning process for both teachers and students.

Peer Assessment and Self-assessment: Effective Learning Tools in Higher Education.

When used appropriately, self- and peer-assessment are very effective learning tools. In the present work, instructor formative assessment and feedback, self-assessment (SA), and peer-assessment (PA) have been compared. During the first part of a semester, the students followed a continuous formative assessment. Subsequently, they were divided into two subgroups based on similar performances. One subgroup performed SAs, and the other followedPAduring the last part of the course. The performances of the two groups in solving problems were compared. Results suggest that PA is a more effective learning tool than SA, and both are more effective than instructor formative assessment. However, a survey that was conducted at the end of the experiment showed higher student confidence in instructor assessment than in PA. The students recognized the usefulness of acting as peer assessors, but believed that SA helped them more than PA.

Using online peer assessment tools to provide feedback and promote learning

A set of slides used for the RAP SIG event on 19 Jan 2017

Reliability assessment of a peer evaluation instrument in a team-based learning course

Objective: To evaluate the reliability of a peer evaluation instrument in a longitudinal team-based learning setting. Methods: Student pharmacists were instructed to evaluate the contributions of their peers. Evaluations were analyzed for the variance of the scores by identifying low, medium, and high scores. Agreement between performance ratings within each group of students was assessed via intra-class correlation coefficient (ICC). Results: We found little variation in the standard deviation (SD) based on the score means among the high, medium, and low scores within each group. The lack of variation in SD of results between groups suggests that the peer evaluation instrument produces precise results. The ICC showed strong concordance among raters. Conclusions: Findings suggest that our student peer evaluation instrument provides a reliable method for peer assessment in team-based learning settings.

Assessment of Young Learners’ English Proficiency in Bilingual Content Instruction CLIL

This applied linguistic study in the field of second language acquisition investigated the assessment practices of class teachers as well as the challenges and visions of language assessment in bilingual content instruction (CLIL) at primary level in Finnish basic education. Furthermore, pupils’ and their parents’ perceptions of language assessment and LangPerform computer simulations as an alternative, modern assessment method in CLIL contexts were examined. The study was conducted for descriptive and developmental purposes in three phases: 1) a CLIL assessment survey; 2) simulation 1; and 3) simulation 2. All phases had a varying number of participants. The population of this mixed methods study were CLIL class teachers, their pupils and the pupils’ parents. The sampling was multi-staged and based on probability and random sampling. The data were triangulated. Altogether 42 CLIL class teachers nationwide, 109 pupils from the 3rd, 4th and 5th grade as well as 99 parents from two research schools in South-Western Finland participated in the CLIL assessment survey followed by an audio-recorded theme interview of volunteers (10 teachers, 20 pupils and 7 parents). The simulation experimentations 1 and 2 produced 146 pupil and 39 parental questionnaires as well as video interviews of volunteered pupils. The data were analysed both quantitatively using percentages and numerical frequencies and qualitatively employing thematic content analysis. Based on the data, language assessment in primary CLIL is not an established practice. It largely appears to be infrequent, incidental, implicit and based on impressions rather than evidence or the curriculum. The most used assessment methods were teacher observation, bilingual tests and dialogic interaction, and the least used were portfolios, simulations and peer assessment. Although language assessment was generally perceived as important by teachers, a fifth of them did not gather assessment information systematically, and 38% scarcely gave linguistic feedback to pupils. Both pupils and parents wished to receive more information on CLIL language issues; 91% of pupils claimed to receive feedback rarely or occasionally, and 63% of them wished to get more information on their linguistic coping in CLIL subjects. Of the parents, 76% wished to receive more information on the English proficiency of their children and their linguistic development. This may be a response to indirect feedback practices identified in this study. There are several challenges related to assessment; the most notable is the lack of a CLIL curriculum, language objectives and common ground principles of assessment. Three diverse approaches to language in CLIL that appear to affect teachers’ views on language assessment were identified: instrumental (language as a tool), dual (language as a tool and object of learning) and eclectic (miscellaneous views, e.g. affective factors prioritised). LangPerform computer simulations seem to be perceived as an appropriate alternative assessment method in CLIL. It is strongly recommended that the fundamentals for assessment (curricula and language objectives) and a mutual assessment scheme should be determined and stakeholders’ knowledge base of CLIL strengthened. The principles of adequate assessment in primary CLIL are identified as well as several appropriate assessment methods suggested.

A Teacher’s Formative Assessment Perceptions and Practices in Oral Intermediate English Courses at the Université de Montréal

Au cours des dernières années, les pratiques de l’évaluation comme aide à l’apprentissage (c’est-à-dire l’auto-évaluation, l’évaluation par les pairs, la rétroaction) dans la salle de classe ont été de plus en plus considérées comme des éléments essentiels (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). Cependant, dans le domaine de l’apprentissage d’une langue seconde la recherche sur ce sujet est plutôt limitée. En nous fondant sur les études de Colby-Kelly et Turner (2007) et de Lyster et Ranta (1997), nous avons mené une recherche exploratoire visant à combler ce besoin. L’objectif général était de comprendre comment l’évaluation formative se réalise dans deux cours d’anglais intermédiaire à l’Université de Montréal, et de comparer comment ces pratiques sont perçues et vécues par la professeure et ses étudiants. Trois questions de recherche étaient posées: 1. Quelle est la nature de l’évaluation formative dans une salle de classe d’anglais langue seconde? 2. Est-ce que les pratiques de la professeure reflètent ce qu’elle pense de l’évaluation formative? 3. Quels sont les correspondances et différences entre les croyances de la professeure et les perceptions de ses étudiants quant aux bénéfices de l’évaluation formative pour l’apprentissage de l’anglais langue seconde? La collecte de données comprend des observations en classe, des entrevues auprès de la professeure et des questionnaires pour les étudiants. Pour répondre à notre première question de recherche, nous avons analysé la rétroaction donnée par la professeure pendant une production orale réalisée par ses étudiants à partir de la typologie de feedback et de uptake de l’étude de Lyster et Ranta (1997). En ce qui a trait à la deuxième question de recherche, nous avons fait des entrevues avec la professeure en vue d’expliciter ses perceptions relativement à l’évaluation formative. À la fin du trimestre, nous avons comparé ses réponses avec sa pratique à partir des enregistrements vidéo produits en classe. Finalement, pour répondre à notre dernière question de recherche nous avons comparé les réponses données par la professeure aux entrevues avec les réponses des étudiants à un questionnaire adapté à partir de celui de Colby-Kelly et Turner. Finalement, nous présentons et discutons les résultats les plus significatifs obtenus dans cette étude qualitative Nous concluons cette thèse en proposant de avenues pour des recherches futures.

Assessment and Feedback for the CIP: educational issues

This 11-minute video focuses on how self and peer assessment could be used as an integral part of CIP modules, and discusses some innovative methods for assessing written work. The first link is to the native Panopto podcast, which requires the Silverlight player to be installed. The second link is the MP4 video version of the podcast, which should play on all PCs, Macs and suitable mobile devices. This share also includes links to the papers discussed in the video - these are also provided at the end of the script.

PAR evaluation of treated Class I extraction patients

Objective: To evaluate treatment changes and quality of finishing occlusion in Class I patients treated with four premolar extractions. Material and Methods: Dental casts of 94 subjects (50 males and 44 females) were evaluated. Mean pretreatment age was 13.46 years, and mean treatment time was 2.09 years. The peer assessment rating (PAR) index was obtained from pretreatment and posttreatment dental casts. Results: The mean pretreatment PAR index of 29.46 was reduced to 6.32 at posttreatment stage, achieving a reduction of 78.54% with treatment. There was correlation between the initial PAR and correction during treatment, that is, the more severe the malocclusion the greater the treatment changes. Conclusion: The cases evaluated showed a high-standard orthodontic finishing.