Pachler N (2008) SCOPING A VISION FOR FORMATIVE E-ASSESSMENT - FEEDBACK Findings from the literature review

Pachler N (2008) SCOPING A VISION FOR FORMATIVE E-ASSESSMENT - FEEDBACK Findings from the literature review

A digital ecosystems model of assessment feedback on student learning

The term ecosystem has been used to describe complex interactions between living organisms and the physical world. The principles underlying ecosystems can also be applied to complex human interactions in the digital world. As internet technologies make an increasing contribution to teaching and learning practice in higher education, the principles of digital ecosystems may help us understand how to maximise technology to benefit active, self-regulated learning especially among groups of learners. Here, feedback on student learning is presented within a conceptual digital ecosystems model of learning. Additionally, we have developed a Web 2.0-based system, called ASSET, which incorporates multimedia and social networking features to deliver assessment feedback within the functionality of the digital ecosystems model. Both the digital ecosystems model and the ASSET system are described and their implications for enhancing feedback on student learning are discussed.

The therapeutic effects of Millon Clinical Multiaxial Inventory -III assessment feedback

A study was conducted to test the therapeutic effects of assessment feedback on rapport-building and self-enhancement variables (self-verification, self-discovery, self-esteem), as well as symptomatology. Assessment feedback was provided in the form of interpretive information based on the results of the Millon Clinical Multiaxial Inventory-III (MCMI-III). Participants (N = 89) were randomly assigned to three groups: a Feedback group, a Reflective-Counseling group, and a No-Feedback group. The Feedback group was provided with assessment feedback, the Reflective-Counseling group was asked to comment on the meaning of the taking the MCMI-III, the No-Feedback group received general information about the MCMI-III. Results revealed that assessment feedback, when provided in the form of interpretive interpretation positively affects rapport-building and self-enhancement variables (self-verification and self-discovery). No significant results were found in terms of self-esteem or symptom decrease as a function of feedback. However, a significant decrease in symptoms across groups was found. Results indicate that assessment feedback in the form of interpretive information can be used as a starting point in therapy. Implications of the findings are discussed with respect to theory and clinical practice. ^

Informational effects of assessment feedback

This study tested the effects of informational assessment feedback on satisfaction, selfesteem and examiner ratings. 83 participants completed a self-report personality inventory (Millon Index of Personality Styles). Participants of the experimental group were given standardized, informational assessment. Participants in the control group received only general information about the personality inventory. Significant group differences were found for the Feedback Assessment Questionnaire with t (81) = 11.67, p

The therapeutic effects of Millon Clinical Multiaxial Inventory-III assessment feedback

A study was conducted to test the therapeutic effects of assessment feedback on rapport-building and self-enhancement variables (self-verification, self-discovery, self-esteem), as well as symptomatology. Assessment feedback was provided in the form of interpretive information based on the results of the Millon Clinical Multiaxial Inventory- III (MCMI-III). Participants (N = 89) were randomly assigned to three groups: a Feedback group, a Reflective-Counseling group, and a No-Feedback group. The Feedback group was provided with assessment feedback, the Reflective-Counseling group was asked to comment on the meaning of the taking the MCMI-III, the No- Feedback group received general information about the MCMI-III. Results revealed that assessment feedback, when provided in the form of interpretive interpretation positively affects rapport-building and self-enhancement variables (self-verification and self-discovery). No significant results were found in terms of self-esteem or symptom decrease as a function of feedback. However, a significant decrease in symptoms across groups was found. Results indicate that assessment feedback in the form of interpretive information can be used as a starting point in therapy. Implications of the findings are discussed with respect to theory and clinical practice.

Objectifying the Subjective : Assessment and Feedback in Creative Arts Studio Learning and Teaching

This paper examines the Assessment and Feedback aspects of Studio Teaching as Creative Arts pedagogy. Prompted by USQ’s newly offered Bachelor of Creative Arts (BCA), the author has developed an Assessment Matrix specifically designed to satisfy a number of imperatives, including: • ‘objectifying’ the subjective aspects of creative practice as assessable coursework/research • providing the means by which accurate, detailed, personalised and confidential feedback may be provided to students individually • providing consistent, accurate, meaningful assessment records for student, lecturer, and institution • ensuring consistency, continuity, and transparency of assessment processes and records to satisfy quality audits • minimising marking and assessment time, whilst maximising assessment integrity and depth • requiring only basic level skills and knowledge of a computer application already in common use (Microsoft Excel) • adaptability to a range of creative courses ‐ across disciplines This Assessment Matrix has been in development (and trialled) since January 2009.

Perceptions of audio feedback in higher education assessment

The aim of this paper is to present results of research investigating the effectiveness of audio feedback in a third year undergraduate unit. While there is a large and growing body of literature about providing assessment feedback, there is little focussing on the use of audio media. This study employs a mixed method approach, involving semi-structured interviews with academic staff and a survey of students. Analysis of the interview data suggests that there are a number of issues surrounding acceptance of using audio feedback by lecturers. The next stage of the study is to examine the extent to which lecturers change their perceptions as they use audio feedback and to analyse the perceptions of the students (n=120), including the perceived importance of feedback, the ways in which they used the audio feedback and the extent to which they believe they control events that affect them. Ultimately, this study seeks to provide recommendations appropriate to the implementation of audio feedback in higher education.

Time Efficient Assessment and Feedback Methods for Large Computer-Aided-Design Cohorts

Timely and individualized feedback on coursework is desirable from a student perspective as it facilitates formative development and encourages reflective learning practice. Faculty however are faced with a significant and potentially time consuming challenge when teaching larger cohorts if they are to provide feedback which is timely, individualized and detailed. Additionally, for subjects which assess non-traditional submissions, such as Computer-Aided-Design (CAD), the methods for assessment and feedback tend not to be so well developed or optimized. Issues can also arise over the consistency of the feedback provided. Evaluations of Computer-Assisted feedback in other disciplines (Denton et al, 2008), (Croft et al, 2001) have shown students prefer this method of feedback to traditional “red pen” marking and also that such methods can be more time efficient for faculty.
Herein, approaches are described which make use of technology and additional software tools to speed up, simplify and automate assessment and the provision of feedback for large cohorts of first and second year engineering students studying modules where CAD files are submitted electronically. A range of automated methods are described and compared with more “manual” approaches. Specifically one method uses an application programming interface (API) to interrogate SolidWorks models and extract information into an Excel spreadsheet, which is then used to automatically send feedback emails. Another method describes the use of audio recordings made during model interrogation which reduces the amount of time while increasing the level of detail provided as feedback.
Limitations found with these methods and problems encountered are discussed along with a quantified assessment of time saving efficiencies made.

The automation of radiotherapy treatment planning assessment

Aim The assessment of treatment plans is an important component in the education of radiation therapists. The establishment of a grade for a plan is currently based on subjective assessment of a range of criteria. The automation of assessment could provide a number of advantages including faster feedback, reduced chance of human error, and simpler aggregation of past results. Method A collection of treatments planned by a cohort of 27 second year radiation therapy students were selected for quantitative evaluation. Treatment sites included the bladder, cervix, larynx, parotid and prostate, although only the larynx plans had been assessed in detail. The plans were designed with the Pinnacle system and exported using the DICOM framework. Assessment criteria included beam arrangement optimisation, volume contouring, target dose coverage and homogeneity, and organ-at-risk sparing. The in-house Treatment and Dose Assessor (TADA) software1 was evaluated for suitability in assisting with the quantitative assessment of these plans. Dose volume data were exported in per-student and per-structure data tables, along with beam complexity metrics, dose volume histograms, and reports on naming conventions. Results The treatment plans were exported and processed using TADA, with the processing of all 27 plans for each treatment site taking less than two minutes. Naming conventions were successfully checked against a reference protocol. Significant variations between student plans were found. Correlation with assessment feedback was established for the larynx plans. Conclusion The data generated could be used to inform the selection of future assessment criteria, monitor student development, and provide useful feedback to the students. The provision of objective, quantitative evaluations of plan quality would be a valuable addition to not only radiotherapy education programmes but also for staff development and potentially credentialing methods. New functionality within TADA developed for this work could be applied clinically to, for example, evaluate protocol compliance.

'Take away from the dry sixties style marking': Lecturer and student perceptions and experiences of audio feedback

Providing audio feedback to assessment is relatively uncommon in higher education. However, published research suggests that it is preferred over written feedback by students but lecturers were less convinced. The aim of this paper is to examine further these findings in the context of a third year business ethics unit. Data was collected from two sources. The first is a series of in-depth, semi-structured interviews conducted with three lecturers providing audio feeback for the first time in Semester One 2011. The second source of data was drawn from the university student evaluation system. A total of 363 responses were used providing 'before' and 'after' perspectives about the effectiveness of audio feedback versus written feedback. Between 2005 and 2009 the survey data provided information about student attitudes to written assessment feedback (n=261). From 2010 onwards the data relates to audio (mp3) feedback (n=102). The analysis of he interview data indicated that introducing audio feedback should be done with care. The perception of the participating lecturers was mixed, ranging from sceptism to outright enthusiasm, but over time the overall approach became positive. It was found that particular attention needs to be paid to small (but important) technical details, and lecturers need to be convinced of its effectieness, especially that it is not necessarily more time consuming than providing written feedback. For students, the analysis revealed a clear preference for audio feedback. It is concluded that there is cause for concern and reason for optimism. It is a cause for concern because there is a possibility that scepticism on the part of academic staff seems to be based on assumptions about what students prefer and a concern about using the technology. There is reason for optimism because the evidence points towards students preferring audio feedback and as academic staff become more familiar with the technology the scepticism tends to evaporate. While this study is limited in scope, questions are raised about tackling negative staff perceptions of audio feedback that are worthy of further research.

Improving Learners’ Motivation through Feedback in English Language Teaching and Learning at Polytechnic School in Benguela, Angola

Currently, it is widely perceived among the English as a Foreign Language (EFL) teaching professionals, that motivation is a central factor for success in language learning. This work aims to examine and raise teachers’ awareness about the role of assessment and feedback in the process of language teaching and learning at polytechnic school in Benguela to develop and/or enhance their students’ motivation for learning. Hence the paper defines and discusses the key terms and, the techniques and strategies for an effective feedback provision in the context under study. It also collects data through the use of interview and questionnaire methods, and suggests the assessment and feedback types to be implemented at polytechnic school in Benguela

A Teacher’s Formative Assessment Perceptions and Practices in Oral Intermediate English Courses at the Université de Montréal

Au cours des dernières années, les pratiques de l’évaluation comme aide à l’apprentissage (c’est-à-dire l’auto-évaluation, l’évaluation par les pairs, la rétroaction) dans la salle de classe ont été de plus en plus considérées comme des éléments essentiels (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). Cependant, dans le domaine de l’apprentissage d’une langue seconde la recherche sur ce sujet est plutôt limitée. En nous fondant sur les études de Colby-Kelly et Turner (2007) et de Lyster et Ranta (1997), nous avons mené une recherche exploratoire visant à combler ce besoin. L’objectif général était de comprendre comment l’évaluation formative se réalise dans deux cours d’anglais intermédiaire à l’Université de Montréal, et de comparer comment ces pratiques sont perçues et vécues par la professeure et ses étudiants. Trois questions de recherche étaient posées: 1. Quelle est la nature de l’évaluation formative dans une salle de classe d’anglais langue seconde? 2. Est-ce que les pratiques de la professeure reflètent ce qu’elle pense de l’évaluation formative? 3. Quels sont les correspondances et différences entre les croyances de la professeure et les perceptions de ses étudiants quant aux bénéfices de l’évaluation formative pour l’apprentissage de l’anglais langue seconde? La collecte de données comprend des observations en classe, des entrevues auprès de la professeure et des questionnaires pour les étudiants. Pour répondre à notre première question de recherche, nous avons analysé la rétroaction donnée par la professeure pendant une production orale réalisée par ses étudiants à partir de la typologie de feedback et de uptake de l’étude de Lyster et Ranta (1997). En ce qui a trait à la deuxième question de recherche, nous avons fait des entrevues avec la professeure en vue d’expliciter ses perceptions relativement à l’évaluation formative. À la fin du trimestre, nous avons comparé ses réponses avec sa pratique à partir des enregistrements vidéo produits en classe. Finalement, pour répondre à notre dernière question de recherche nous avons comparé les réponses données par la professeure aux entrevues avec les réponses des étudiants à un questionnaire adapté à partir de celui de Colby-Kelly et Turner. Finalement, nous présentons et discutons les résultats les plus significatifs obtenus dans cette étude qualitative Nous concluons cette thèse en proposant de avenues pour des recherches futures.