A quest for helpful feedback to programming coursework

The 2011 National Student Survey (NSS) revealed that 40% of full-time students in England do not think that the feedback on their work has been helpful, even though 66% of these students agreed that the feedback was detailed and 62% of them agreed that the feedback has been prompt. Detailed feedback that is not considered helpful by students means a waste of tutors' time while students continue to struggle with their learning. What do students consider as helpful feedback? What are the qualities of helpful feedback? What are the preferred forms of feedback? How should tutors write feedback so that students will find it helpful? Can ICT help to improve the quality of feedback? In our ongoing search for answers to the above questions, we have trialled the use of a novel online application (eCAF) to assess programming coursework from engineering, mathematics and computing students and, through a survey, have collected their views on the feedback received. The survey reveals that most students prefer electronic feedback as given through eCAF, with verbal feedback ranked second and hand-written feedback ranked even lower. The survey also indicates that the feedback from some tutors is considered more helpful than others. We report on the detailed findings of the survey. By comparing the kinds of feedback given by each tutor who took part in the trial, we explore ways to improve the helpfulness of feedback on programming coursework in a bid to promote learning amongst engineering students.

'It’d be useful, but I wouldn’t use it':barriers to university students’ feedback seeking and recipience

For feedback to be effective, it must be used by the receiver. Prior research has outlined numerous reasons why students’ use of feedback is sometimes limited, but there has been little systematic exploration of these barriers. In 11 activity-oriented focus groups, 31 undergraduate Psychology students discussed how they use assessment feedback. The data revealed many barriers that inhibit use of feedback, ranging from students’ difficulties with decoding terminology, to their unwillingness to expend effort. Thematic analysis identified four underlying psychological processes: awareness, cognisance, agency, and volition. We argue that these processes should be considered when designing interventions to encourage students’ engagement with feedback. Whereas the barriers identified could all in principle be removed, we propose that doing so would typically require – or would at least benefit from – a sharing of responsibility between teacher and student. The data highlight the importance of training students to be proactive receivers of feedback.

Supporting learners’ agentic engagement with feedback:a systematic review and a taxonomy of recipience processes

Much has been written in the educational psychology literature about effective feedback and how to deliver it. However, it is equally important to understand how learners actively receive, engage with, and implement feedback. This article reports a systematic review of the research evidence pertaining to this issue. Through an analysis of 195 outputs published between 1985 and early 2014, we identified various factors that have been proposed to influence the likelihood of feedback being used. Furthermore, we identified diverse interventions with the common aim of supporting and promoting learners' agentic engagement with feedback processes. We outline the various components used in these interventions, and the reports of their successes and limitations. Moreover we propose a novel taxonomy of four recipience processes targeted by these interventions. This review and taxonomy provide a theoretical basis for conceptualizing learners' responsibility within feedback dialogues and for guiding the strategic design and evaluation of interventions. Receiving feedback on one's skills and understanding is an invaluable part of the learning process, benefiting learners far more than does simply receiving praise or punishment (Black & Wiliam, 1998 Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5, 7–74. doi:10.1080/0969595980050102[Taylor & Francis Online]; Hattie & Timperley, 2007 Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112. doi:10.3102/003465430298487[CrossRef], [Web of Science ®]). Inevitably, the benefits of receiving feedback are not uniform across all circumstances, and so it is imperative to understand how these gains can be maximized. There is increasing consensus that a critical determinant of feedback effectiveness is the quality of learners' engagement with, and use of, the feedback they receive. However, studies investigating this engagement are underrepresented in academic research (Bounds et al., 2013 Bounds, R., Bush, C., Aghera, A., Rodriguez, N., Stansfield, R. B., & Santeen, S. A. (2013). Emergency medicine residents' self-assessments play a critical role when receiving feedback. Academic Emergency Medicine, 20, 1055–1061. doi:10.1111/acem.12231[CrossRef], [PubMed], [Web of Science ®]), which leaves a “blind spot” in our understanding (Burke, 2009 Burke, D. (2009). Strategies for using feedback students bring to higher education. Assessment & Evaluation in Higher Education, 34, 41–50. doi:10.1080/02602930801895711[Taylor & Francis Online], [Web of Science ®]). With this blind spot in mind, the present work sets out to systematically map the research literature concerning learners' proactive recipience of feedback. We use the term “proactive recipience” here to connote a state or activity of engaging actively with feedback processes, thus emphasizing the fundamental contribution and responsibility of the learner (Winstone, Nash, Rowntree, & Parker, in press Winstone, N. E., Nash, R. A., Rowntree, J., & Parker, M. (in press). ‘It'd be useful, but I wouldn't use it’: Barriers to university students' feedback seeking and recipience. Studies in Higher Education. doi: 10.1080/03075079.2015.1130032[Taylor & Francis Online]). In other words, just as Reeve and Tseng (2011 Reeve, J., & Tseng, M. (2011). Agency as a fourth aspect of student engagement during learning activities. Contemporary Educational Psychology, 36, 257–267. doi:10.1016/j.cedpsych.2011.05.002[CrossRef], [Web of Science ®]) defined “agentic engagement” as a “student's constructive contribution into the flow of the instruction they receive” (p. 258), likewise proactive recipience is a form of agentic engagement that involves the learner sharing responsibility for making feedback processes effective.

Development of a new student evaluation instrument of instructor effectiveness in online courses

The purpose of this study was to (a) develop an evaluation instrument capable of rating students' perceptions of the instructional quality of an online course and the instructor’s performance, and (b) validate the proposed instrument with a study conducted at a major public university. The instrument was based upon the Seven Principles of Good Practice for Undergraduate Education (Chickering & Gamson, 1987). The study examined four specific questions. 1. Is the underlying factor structure of the new instrument consistent with Chickering and Gamson's Seven Principles? 2. Is the factor structure of the new instrument invariant for male and female students? 3. Are the scores on the new instrument related students’ expected grades? 4. Are the scores on the new instrument related to the students' perceived course workload? ^ The instrument was designed to measure students’ levels of satisfaction with their instruction, and also gathered information concerning the students’ sex, the expected grade in the course, and the students’ perceptions of the amount of work required by the course. A cluster sample consisting of an array of online courses across the disciplines yielded a total 297 students who responded to the online survey. The students for each course selected were asked to rate their instructors with the newly developed instrument. ^ Question 1 was answered using exploratory factor analysis, and yielded a factor structure similar to the Seven Principles.^ Question 2 was answered by separately factor-analyzing the responses of male and female students and comparing the factor structures. The resulting factor structures for men and women were different. However, 14 items could be realigned under five factors that paralleled some of the Seven Principles. When the scores of only those 14 items were entered in two principal components factor analyses using only men and only women, respectively and restricting the factor structure to five factors, the factor structures were the same for men and women.^ A weak positive relationship between students’ expected grades and their scores on the instrument was found (Question 3). There was no relationship between students’ perceived workloads for the course and their scores on the instrument (Question 4).^

Online Education, Accrediting Standards, and Student Success: An Examination of the Relationship Between the Southern Association of Colleges and Schools Commission on Colleges Standards for Online Education and Student Success

Prior to 2000, there were less than 1.6 million students enrolled in at least one online course. By fall 2010, student enrollment in online distance education showed a phenomenal 283% increase to 6.1 million. Two years later, this number had grown to 7.1 million. In light of this significant growth and skepticism about quality, there have been calls for greater oversight of this format of educational delivery. Accrediting bodies tasked with this oversight have developed guidelines and standards for online education. There is a lack of empirical studies that examine the relationship between accrediting standards and student success. The purpose of this study was to examine the relationship between the presence of Southern Association of Colleges and Schools Commission on College (SACSCOC) standards for online education in online courses, (a) student support services and (b) curriculum and instruction, and student success. An original 24-item survey with an overall reliability coefficient of .94 was administered to students (N=464) at Florida International University, enrolled in 24 university-wide undergraduate online courses during fall 2014, who rated the presence of these standards in their online courses. The general linear model was utilized to analyze the data. The results of the study indicated that the two standards, student support services and curriculum and instruction were both significantly and positively correlated with student success but with small R2 and strengths of association less than .35 and .20 respectively. Mixed results were produced from Chi-square tests for differences in student success between higher and lower rated online courses when controlling for various covariates such as discipline, gender, race/ethnicity, GPA, age, and number of online courses previously taken. A multiple linear regression analysis revealed that the curriculum and instruction standard was the only variable that accounted for a significant amount of unique variance in student success. Another regression test revealed that no significant interaction effect exists between the two SACSCOC standards and GPA in predicting student success. The results of this study are useful for administrators, faculty, and researchers who are interested in accreditation standards for online education and how these standards relate to student success.

Confirmatory feedback in teacher education:an instigator of student teacher learning

This book examines the intricacies of the discourse of post-observation feedback that student teachers receive following group teaching practice. In particular, the author explores confirmatory feedback as an instigator of student teacher learning, and examines the potential links between feedback and change. The book will be of specific interest to researchers, teacher educators and other professionals involved in feedback-giving settings.

Toward an Understanding of Faculty Perceptions about Factors that Influence Student Success in Online Education

This paper explores factors that faculty perceive as significant in influencing student success in online education. Technological support, degree of comfort with technology and a student’s personal characteristics were deemed by faculty as critical to success. Understanding these key factors could help to improve student success in online education.

Accountability, Quality, and Student Success in Online Education: A Literature Review of Empirical Studies

This paper examines empirical studies on accountability, quality, and student success in online education. It advances that accountability and quality are critical components for student success in online education. It concludes that there is a lack of empirical studies that examine the effects of these measures on student success.

Student Response Systems: enabling instant feedback

These PowerPoint slides supported an ILIaD workshop in September 2016.

Concerns and Expectations of Students Participating in Study Abroad Programs: Blogging to Reveal the Dynamic Student Voice

Study abroad programmes (SAP) have become increasingly popular with university students and within academia. They are often seen as an experiential opportunity to expand student learning and development, including increases in global, international, and intercultural competences. However, despite the increasing popularity of and participation in study abroad programmes, many student concerns and uncertainties remain. This research investigates initial pre-departure concerns and apprehensions of students undertaking a one-semester study abroad programme and uses these as context for an examination of violated expectations of students during their programme. The research uses interpretative phenomenological analysis to interpret data collected from regularly-updated blogs composed by students throughout their SAP experience. The process of using blogs to collect data is less formalised than many other approaches of interpretative phenomenological analysis, enabling ‘in the moment’ feedback during the SAP and lending greater depth to the understanding of student perceptions.

An integrated measure of student perceptions of feedback, engagement and school identification

Teacher feedback influences student learning, identity construction and trajectories. This study tests the measurement properties of a questionnaire designed to assess (a) student perceptions about teacher feedback; (b) student identification with school, and; (c) student engagement. 1089 students in grades 6 through 10 (mean age 13.4) participated in the study. Factor analyses yield dimensions of School Identification, Effective Feedback, Person-Centered Feedback, Engagement, and Social Acceptance. Internal consistency for principal dimensions varied between .77 and .89. The instrument is suitable for assessing student school identification, behavioral engagement, and perceptions of teacher feedback.

Online education, accrediting standards, and student success: An examination of the relationship between the southern association of colleges and schools commission on colleges standards for online education and student success

Prior to 2000, there were less than 1.6 million students enrolled in at least one online course. By fall 2010, student enrollment in online distance education showed a phenomenal 283% increase to 6.1 million. Two years later, this number had grown to 7.1 million. In light of this significant growth and skepticism about quality, there have been calls for greater oversight of this format of educational delivery. Accrediting bodies tasked with this oversight have developed guidelines and standards for online education. ^ There is a lack of empirical studies that examine the relationship between accrediting standards and student success. The purpose of this study was to examine the relationship between the presence of Southern Association of Colleges and Schools Commission on College (SACSCOC) standards for online education in online courses, (a) student support services and (b) curriculum and instruction, and student success. An original 24-item survey with an overall reliability coefficient of .94 was administered to students (N=464) at Florida International University, enrolled in 24 university-wide undergraduate online courses during fall 2014, who rated the presence of these standards in their online courses. The general linear model was utilized to analyze the data. The results of the study indicated that the two standards, student support services and curriculum and instruction were both significantly and positively correlated with student success but with small R2 and strengths of association less than .35 and .20 respectively. Mixed results were produced from Chi-square tests for differences in student success between higher and lower rated online courses when controlling for various covariates such as discipline, gender, race/ethnicity, GPA, age, and number of online courses previously taken. A multiple linear regression analysis revealed that the curriculum and instruction standard was the only variable that accounted for a significant amount of unique variance in student success. Another regression test revealed that no significant interaction effect exists between the two SACSCOC standards and GPA in predicting student success. ^ The results of this study are useful for administrators, faculty, and researchers who are interested in accreditation standards for online education and how these standards relate to student success.^

Using online peer assessment tools to provide feedback and promote learning

A set of slides used for the RAP SIG event on 19 Jan 2017

Supporting law students’ skills development online – a strategy to improve skills and reduce student stress?

Law students internationally suffer from a high level of psychological distress compared with the general and student populations, and anecdotal evidence suggests that students developing skills without adequate support experience significant stress and anxiety. This article considers an initiative at one Australian law school to develop a degree-wide structured online skills development programme as a means to both improve student skills acquisition and reduce student stress. The project implements, through the use of learning technology, the principles proposed by McKinney for making small changes to law school teaching, informed by self-efficacy theory, which can have powerful results.