Delivering the link : using student feedback studies to improve student learning in dissertation studies

Student feedback is essential to enable lecturers to understand whether attempts to improve learning and education experience lead to improvement (George and Cowan, 1999; Gibbs, 1993). Current UK practice relies largely on end of module questionnaires to feedback levels of student satisfaction (Cowan, 2002), however there are inherent weaknesses in this approach; it seldom leads to change for that particular cohort; it relies on uncorroborated opinion, and finally, it may derive from superficial feedback from a minority of students with the remainder suffering from questionnaire fatigue.

This research project involved a cohort of final year building surveying students at Sheffield Hallam University, in England, who were undertaking a dissertation in two modules. During 2002/3 the use of Blackboard software had also been adopted by the module leader as an educational tool to support student learning in the module. The lecturer wanted to identify how students used Blackboard and what they thought about the most appropriate use of the medium. The research methodology sought to redress some of the issues identified above with student feedback, regarding timing of feedback, implementation of change during the teaching period and the lack of depth in the data. Using principles adopted by Angelo and Cross (1993), this research formatively evaluated student perceptions and levels of satisfaction with the dissertation module, the teaching materials, the workshops, the supervisory arrangements and relationships. This paper presents the findings of the research and illustrates the changes that were made during the year and the student’s views of these changes. The paper demonstrates how linking teaching with research has been delivered at Sheffield Hallam.

Lecturing and other face-to-face teaching – too much or too little? An assessment based on student feedback and fail rates

Institutes of higher learning are tending to reduce the amount of face-to-face teaching that they offer, and particularly through the traditional pedagogical method of lecturing. There is ongoing debate about the educational value of lectures as a teaching approach, in terms of both whether they facilitate understanding of subject material and whether they augment the student educational experience. In this study, student evaluation of teaching scores plus academic outcome (percentage of students who fail) was assessed for 236 course units offered by a science faculty at an Australian university over the course of one year. These measures were related to the degree to which lectures and other face-to-face teaching were used in these units, controlling for factors such as class size, school and year level. An information-theoretic model selection approach was employed to identify the best models and predictors of student assessments and fail rates. All the top models of student feedback included a measure reflecting amount of face-to-face teaching, with the evaluation of quality of teaching being higher in units with higher proportions of lectures. However, these models explained only 12–20% of the variation in student evaluation scores, suggesting that many other factors come into play. By contrast, units with fewer lectures have lower failure rates. These results suggest that moving away from lectures and face-to-face teaching may not harm, and indeed may improve the number of students who pass the subject, but that this may be incurred at the expense of greater dissatisfaction in students' learning experience.

Piloting the feasibility of head-mounted video technology to augment student feedback during simulated clinical decision-making: an observational design pilot study

BACKGROUND: Clinical decision-making is a complex activity that is critical to patient safety. Simulation, augmented by feedback, affords learners the opportunity to learn critical clinical decision-making skills. More detailed feedback following simulation exercises has the potential to further enhance student learning, particularly in relation to developing improved clinical decision-making skills. OBJECTIVE: To investigate the feasibility of head-mounted video camera recordings, to augment feedback, following acute patient deterioration simulations. DESIGN: Pilot study using an observational design. METHODS: Ten final-year nursing students participated in three simulation exercises, each focussed on detection and management of patient deterioration. Two observers collected behavioural data using an adapted version of Gaba's Clinical Simulation Tool, to provide verbal feedback to each participant, following each simulation exercise. Participants wore a head-mounted video camera during the second simulation exercise only. Video recordings were replayed to participants to augment feedback, following the second simulation exercise. Data were collected on: participant performance (observed and perceived); participant perceptions of feedback methods; and head-mounted video camera recording feasibility and capability for detailed audio-visual feedback. RESULTS: Management of patient deterioration improved for six participants (60%). Increased perceptions of confidence (70%) and competence (80%), were reported by the majority of participants. Few participants (20%) agreed that the video recording specifically enhanced their learning. The visual field of the head-mounted video camera was not always synchronised with the participant's field of vision, thus affecting the usefulness of some recordings. CONCLUSION: The usefulness of the video recordings, to enhance verbal feedback to participants on detection and management of simulated patient deterioration, was inconclusive. Modification of the video camera glasses, to improve visual-field synchronisation with participants' actual visual field, is recommended to further explore this technology for enhancing student performance.

Improving student learning in dissertation research through feedback studies. The Sheffield Hallam Building surveying experience

According to George and Cowan (1999) student feedback is essential to enable lecturers to understand whether attempts to improve learning and educational experience lead to improvement. Current UK practice uses end of module questionnaires to feedback levels of satisfaction (Cowan, 2002). There are inherent weaknesses, namely that it seldom leads to a change for that particular cohort of students, secondly it relies on uncorroborated opinion, and may derive from superficial feedback from a minority of students with the remainder suffering from questionnaire fatigue. Finally the data may not be especially relevant to a particular module, a particular weakness (Heywood, 2000).

Using principles identified by Angelo and Cross (1993), this research adopted a methodology to formatively evaluate student perceptions and levels of satisfaction with the dissertation module. Using a cohort of Building Surveying students at Sheffield Hallam University, in England, views on course materials, the use of Blackboard software, the workshops and the relationships with supervisors were gathered and analysed.

A number of measures were identified as a result of this study, that may, if implemented, improve student learning. Examples are the use of checklists for student for each research / dissertation stage to ensure nothing is forgotten. Provision of additional ‘drop in’ workshops where students could see the module leader with specific issues. Additional optional workshops for questionnaire coding and review of previous theses for example.

It will not be possible to measure whether this student cohort’s learning and performance improves until the summer of 2003 and the final dissertations are assessed. A statistical analysis, comparing their dissertation marks against marks for other topics will show whether there is improvement in marks as a result of this student feedback study. The 2002/3 cohort result can also be analysed against previous cohorts to establish whether any improvement is evident.

20 years on - have student evaluations made a difference?

This paper poses the question, what impact have student evaluations (SE’s) of teaching had on the improvement in teaching in Australian tertiary education. The paper proposes to assess the effectiveness of SE’s through an investigations of the ways in which have been used in Australian tertiary education over the last 20 years or so. Three approaches are discussed: a) quality assurance - student evaluations used to ensure the quality of the learning environment in which learning takes place; b) quality improvement - student evaluations used as a diagnostic tool for individual teaching staff; and c) student feedback – student evaluations used to provide prospective and current students with information to help inform their choices and expectations. The discussions reviews some of the research and known practise in Australian universities. It concludes that SE’s have had a profound impact on pedagological understandings of tertiary education by placing the student at centre stage of teaching and learning. However it suggests that we do not know how SE’s have impacted on teaching effectiveness and improvement. The paper concludes by recommending further research to define critical success factors, a better understanding of how to make the student feedback and evaluation tools and mechanisms more approachable and meaningful for students, and an assessment of the ‘survey industry’ as a sustainable enterprise.

Is it worth the effort? How feedback influences students' subsequent submission of assessable work

This paper explores the extent to which students appear to their assessors to act on feedback they have received, and questions the assumption that providing feedback alone is sufficient to effect higher standards of work by students. Feedback provided to 51 undergraduate social work students, on two consecutive assignments involving a similar task, was examined to ascertain the number of problem areas noted from seven predefined categories. While the greatest increase in marks was associated with the greatest reductions in the number of problem areas identified in the comments, overall two-thirds of all students (66.7%) were awarded marks for both assignments within four percentage points. As such, this study found only limited support for the idea that students respond to feedback by making changes which are consistent with the intent of the feedback received. Hence the assumption that providing feedback alone is sufficient to effect higher standards of work by students was not supported. These findings invite educators to critically reflect on their own practices in providing feedback to students.

Closing the loop on student evaluations

It is generally agreed that good teaching is dependent on the nature of the learning goals and the quality of the environment in which learning takes place1,2. If this is the case, then planned as opposed to unplanned variations in learning experiences are the hallmark of good teaching. This raises a complex set of relations between assuring and improving teaching within an institution. The essential question for a University is: how do we assure that learning is maximised in all given contexts?

Benchmarking the use of student feedback was identified as a priority project at the ATN annual conference in February 2002. This paper presents a number of issues arising from a study of practices across the ATN in relation to collecting, analysing and using student feedback. The project involved working with those responsible for teaching and learning improvements at the operational level to identify strategies for a systematic approach to the use of student feedback for improving communication of actions arising from results.

The framework for the inquiry involved an exploration of student feedback systems as they operate for different cohorts of students, differing measures (or constructs), and at different levels within institutional structures. The framework also explored the various tracking and reporting systems by which results generated from student feedback systems at each level were utilised to develop strategies for improving teaching and learning. Findings from the study were used to inform recommendations in relation to internal practices within each University, as well as initiatives for benchmarking student evaluation results across universities within the ATN network.

Student responses to activities designed to develop generic professional skills

This paper reports on student responses to a range of assessment activities in a final-year engineering unit. Existing assessment activities were supplemented with new activities, the overall aim being to link the assessment activities more effectively to the material being studied, and to develop a range of generic skills important in professional engineering practice. A class survey was undertaken at the beginning of the semester to establish the initial attitudes to the new assessment activities. This was followed up with an end-of-semester survey to determine the change in perceived value of the assessment activities, and to collect student feedback regarding the activities. The perceived value of the assessment activities was determined using a Likert rating scale, while student feedback was collected using open-ended questions. The assessment activities evaluated were group work, case study investigation, report writing, oral presentation, group self-assessment, industrial interviews, and written reflective journals. The responses indicate that engineering students value a range of assessment activities. They value highly visits to real engineering organizations, and—contrary to popular belief —value and enjoy oral presentation exercises.

Rethinking models of feedback for learning: the challenge of design

Student feedback is a contentious and confusing issue throughout higher education institutions. This paper develops and analyses two models of feedback: the first is based on the origins of the term in the disciplines of engineering and biology. It positions teachers as the drivers of feedback. The second draws on ideas of sustainable assessment. This positions learners as having a key role in driving learning, and thus generating and soliciting their own feedback. It suggests that the second model equips students beyond the immediate task and does not lead to false expectations that courses cannot deliver. It identifies the importance of curriculum design in creating opportunities for students to develop the capabilities to operate as judges of their own learning.

Student opportunities in materials design and manufacture: Introducing a new manufacturing with composites course

The shift towards strong and lightweight fibre reinforced polymer-matrix composites for many high performance applications has resulted in an increasing need to expose students to composite design and manufacture courses in the undergraduate curriculum. In contrast, student exposure to composite materials is often still limited to a topic within a materials or manufacturing related course (unit). This paper presents the initial offering of a composite materials elective at Griffith University in Australia. The course also addresses environmental concerns through the inclusion of natural fibre composites. An evaluation of student perceptions is considered from Griffith’s Student Experience of Course (SEC) and separate Student Experience of Teaching (SET) surveys. These evaluations demonstrate the high level of student engagement with the course, but also highlighted areas for improvement, including the need to incorporate even more hands-on practical work. Interestingly, the inclusion of natural fibre composites and the related discussion surrounding environmental and societal issues are not focused on in student feedback.

Responding to student evaluation of teaching : closing the loop

Our undergraduate curriculum focuses on preparing students to be critical thinkers, problem solvers, and knowledgeable, responsible information technology (IT) professionals. Students often question the relevance of core subjects which are designed to introduce and develop these attributes; such units regularly perform poorly in student evaluation of teaching. Although these subjects are strongly grounded in meeting industry expectations regarding employability skills, students do not value the learning as the curriculum is deemed non-technical and not worthy of inclusion in an IT qualification. As these are core subjects, improving student perceptions and learning outcomes is of critical importance. Simply responding to student feedback received via student evaluation of teaching had not resulted in any improvement; so a holistic approach was adopted. A workshop was organised to explore the issues and develop a strategy to improve outcomes and perceptions of these subjects. Rather than addressing student feedback specifically, the underlying issues that led to students’ negative perceptions were identified and addressed. Recommendations were implemented in 2014 resulting in improvements in student evaluation for relevant subjects. In this paper, we will discuss the process that was adopted to respond effectively to student evaluations, and explore the impact that this had on them.

Teaching an old game new tricks : long-term feedback on a re-designed online role play

Despite an extensive history of use in teaching Political Science subjects, long-term scholarly studies of online role plays are uncommon. This paper redresses that balance by presenting five years of data on the Middle East Politics Simulation. This online role play has been run since the 1990s and underwent significant technical upgrade in 2013-14. The data presented here covers student feedback to this upgrade process and the factors that can influence their response. Key indications are that students tend to recognise when something is fit (or not) for its purpose and will forgo attractive and well-appointed online environments if the underlying learning exercise is valued. However, there are limits to this minimalism and whilst designers do not need to replicate every Internet trend, attention needs to be paid to broader changes in technology, such as access platform and changing avenues of political communication. The study demonstrates that long-term monitoring of online role play exercises is important to allow informed changes to be implemented and their impacts properly assessed.

Developing wikis to foster web-based learning communities : an iterative approach

This paper discusses ongoing research at Deakin University, which focuses on developing wikis to foster web-based learning communities. Research to date has used wikis to facilitate collaborative icebreaker exercises, discussions, and to create knowledge repositories. Student feedback has contributed to the iterative revision of the wiki interface, the icebreaker exercise and the development of new tasks for students to complete using the wiki. The analysis and discussion of the experiments presented in this paper focuses on usage trends such as the signature, viewing and editing patterns exhibited by the student cohort. The community building potential of wikis is discussed, highlighting the specific wiki features that can be used to foster a sense of community in a web-based learning environment. Finally, issues surrounding the development of web-based learning communities, which have emerged through the wiki study, are discussed and future directions are outlined.