Final Report: University of Technology, Sydney Student Feedback Survey Review

This evaluation was commissioned by Martin Hanlon, Director of the Planning and Quality Unit of the University of Technology, Sydney (UTS) to investigate the Student Feedback Survey (SFS) system, engagement in stakeholder feedback and provide recommendations against the Terms of Reference.

Student feedback systems in higher education: a focused literature review and environmental scan

In recent times, higher education institutions have paid increasing attention to the views of students to obtain feedback on their experience of learning and teaching through internal surveys. This article reviews research in the field and reports on practices in other Australian universities. Findings demonstrate that while student feedback is valued and used by all Australian universities, survey practices are idiosyncratic and in the majority of cases, questionnaires lack validity and reliability; data are used inadequately or inappropriately; and they offer limited potential for cross-sector benchmarking. The study confirms the need for institutions to develop an overarching framework for evaluation in which a valid, reliable, multidimensional and useful student feedback survey constitutes just one part. Given external expectations and internal requirements to collect feedback from students on their experience of learning and teaching, the pursuit of sound evaluation practices will continue to be of interest at local, national and international levels.

Marketing Brock University in the 1980's: assessing freshman student feedback in strategic planning approach

Universities have entered a period of rapid change and upheaval due to an external environment beyond their control which includes shifting demographic patterns, accelerating technology, funding shortages, and keener competition for students. Strategic planning, a comprehensive vision which challenges universities to take bold and creative measures to meet the threats and opportunities of the future, is an institutional imperative in the 1980's. This paper examines freshman student feedback in an effort to incorporate this important element into a strategic plan for Brock University, a small, predominantly liberal arts university in St. Catharines, Ontario. The study was designed to provide information on the characteristics of the 1985-86 pool of freshman registrants: their attitudes towards Brock's recruitment measures, their general university priorities, and their influences in regard to university selection (along with other demographical and attitudinal data). A survey involving fixed-alternative questions of a subjective and objective nature was administered in two large freshman classes at Brock in which a broad cross-section of academic programs was anticipated. Computer analysis of the data for the 357 respondents included total raw frequencies and rounded percentages, as well as subgroup cross-tabulation by geographic home area of respondent, academic major, and high school graduating average. The four directional hypotheses put forward were all substantiatied by the survey data, indicating that 1) the university's current recruitment program had been a positive influence during their university search 2) parents were the most influential group in the students' decisions related to university 3) respondents viewed institutional reputation as less of a priority than an enjoyable university lifestyle in a personal learning atmosphere 4) students had a decided preference for co-operative study and internship programs. Strategic planning recommendations included a reduction in the faculty/student ratio through faculty hirings to restore the close rapport between professors and students, increased recruitment presentations in Ontario high schools to enlarge the applicant pool, creation of an Office of Co-operative Study and Internship Programs, institutional emphasis on a "customer orientation", and an extension of research into student demographics and attitudinal data.

Development, evaluation and use of a student experience survey in undergraduate science laboratories: the advancing science by enhancing learning in the laboratory student laboratory learning experience survey

Student experience surveys have become increasingly popular to probe various aspects of processes and outcomes in higher education, such as measuring student perceptions of the learning environment and identifying aspects that could be improved. This paper reports on a particular survey for evaluating individual experiments that has been developed over some 15 years as part of a large national Australian study pertaining to the area of undergraduate laboratories—Advancing Science by Enhancing Learning in the Laboratory. This paper reports on the development of the survey instrument and the evaluation of the survey using student responses to experiments from different institutions in Australia, New Zealand and the USA. A total of 3153 student responses have been analysed using factor analysis. Three factors, motivation, assessment and resources, have been identified as contributing to improved student attitudes to laboratory activities. A central focus of the survey is to provide feedback to practitioners to iteratively improve experiments. Implications for practitioners and researchers are also discussed.

Please take out your phones : on the spot solicitation of student feedback in class

The use of mobile devices such as smart phones and tablets in classrooms has been met with mixed sentiments. Some instructors and teachers see them as a distraction and regularly ban their usage. Others who see their potential to enhance learning have started to explore ways to integrate them into their teaching in an attempt to improve student engagement. In this paper we report on a pilot study that forms part of a university-wide project reconceptualising its approach to the student evaluation of learning and teaching. In a progressive decision to embrace mobile technology, the university decided to trial a smart phone app designed for students to check-in to class and leave feedback on the spot. Our preliminary findings from trialling the app indicate that the application establishes a more immediate feedback loop between students and teachers. However, the app’s impact depends on how feedback is shared with students and how the teaching team responds.

Evolution of Various Library Instruction Strategies: Using Student Feedback to Create and Enhance Online Active Learning Assignments

This case study traces the evolution of library assignments for biological science students from paper-based workbooks in a blended (hands-on) workshop to blended learning workshops using online assignments to online active learning modules which are stand-alone without any face-to-face instruction. As the assignments evolved to adapt to online learning supporting materials in the form of PDFs (portable document format), screen captures and screencasting were embedded into the questions as teaching moments to replace face-to-face instruction. Many aspects of the evolution of the assignment were based on student feedback from evaluations, input from senior lab demonstrators and teaching assistants, and statistical analysis of the students’ performance on the assignment. Advantages and disadvantages of paper-based and online assignments are discussed. An important factor for successful online learning may be the ability to get assistance.

Delivering the link : using student feedback studies to improve student learning in dissertation studies

Student feedback is essential to enable lecturers to understand whether attempts to improve learning and education experience lead to improvement (George and Cowan, 1999; Gibbs, 1993). Current UK practice relies largely on end of module questionnaires to feedback levels of student satisfaction (Cowan, 2002), however there are inherent weaknesses in this approach; it seldom leads to change for that particular cohort; it relies on uncorroborated opinion, and finally, it may derive from superficial feedback from a minority of students with the remainder suffering from questionnaire fatigue.

This research project involved a cohort of final year building surveying students at Sheffield Hallam University, in England, who were undertaking a dissertation in two modules. During 2002/3 the use of Blackboard software had also been adopted by the module leader as an educational tool to support student learning in the module. The lecturer wanted to identify how students used Blackboard and what they thought about the most appropriate use of the medium. The research methodology sought to redress some of the issues identified above with student feedback, regarding timing of feedback, implementation of change during the teaching period and the lack of depth in the data. Using principles adopted by Angelo and Cross (1993), this research formatively evaluated student perceptions and levels of satisfaction with the dissertation module, the teaching materials, the workshops, the supervisory arrangements and relationships. This paper presents the findings of the research and illustrates the changes that were made during the year and the student’s views of these changes. The paper demonstrates how linking teaching with research has been delivered at Sheffield Hallam.

Measuring the quality of VET using the Student Outcomes Survey

The aim of this report is to examine the use of student course satisfaction information and post-study outcomes as a means of determining markers of training quality.

Lecturing and other face-to-face teaching – too much or too little? An assessment based on student feedback and fail rates

Institutes of higher learning are tending to reduce the amount of face-to-face teaching that they offer, and particularly through the traditional pedagogical method of lecturing. There is ongoing debate about the educational value of lectures as a teaching approach, in terms of both whether they facilitate understanding of subject material and whether they augment the student educational experience. In this study, student evaluation of teaching scores plus academic outcome (percentage of students who fail) was assessed for 236 course units offered by a science faculty at an Australian university over the course of one year. These measures were related to the degree to which lectures and other face-to-face teaching were used in these units, controlling for factors such as class size, school and year level. An information-theoretic model selection approach was employed to identify the best models and predictors of student assessments and fail rates. All the top models of student feedback included a measure reflecting amount of face-to-face teaching, with the evaluation of quality of teaching being higher in units with higher proportions of lectures. However, these models explained only 12–20% of the variation in student evaluation scores, suggesting that many other factors come into play. By contrast, units with fewer lectures have lower failure rates. These results suggest that moving away from lectures and face-to-face teaching may not harm, and indeed may improve the number of students who pass the subject, but that this may be incurred at the expense of greater dissatisfaction in students' learning experience.

Piloting the feasibility of head-mounted video technology to augment student feedback during simulated clinical decision-making: an observational design pilot study

BACKGROUND: Clinical decision-making is a complex activity that is critical to patient safety. Simulation, augmented by feedback, affords learners the opportunity to learn critical clinical decision-making skills. More detailed feedback following simulation exercises has the potential to further enhance student learning, particularly in relation to developing improved clinical decision-making skills. OBJECTIVE: To investigate the feasibility of head-mounted video camera recordings, to augment feedback, following acute patient deterioration simulations. DESIGN: Pilot study using an observational design. METHODS: Ten final-year nursing students participated in three simulation exercises, each focussed on detection and management of patient deterioration. Two observers collected behavioural data using an adapted version of Gaba's Clinical Simulation Tool, to provide verbal feedback to each participant, following each simulation exercise. Participants wore a head-mounted video camera during the second simulation exercise only. Video recordings were replayed to participants to augment feedback, following the second simulation exercise. Data were collected on: participant performance (observed and perceived); participant perceptions of feedback methods; and head-mounted video camera recording feasibility and capability for detailed audio-visual feedback. RESULTS: Management of patient deterioration improved for six participants (60%). Increased perceptions of confidence (70%) and competence (80%), were reported by the majority of participants. Few participants (20%) agreed that the video recording specifically enhanced their learning. The visual field of the head-mounted video camera was not always synchronised with the participant's field of vision, thus affecting the usefulness of some recordings. CONCLUSION: The usefulness of the video recordings, to enhance verbal feedback to participants on detection and management of simulated patient deterioration, was inconclusive. Modification of the video camera glasses, to improve visual-field synchronisation with participants' actual visual field, is recommended to further explore this technology for enhancing student performance.