A comparison of student evaluation of teaching with student performance

This study measures the evaluation of teaching given by students against their final outcomes in a subject. The subject in question had an enrolment across four campuses of 1073 students at the time of the evaluation and is a statistics subject that is core (i.e. compulsory) to several undergraduate business degrees. This study is based on the 373 students (34.8%) who responded to the survey, and their final results. The evaluations were open for a period of six weeks leading up to and just after the final exam. The study matches the responses to the question “This unit was well taught” to final outcomes, in an attempt to ascertain whether there is a link between student evaluation of teaching and performance. The analysis showed that for the students who self -selected to complete the survey:

· Students who perform well in the subject generally give higher scores than lower performing students.

· The same general pattern prevailed when other secondary factors were taken into account, such as, when the evaluation was completed, campus and gender.

· The timing of when a student completes the evaluation seems the most important of these secondary variables.

· In general, students who submitted their evaluations after the exam gave higher ratings if they eventually obtained a pass grade or better, and lower grades if they failed.

An institutional study of the influence of ‘onlineness’ on student evaluation of teaching in a dual mode Australian university

Student evaluation of teaching (SET) is now commonplace in many universities internationally. The most common criticism of SET practices is that they are influenced by a number of non-teaching-related factors. More recently, there has been dramatic growth in online education internationally, but only limited research on the use of SET to evaluate online teaching. This paper presents a large-scale and detailed investigation, using the institutional SET data from an Australian university with a significant offering of wholly online units, and whose institutional SET instrument contains items relating to student perceptions of online technologies in teaching and learning. The relationship between educational technology and SET is not neutral. The mean ratings for the ‗online‘ aspects of SET are influenced by factors in the wider teaching and learning environment, and the overall perception of teaching quality is influenced by whether a unit is offered in wholly online mode or not.

Student evaluation of teaching data 2009-2010

This dataset consists of a summary of publicly available student evaluation of teaching (SET) data for the annual period of trimester 2 2009, trimester 3 2009/2010 and trimester 1 2010, from the Student Evaluation of Teaching and Units (SETU) portal.

The data was analysed to include mean rating sets for 1432 units of study, and represented 74498 sets of SETU ratings, 188391 individual student enrolements and 58.5 percent of all units listed in the Deakin University handbook for the period under consideration, to identify any systematic influences on SET results at Deakin University.

The data reported for a unit included:
• total enrolment;
• total number of responses; and
• computed response rate for the enrolment location(s) selected

And, the data reported for each of the ten core SETU items included:
• number of responses;
• mean rating;
• standard deviation of the mean rating;
• percentage agreement;
• percentage disagreement; and
• percentage difference.

Student evaluation of teaching : keeping in touch with reality

Student evaluation of teaching is commonplace in many universities and may be the predominant input into the performance evaluation of staff and organisational units. This article used publicly available student evaluation of teaching data to present examples of where institutional responses to evaluation processes appeared to be educationally ineffective and where the pursuit of the ‘right’ student evaluation results appears to have been mistakenly equated with the aim of improved teaching and learning. If the vast resources devoted to student evaluation of teaching are to be effective, then the data produced by student evaluation systems must lead to real and sustainable improvements in teaching quality and student learning, rather than becoming an end in itself.

The performance of a student evaluation of teaching system

Student evaluation of teaching (SET) is now commonplace in many universities internationally. While much effort has been devoted to examining the statistical validity of SET instruments, there has been limited examination of the methodological and consequential validity (together referred to as ‘utility’) of the ways in which SET data are used. This paper examines the SET system at Deakin University from the perspective of utility. It draws on publicly available SET results for an entire annual cycle of unit offerings. Consideration is given to the representativeness of the data produced, and to the utility of the data reported, by the system. While this investigation focuses on the SET system currently employed at Deakin University, it offers both an analysis methodology and conclusions that can be applied more generally.

Updating RIGs : including the systematic influence of online study on student evaluation of teaching

Based on student evaluation of teaching (SET) ratings from 1,432 units of study over a period of a year, representing 74,490 individual sets of ratings, and including a significant number of units offered in wholly online mode, we confirm the significant influence of class size, year level, and discipline area on at least some SET ratings. We also find online mode of offer to significantly influence at least some SET ratings. We reveal both the statistical significance and effect sizes of these influences, and find that the magnitudes of the effect sizes of all factors are small, but potentially cumulative. We also show that the influence of online mode of offer is of the same magnitude as the other 3 factors. These results support and extend the rating interpretation guides (RIGs) model proposed by Neumann and colleagues, and we present a general method for the development of a RIGs system.

The impact of increasing course enrolment on student evaluation of teaching in engineering education

Student evaluation of teaching (SET) is important, commonplace and may be used in staff performance management. The SET literature suggests that class size is a negative systematic influence on SET ratings. In this paper we investigate time-series SET data from a large first-year engineering class where a decline in SET ratings was observed over time as course enrolment increased. We observe a negative halo effect of increasing class size on mean SET ratings and conclude that increasing course enrolment leads to a significant reduction in all mean SET ratings, even when the course learning design remains essentially unchanged. We also find an additional differential effect of increasing course enrolment on mean SET ratings. We observe that the marginal reduction in mean SET ratings for each additional student in the course enrolment is greater for those aspects of the student learning experience that are likely to be most directly impacted by increasing class size. We provide implications for practice from these findings.

The systematic influences on student evaluation of teaching in engineering education

BACKGROUND Student evaluation of teaching (SET) has a long history, has grown in prevalence and importance over a period of decades, and is now common-place in many universities internationally. SET data are collected for a range of purposes, including: as diagnostic feedback to improve the quality of teaching and learning; as an input to staff performance management processes and personnel decisions such as promotion for staff; to provide information to prospective students in their selection of courses and programs; and as a source of data for research on teaching. Rovai et al. (2006) report that while SET research provides mixed results, there is evidence that, for course-related factors, smaller classes are rated more favourably than large classes, upper-year-level classes are rated more favourably than lower-year classes, and that there are rating differences between discipline areas. While additional course-related factors are also noted, other reviews of the literature on SET also identify these three factors as commonly reported systematic influences on SET ratings. The School of Engineering at Deakin University in Australia offers undergraduate and postgraduate engineering programs, and these programs are delivered in both on-campus and off-campus modes.PURPOSEThe paper presents a quantitative investigation of SET data for the School of Engineering at Deakin University to identify whether the commonly reported systematic influences on SET ratings of class size and year level are also observed here. The influence of online mode of offer is also explored.DESIGN/METHOD Deakin University’s Student Evaluation of Teaching and Units (SETU) questionnaire is administered to students enrolled in every unit of study every time that unit is offered, unless it is specially exempted. Following data collation, summary results are reported via a public website. The publicly available SETU data for all School of Engineering units of study were collected for a two year period. The collected data were subjected to analysis of variance (ANOVA) analysis to identify any significant systematic influences on mean student SETU ratings.RESULTS SETU data from 100 separate units of study over the two year period were collected, representing 3375 sets of SETU ratings, and covering unit enrolment sizes from 12 to 462 students. Although this was a modest sized investigation, significantly higher mean ratings for some SETU items were observed for units with small enrolments, for postgraduate level units compared to undergraduate level units, and for units offered in conventional mode compared to online mode of offer. The presence of the commonly observed systematic influences on SET ratings was confirmed.CONCLUSIONS While the use of SET data may have originally been primarily for formative purposes to improve teaching and learning, they are also increasingly used for summative judgements of teaching quality and teaching staff performance that have implications for personnel decision making. There may be an acceptance of the need for SET, however there remains no universal consensus as to what constitutes quality in university teaching and learning, and the increasing use of SET for high-stakes decision making puts pressure on institutions to ensure that their SET practices are sound, equitable and defensible.

Responding to student evaluation of teaching : closing the loop

Our undergraduate curriculum focuses on preparing students to be critical thinkers, problem solvers, and knowledgeable, responsible information technology (IT) professionals. Students often question the relevance of core subjects which are designed to introduce and develop these attributes; such units regularly perform poorly in student evaluation of teaching. Although these subjects are strongly grounded in meeting industry expectations regarding employability skills, students do not value the learning as the curriculum is deemed non-technical and not worthy of inclusion in an IT qualification. As these are core subjects, improving student perceptions and learning outcomes is of critical importance. Simply responding to student feedback received via student evaluation of teaching had not resulted in any improvement; so a holistic approach was adopted. A workshop was organised to explore the issues and develop a strategy to improve outcomes and perceptions of these subjects. Rather than addressing student feedback specifically, the underlying issues that led to students’ negative perceptions were identified and addressed. Recommendations were implemented in 2014 resulting in improvements in student evaluation for relevant subjects. In this paper, we will discuss the process that was adopted to respond effectively to student evaluations, and explore the impact that this had on them.

Exploring the impact of faculty reflection on weekly student evaluations of teaching

This exploratory study considered Larrivee’s assessment of teachers’ reflective practice levels by using a formative, weekly, online student evaluation of teaching (SET) tool through a virtual learning environment (VLE) as a means to encourage reflective practice. In-depth interviews were conducted with six faculty members in three departments at a university college in the UK. The study found that: (a) faculty who experienced surface-level reflection were more likely to have a reactive reflection style; and (b) faculty who experienced higher levels of reflection were more likely to have a proactive reflection style. Overall, the tool was found to be an efficient means of encouraging reflection by all participants and demonstrated that reflective practice could come about as a result of these weekly formative SETs. The study concludes with suggestions for academic development and future research on reflection that could be conducted using SETs via a VLE.

Investigating sample bias in student evaluations of teaching

Student evaluations of teaching have increased in importance to universities in Australia over recent years due to changes in government policy. There has been significant debate in the literature as to the validity and usefulness of such evaluations and as to whether students who respond to the evaluations are indeed representative of the student population. A potential invalidating issue is self selection in the evaluation process. In this paper, we consider student evaluations of a large first year business statistics subject that had 1073 eligible students enrolled across four campuses at the time of the evaluation. The study is based on the 373 students (34.8%) who responded to the survey, and their final results. The evaluations were open for a period of six weeks leading up to and just after the final exam. The study looks in detail at the student population identifying such attributes as gender; home campus; course of study; domestic/international; Commonwealth Supported Place/full fee paying, etc. and then mapping these results to those of the students who responded to the survey.

Learning approaches and student perceptions of teaching quality

This research investigated the links between students' personal characteristics, their approaches to learning and their perceptions of teaching quality in a first-year accounting unit. Significant learning differences were found between Australian and Chinese students, and strong inter-relationships between contextual factors, such as teaching quality, and learning approaches were identified.

If you build it will they come? Exploring the student perspective of weekly student evaluations of teaching

This exploratory study investigated the students’ use of formative, weekly, online evaluations of teaching through a virtual learning environment. Results were based on in-depth interviews of seven students at a rural university college in the UK. Students from different genders, education levels and backgrounds volunteered for the study. The students thought it was a good tool and useful for providing anonymous feedback. However, their motivation to fill in the evaluations every week varied throughout the period of study, and the weekly feedback soon became routine and too onerous a task, and thus had a tendency towards being superficially conducted. Students were more inclined to comment on negative issues, rather than critically analyse positive ones. They also tended to be more positive towards conducting the evaluation if the lecturer discussed them and/or made changes to their future lectures.