A comparison of student evaluation of teaching with student performance

This study measures the evaluation of teaching given by students against their final outcomes in a subject. The subject in question had an enrolment across four campuses of 1073 students at the time of the evaluation and is a statistics subject that is core (i.e. compulsory) to several undergraduate business degrees. This study is based on the 373 students (34.8%) who responded to the survey, and their final results. The evaluations were open for a period of six weeks leading up to and just after the final exam. The study matches the responses to the question “This unit was well taught” to final outcomes, in an attempt to ascertain whether there is a link between student evaluation of teaching and performance. The analysis showed that for the students who self -selected to complete the survey:

· Students who perform well in the subject generally give higher scores than lower performing students.

· The same general pattern prevailed when other secondary factors were taken into account, such as, when the evaluation was completed, campus and gender.

· The timing of when a student completes the evaluation seems the most important of these secondary variables.

· In general, students who submitted their evaluations after the exam gave higher ratings if they eventually obtained a pass grade or better, and lower grades if they failed.

An institutional study of the influence of ‘onlineness’ on student evaluation of teaching in a dual mode Australian university

Student evaluation of teaching (SET) is now commonplace in many universities internationally. The most common criticism of SET practices is that they are influenced by a number of non-teaching-related factors. More recently, there has been dramatic growth in online education internationally, but only limited research on the use of SET to evaluate online teaching. This paper presents a large-scale and detailed investigation, using the institutional SET data from an Australian university with a significant offering of wholly online units, and whose institutional SET instrument contains items relating to student perceptions of online technologies in teaching and learning. The relationship between educational technology and SET is not neutral. The mean ratings for the ‗online‘ aspects of SET are influenced by factors in the wider teaching and learning environment, and the overall perception of teaching quality is influenced by whether a unit is offered in wholly online mode or not.

Student evaluation of teaching data 2009-2010

This dataset consists of a summary of publicly available student evaluation of teaching (SET) data for the annual period of trimester 2 2009, trimester 3 2009/2010 and trimester 1 2010, from the Student Evaluation of Teaching and Units (SETU) portal.

The data was analysed to include mean rating sets for 1432 units of study, and represented 74498 sets of SETU ratings, 188391 individual student enrolements and 58.5 percent of all units listed in the Deakin University handbook for the period under consideration, to identify any systematic influences on SET results at Deakin University.

The data reported for a unit included:
• total enrolment;
• total number of responses; and
• computed response rate for the enrolment location(s) selected

And, the data reported for each of the ten core SETU items included:
• number of responses;
• mean rating;
• standard deviation of the mean rating;
• percentage agreement;
• percentage disagreement; and
• percentage difference.

Student evaluation of teaching : keeping in touch with reality

Student evaluation of teaching is commonplace in many universities and may be the predominant input into the performance evaluation of staff and organisational units. This article used publicly available student evaluation of teaching data to present examples of where institutional responses to evaluation processes appeared to be educationally ineffective and where the pursuit of the ‘right’ student evaluation results appears to have been mistakenly equated with the aim of improved teaching and learning. If the vast resources devoted to student evaluation of teaching are to be effective, then the data produced by student evaluation systems must lead to real and sustainable improvements in teaching quality and student learning, rather than becoming an end in itself.

The performance of a student evaluation of teaching system

Student evaluation of teaching (SET) is now commonplace in many universities internationally. While much effort has been devoted to examining the statistical validity of SET instruments, there has been limited examination of the methodological and consequential validity (together referred to as ‘utility’) of the ways in which SET data are used. This paper examines the SET system at Deakin University from the perspective of utility. It draws on publicly available SET results for an entire annual cycle of unit offerings. Consideration is given to the representativeness of the data produced, and to the utility of the data reported, by the system. While this investigation focuses on the SET system currently employed at Deakin University, it offers both an analysis methodology and conclusions that can be applied more generally.

Updating RIGs : including the systematic influence of online study on student evaluation of teaching

Based on student evaluation of teaching (SET) ratings from 1,432 units of study over a period of a year, representing 74,490 individual sets of ratings, and including a significant number of units offered in wholly online mode, we confirm the significant influence of class size, year level, and discipline area on at least some SET ratings. We also find online mode of offer to significantly influence at least some SET ratings. We reveal both the statistical significance and effect sizes of these influences, and find that the magnitudes of the effect sizes of all factors are small, but potentially cumulative. We also show that the influence of online mode of offer is of the same magnitude as the other 3 factors. These results support and extend the rating interpretation guides (RIGs) model proposed by Neumann and colleagues, and we present a general method for the development of a RIGs system.

The impact of increasing course enrolment on student evaluation of teaching in engineering education

Student evaluation of teaching (SET) is important, commonplace and may be used in staff performance management. The SET literature suggests that class size is a negative systematic influence on SET ratings. In this paper we investigate time-series SET data from a large first-year engineering class where a decline in SET ratings was observed over time as course enrolment increased. We observe a negative halo effect of increasing class size on mean SET ratings and conclude that increasing course enrolment leads to a significant reduction in all mean SET ratings, even when the course learning design remains essentially unchanged. We also find an additional differential effect of increasing course enrolment on mean SET ratings. We observe that the marginal reduction in mean SET ratings for each additional student in the course enrolment is greater for those aspects of the student learning experience that are likely to be most directly impacted by increasing class size. We provide implications for practice from these findings.

The systematic influences on student evaluation of teaching in engineering education

BACKGROUND Student evaluation of teaching (SET) has a long history, has grown in prevalence and importance over a period of decades, and is now common-place in many universities internationally. SET data are collected for a range of purposes, including: as diagnostic feedback to improve the quality of teaching and learning; as an input to staff performance management processes and personnel decisions such as promotion for staff; to provide information to prospective students in their selection of courses and programs; and as a source of data for research on teaching. Rovai et al. (2006) report that while SET research provides mixed results, there is evidence that, for course-related factors, smaller classes are rated more favourably than large classes, upper-year-level classes are rated more favourably than lower-year classes, and that there are rating differences between discipline areas. While additional course-related factors are also noted, other reviews of the literature on SET also identify these three factors as commonly reported systematic influences on SET ratings. The School of Engineering at Deakin University in Australia offers undergraduate and postgraduate engineering programs, and these programs are delivered in both on-campus and off-campus modes.PURPOSEThe paper presents a quantitative investigation of SET data for the School of Engineering at Deakin University to identify whether the commonly reported systematic influences on SET ratings of class size and year level are also observed here. The influence of online mode of offer is also explored.DESIGN/METHOD Deakin University’s Student Evaluation of Teaching and Units (SETU) questionnaire is administered to students enrolled in every unit of study every time that unit is offered, unless it is specially exempted. Following data collation, summary results are reported via a public website. The publicly available SETU data for all School of Engineering units of study were collected for a two year period. The collected data were subjected to analysis of variance (ANOVA) analysis to identify any significant systematic influences on mean student SETU ratings.RESULTS SETU data from 100 separate units of study over the two year period were collected, representing 3375 sets of SETU ratings, and covering unit enrolment sizes from 12 to 462 students. Although this was a modest sized investigation, significantly higher mean ratings for some SETU items were observed for units with small enrolments, for postgraduate level units compared to undergraduate level units, and for units offered in conventional mode compared to online mode of offer. The presence of the commonly observed systematic influences on SET ratings was confirmed.CONCLUSIONS While the use of SET data may have originally been primarily for formative purposes to improve teaching and learning, they are also increasingly used for summative judgements of teaching quality and teaching staff performance that have implications for personnel decision making. There may be an acceptance of the need for SET, however there remains no universal consensus as to what constitutes quality in university teaching and learning, and the increasing use of SET for high-stakes decision making puts pressure on institutions to ensure that their SET practices are sound, equitable and defensible.

Overseas students evaluation of teaching effectiveness

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Responding to student evaluation of teaching : closing the loop

Our undergraduate curriculum focuses on preparing students to be critical thinkers, problem solvers, and knowledgeable, responsible information technology (IT) professionals. Students often question the relevance of core subjects which are designed to introduce and develop these attributes; such units regularly perform poorly in student evaluation of teaching. Although these subjects are strongly grounded in meeting industry expectations regarding employability skills, students do not value the learning as the curriculum is deemed non-technical and not worthy of inclusion in an IT qualification. As these are core subjects, improving student perceptions and learning outcomes is of critical importance. Simply responding to student feedback received via student evaluation of teaching had not resulted in any improvement; so a holistic approach was adopted. A workshop was organised to explore the issues and develop a strategy to improve outcomes and perceptions of these subjects. Rather than addressing student feedback specifically, the underlying issues that led to students’ negative perceptions were identified and addressed. Recommendations were implemented in 2014 resulting in improvements in student evaluation for relevant subjects. In this paper, we will discuss the process that was adopted to respond effectively to student evaluations, and explore the impact that this had on them.

Exploring the impact of faculty reflection on weekly student evaluations of teaching

This exploratory study considered Larrivee’s assessment of teachers’ reflective practice levels by using a formative, weekly, online student evaluation of teaching (SET) tool through a virtual learning environment (VLE) as a means to encourage reflective practice. In-depth interviews were conducted with six faculty members in three departments at a university college in the UK. The study found that: (a) faculty who experienced surface-level reflection were more likely to have a reactive reflection style; and (b) faculty who experienced higher levels of reflection were more likely to have a proactive reflection style. Overall, the tool was found to be an efficient means of encouraging reflection by all participants and demonstrated that reflective practice could come about as a result of these weekly formative SETs. The study concludes with suggestions for academic development and future research on reflection that could be conducted using SETs via a VLE.

An evaluation of English language instruction in universities/colleges in Taiwan

The purpose of this study is about to survey the effectiveness of English teaching in universities/colleges in Taiwan and provide suggestions to the educational authorities for upgrading or improving the English language teaching techniques or methods currently in use. On the one hand, the suggestions of English teachers in universities/colleges are the main concerns in this study. On the other hand, equal attention is given to the opinions and learning attitudes of the students, including the interaction between the teachers and the students.^ A social investigation methodology, consisting of questionnaire and data analysis, is devoted to this evaluation and analysis of teaching effectiveness. Among the significant findings reported in this study is the fact that both students and teachers consider evaluation of teaching effectiveness as a very important aspect of English Language programs at the university/college level. Another significant finding is that students are less concerned about the grades earned or awarded in English Language courses and more concerned about how much they learned in these classes.^ The analysis and suggestions based on the results for English teaching evaluation of universities/colleges in this dissertation can help teachers modify their teaching techniques and methods, the students to adjust their learning attitudes, and the authorities to schematize the English teaching plan that can facilitate the teaching goals to be realized. ^

A text analytics evaluation of a first-year engineering project-based unit

In the undergraduate engineering program at Griffith University in Australia, the unit 1006ENG Design and Professional Skills aims to provide an introduction to engineering design and professional practice through a project-based learning (PBL) approach to problem solving. It provides students with an experience of PBL in the first-year of their programme. The unit comprises an underpinning lecture series, design work including group project activities, an individual computer-aided drawing exercise/s and an oral presentation. Griffith University employs a ‘Student Experience of Course’ (SEC) online survey as part of its student evaluation of teaching, quality improvement and staff performance management processes. As well as numerical response scale items, it includes the following two questions inviting open-ended text responses from students: i) What did you find particularly good about this course? and ii) How could this course be improved? The collection of textual data in in student surveys is commonplace, due to the rich descriptions of respondent experiences they can provide at relatively low cost. However, historically these data have been underutilised because they are time consuming to analyse manually, and there has been a lack of automated tools to exploit such data efficiently. Text analytics approaches offer analysis methods that result in visual representations of comment data that highlight key individual themes in these data and the relationships between those themes. We present a text analytics-based evaluation of the SEC open-ended comments received in the first two years of offer of the PBL unit 1006ENG. We discuss the results obtained in detail. The method developed and documented here is a practical and useful approach to analysing/visualising open-ended comment data that could be applied by others with similar comment data sets.