The systematic influences on student evaluation of teaching in engineering education

BACKGROUND Student evaluation of teaching (SET) has a long history, has grown in prevalence and importance over a period of decades, and is now common-place in many universities internationally. SET data are collected for a range of purposes, including: as diagnostic feedback to improve the quality of teaching and learning; as an input to staff performance management processes and personnel decisions such as promotion for staff; to provide information to prospective students in their selection of courses and programs; and as a source of data for research on teaching. Rovai et al. (2006) report that while SET research provides mixed results, there is evidence that, for course-related factors, smaller classes are rated more favourably than large classes, upper-year-level classes are rated more favourably than lower-year classes, and that there are rating differences between discipline areas. While additional course-related factors are also noted, other reviews of the literature on SET also identify these three factors as commonly reported systematic influences on SET ratings. The School of Engineering at Deakin University in Australia offers undergraduate and postgraduate engineering programs, and these programs are delivered in both on-campus and off-campus modes.PURPOSEThe paper presents a quantitative investigation of SET data for the School of Engineering at Deakin University to identify whether the commonly reported systematic influences on SET ratings of class size and year level are also observed here. The influence of online mode of offer is also explored.DESIGN/METHOD Deakin University’s Student Evaluation of Teaching and Units (SETU) questionnaire is administered to students enrolled in every unit of study every time that unit is offered, unless it is specially exempted. Following data collation, summary results are reported via a public website. The publicly available SETU data for all School of Engineering units of study were collected for a two year period. The collected data were subjected to analysis of variance (ANOVA) analysis to identify any significant systematic influences on mean student SETU ratings.RESULTS SETU data from 100 separate units of study over the two year period were collected, representing 3375 sets of SETU ratings, and covering unit enrolment sizes from 12 to 462 students. Although this was a modest sized investigation, significantly higher mean ratings for some SETU items were observed for units with small enrolments, for postgraduate level units compared to undergraduate level units, and for units offered in conventional mode compared to online mode of offer. The presence of the commonly observed systematic influences on SET ratings was confirmed.CONCLUSIONS While the use of SET data may have originally been primarily for formative purposes to improve teaching and learning, they are also increasingly used for summative judgements of teaching quality and teaching staff performance that have implications for personnel decision making. There may be an acceptance of the need for SET, however there remains no universal consensus as to what constitutes quality in university teaching and learning, and the increasing use of SET for high-stakes decision making puts pressure on institutions to ensure that their SET practices are sound, equitable and defensible.

The impact of teaching models, group structures and assessment modes on cooperative learning in the student design studio

As a result of ever diminishing teaching resources, an increasing number of architectural educators are setting group design projects, rather then spreading their time thinly over a large number of individual projects. This allows them to co-ordinate longer and more in-depth review sessions on a smaller number of assignment submissions. However, while the group
model may offer an authentic learning model by reflecting design in practice, the approach is not without its obvious shortcomings as a teaching archetype for the assessment of the knowledge and skill competencies of individual students. Hence, what is clear is the need for a readily adoptable andragogy for the teaching and assessment of group design projects.
The following paper describes the background, methodology and findings of a Strategic Teaching and Learning Grant funded research project carried out in the year 2005 at the School of Architecture and Building at Deakin University. The project aimed to inform a change of classroom/studio practice governing the assemblage, teaching and assessment of student design teams. The development through these changes of cooperative and student centred learning principles focused on effective design collaboration and fair assessment should, it will be argued, lead to an enhanced group-learning experience in studio, which will subsequently and ultimately enhance professional practice.

Taking responsibility for learning isn't everything : a case for developing tertiary students' conceptions of learning

This study examines first year university student perceptions of responsibility for their learning, within the context of their conceptions of learning, with a view to meeting two of the objectives of higher education in Australia: teaching students to think and to learn. A questionnaire was distributed to 100 students undertaking at least one first year subject at the University of Western Sydney (UWS) in 1998. Their written responses provided information about their conceptions of learning, as well as both direct and indirect indications of their perceptions of responsibility. Results indicated that students held perceptions of personal responsibility for their learning, but that their conceptions of learning were essentially quantitative in nature and were at the lower levels of complexity. The implications of these findings are discussed in terms of university teaching and learning, and of meeting the ultimate objectives of higher education.

Comparative evaluation of PBL and traditional lecturebased teaching in undergraduate engineering courses : evidence from controlled learning environment

Project and problem-based learning (PBL) has been widely recognised as an active, collaborative, cumulative and integrative learning approach that engages learners, motivates team creativity and centres on practical education. On the other hand, traditional lecture-tutorial teaching is often criticised for being a passive, surface learning and exam-focused approach. In spite of these evidence-based observations and claims over the years, the traditional lecture-tutorial teaching approach still dominates as the preferred teaching approach at Australian universities. This study sets up a control environment to compare these two teaching and learning approaches by analysing data from students' actual performance, course evaluation and expectation in two large undergraduate engineering courses in 2009 and 2010. The evidence reported in this study is broadly interesting in that both courses were taught by the same teaching staff using two entirely different learning and teaching approaches to the same cohort of students in the same semester within the same degree program. The analysis shows that there are significant differences between the students' actual performance, course evaluation and their expectation. Such conflicting differences may be some of the reasons that may negatively impact teaching staff deterring them from switching to PBL from traditional lecture-tutorial teaching.

Fostering cultural diversity in Australia: a case study of teaching, learning and assessing African musical arts

Australia is forged by ongoing migration welcoming a range of cultures, languages and ethnicities thus celebrating a diverse range of musical arts. In this multicultural society, music and dance may serve as a positive medium to transmit and promote social cohesion. I argue that the inclusion of innovative and immersive practice of African music in particular where authentic teaching and learning is facilitated may help foster understandings of culture in educational settings and the wider society. As a migrant forming part of the African Diaspora in Melbourne, I am strongly connected to my ancestral homeland (South Africa) when teaching African music to Australian tertiary students. Having gained ethical clearance to undertake the two research projects at Deakin University in Melbourne (Attitudes and perceptions of Arts Education Students: preparing culturally responsive teachers and Pre-service teacher attitudes and understandings of Music Education), I discuss tertiary students experience in relation to the teaching and learning of African music within higher education courses. Drawing on interview, questionnaire, observation notes, anecdotal feedback and narrative reflection, I employ Interpretative Phenomenological Analysis to analyse and code the data into themes. By offering a discussion of assessment and evaluation, I explore and invite international dialogue in regards to how best we can prepare, assess and evaluate our students to improve the quality of musical arts education.