Student feedback systems in higher education: a focused literature review and environmental scan

In recent times, higher education institutions have paid increasing attention to the views of students to obtain feedback on their experience of learning and teaching through internal surveys. This article reviews research in the field and reports on practices in other Australian universities. Findings demonstrate that while student feedback is valued and used by all Australian universities, survey practices are idiosyncratic and in the majority of cases, questionnaires lack validity and reliability; data are used inadequately or inappropriately; and they offer limited potential for cross-sector benchmarking. The study confirms the need for institutions to develop an overarching framework for evaluation in which a valid, reliable, multidimensional and useful student feedback survey constitutes just one part. Given external expectations and internal requirements to collect feedback from students on their experience of learning and teaching, the pursuit of sound evaluation practices will continue to be of interest at local, national and international levels.

Evaluation of the quality of diverse teaching practice in higher education research methods

The purpose of this paper is to report on a methods research project investigating the evaluation of diverse teaching practice in Higher Education. The research method is a single site case study of an Australian university with data collected through published documents, surveys, interviews and focus groups. This project provides evidence of the wide variety of evaluation practice and diverse teaching practice across the university. This breadth identifies the need for greater flexibility of evaluation processes, tools and support to assist teaching staff to evaluate their diverse teaching practice. The employment opportunities for academics benchmark the university nationally and position the case study in the field. Finally this reaffirms the institutional responsibility for services to support teaching staff in an ongoing manner.

Factors affecting the motivation of TEFL academics in higher education in China

This study investigates the motivation of English language lecturers in a Chinese university. Recent studies have shown that low morale and job dissatisfaction are significant problems identified in lecturers who teach English in universities in China. Given the importance of teaching English as a second language in China, this problem has potentially significant ramifications for the nation’s future. Low staff morale is likely to be associated with less effective teaching and poor student learning outcomes. Although the problem is acknowledged, there has been limited research to understand the underlying contributing factors. To address this, a sequential explanatory mixed methods approach was adopted and implemented in two phases at a large regional university in Northern China. The participants in the main study were 100 lecturers from two colleges at this university. All of the lecturers were responsible for teaching English as a foreign language (TEFL); 50 were teaching English majors and 50 were teaching university students whose majors were not English. The research was informed by a synthesis of self determination theory and theories of organisational culture. The study found: 1) in contrast to previously reported studies, lecturers in this institution were in general autonomously motivated in teaching. 2) However, their level of motivation was influenced by their personal experiences and varied sense of competence, relatedness and autonomy. 3) In particular, personal experiences and contextual factors such as the influence of Chinese culture, societal context, and organisational climate were significant in regulating lecturers’ motivation to teach. The findings are significant for leaders in higher education who need to implement policies that foster effective work environments. The study has also provided insights into the capacity of self determination theory to explain motivation in a Chinese culture.

Towards success in higher education in engineering and technology : a coaching approach to develop holistic graduates

There is a growing gap between engineering practice and engineering education that may be contributing to less engineers practicing in industry. Coaching approach to learning and teaching has been proven to be an effective way to develop people in the workplace. A pilot coaching program is offered to Engineering and Technology students in Queensland University of Technology to enable holistic growth in order to better integrate them to the work force and society at large. The results and findings of this program will be published once the program has been completed

Action research and unit improvement in higher education : a case study approach

Student satisfaction data has been collected on a national basis in Australia since 1972. In recent years this data has been used by federal government agencies to allocate funding, and by students in selecting their universities of choice. The purpose of this paper is to present the findings of an action research project designed to identify and implement unit improvement initiatives over a three year period for an underperforming unit. This research utilises student survey data and teacher reflections to identify areas of unit improvement, with a view to aligning learning experiences, teaching and assessment to learning outcomes and improved student satisfaction. This research concludes that whilst a voluntary student survey system may be imperfect, it nevertheless provides important data that can be utilised to the benefit of the unit, learning outcomes and student satisfaction ratings, as well as wider course related outcomes. Extrapolation of these findings is recommended to other underperforming units.

Identifying discourses of moderation in higher education

Moderation of student assessment is a critical component of teaching and learning in contemporary universities. Yet, despite this, it tends to be marked by idiosyncratic and sporadic processes informed by liminal understanding. This paper, in the light of forthcoming radical national requirements for the declaration of moderation processes in tertiary curricula in Australia, will present four discourses of moderation we identified in a recent study in a Faculty of Education in a large metropolitan university. The discourses are equity, justification, community building and accountability. Together, they will act as a starting point for academics to review their beliefs and attitudes towards the moderation of student assessment.

The adoption of e-learning 2.0 in higher education by teachers and students : an investigation using mixed methods approach

This paper describes an approach to investigate the adoption of Web 2.0 in the classroom using a mixed methods study. By using a combination of qualitative or quantitative data collection and analysis techniques, we attempt to synergize the results and provide a more valid understanding of Web 2.0 adoption for learning by both teachers and students. This approach is expected to yield a better holistic view on the adoption issues associated with the e-learning 2.0 concept in current higher education as opposed to single method studies done previously. This paper also presents some early findings of e-learning 2.0 adoption using this research method

Good practice for safeguarding student learning engagement in higher education institutions : Final Report 2012

The project 'Good practice for safeguarding student learning engagement in higher education institutions' commenced in late 2010 as a Competitive Grant with funding provided by the Australian Learning and Teaching Council. The project is now overseen by the Office for Learning and Teaching within the Australian Department of Industry, Innovation, Science, Research and Tertiary Education. The project was completed in December 2012. The project was lead by QUT and comprised of the project team: Professor Karen Nelson, (project leader), Ms Tracy Creagh, (project manager) and Adjunct Professor John Clarke. Commencing in late 2010 the project invited a total of eight institutions across Australia and New Zealand (including QUT) who had either: existing programs and activities that monitored student learning engagement (MSLE); were in the early stages of implementing MSLE programs, or; who were piloting MSLE activities. As well, the project involved an advisory group and project evaluator comprising of academic and professional staff across two additional universities.

REFRAME : a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

The practice of quality assuring learning in higher education

There remains a lack of published empirical data on the substantive outcomes of higher learning and the establishment of quality processes for determining them. Studies that do exist are nationally focused with available rankings of institutions reflecting neither the quality of teaching and learning nor the diversity of institutions. This paper describes two studies in which Associate Deans from Australian higher education institutions and focus groups of management and academics identify current issues and practices in the design, development and implementation of processes for assuring the quality of learning and teaching. Results indicate that developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program necessitates knowledge creation and new inclusive processes. Common elements supporting consistently superior outcomes included: inclusivity; embedded graduate attributes; consistent and appropriate assessment; digital collection mechanisms; and systematic analysis of outcomes used in program review. Quality measures for assuring learning are proliferating nationally and changing the processes, systems and culture of higher education as a result.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

The potential of combining phenomenography, variation theory and threshold concepts to inform curriculum design in higher education

This chapter describes an innovative method of curriculum design that is based on combining phenomenographic research, and the associated variation theory of learning, with the notion of disciplinary threshold concepts to focus specialised design attention on the most significant and difficult parts of the curriculum. The method involves three primary stages: (i) identification of disciplinary concepts worthy of intensive curriculum design attention, using the criteria for threshold concepts; (ii) action research into variation in students’ understandings/misunderstandings of those concepts, using phenomenography as the research approach; (iii) design of learning activities to address the poorer understandings identified in the second stage, using variation theory as a guiding framework. The curriculum design method is inherently theory and evidence based. It was developed and trialed during a two-year project funded by the Australian Learning and Teaching Council, using physics and law disciplines as case studies. Disciplinary teachers’ perceptions of the impact of the method on their teaching and understanding of student learning were profound. Attempts to measure the impact on student learning were less conclusive; teachers often unintentionally deviated from the design when putting it into practice for the first time. Suggestions for improved implementation of the method are discussed.

Realigning capital portfolios : international students' educational experiences in higher education

This paper reports on first year experiences of international students who use English as an additional language (EAL) in higher education in Australia. It examines how valued resources can foster a positive educational experience of these students from sociological perspectives. It draws data from an interview study, exploring narrative accounts of 17 EAL international students from nine countries about their educational relations and strategies across their first year of study. Their narratives were analysed through Bourdieu's concepts of field, habitus, capital and legitimation, as well as tools of narrative inquiry. The paper finds that the students took up strategies to realign their capital portfolios with new rules of the game. Their decisions were dependent on their personal trajectories and conditions on offer. This paper suggests that more effort needs to be made to understand international students' differentiated access to valued resources in higher education.

Possible, inevitable or fait accompli? An analysis of streaming video acquisition, acceptance and use in higher education

This paper explores the potential for online video as a mechanism to transform the ways students learn, as measured by research, user experience and usage following surveys and trials of patron-driven acquisition collaboratively undertaken by Queensland University of Technology, La Trobe University and Kanopy.

E-motion diaries : blogging emotional responses to online learning in higher education

Emotions are inherently social, and are central to learning, online interaction and literacy practices (Shen, Wang, & Shen, 2009). Demonstrating the dynamic sociality of literacy practice, we used e-motion diaries or web logs to explore the emotional states of pre-service high school teachers’ experiences of online learning activities. This is because the methods of communication used by university educators in online learning and writing environments play an important role in fulfilling students’ need for social interaction and inclusion (McInnerney & Roberts, 2004). Feelings of isolation and frustration are common emotions experienced by students in many online learning environments, and are associated with the success or failure of online interactions and learning (Su, et al., 2005). The purpose of the study was to answer the research question: What are the trajectories of pre-service teachers’ emotional states during online learning experiences? This is important because emotions are central to learning, and the current trend toward Massive Open Online Courses (MOOCs) needs research about students’ emotional connections in online learning environments (Kop, 2011). The project was conducted with a graduate class of 64 high school science pre-service teachers in Science Education Curriculum Studies in a large Australian university, including males and females from a variety of cultural backgrounds, aged 22-55 years. Online activities involved the students watching a series of streamed live lectures for the first 5 weeks providing a varied set of learning experiences, such as viewing science demonstrations (e.g., modeling the use of discrepant events). Each week, students provided feedback on learning by writing and posting an e-motion diary or web log about their emotional response. Students answered the question: What emotions did you experience during this learning experience? The descriptive data set included 284 online posts, with students contributing multiple entries. Linguistic appraisal theory, following Martin and White (2005), was used to regroup the 22 different discrete emotions reported by students into the six main affect groups – three positive and three negative: unhappiness/happiness, insecurity/security, and dissatisfaction/satisfaction. The findings demonstrated that the pre-service teachers’ emotional responses to the streamed lectures tended towards happiness, security, and satisfaction within the typology of affect groups – un/happiness, in/security, and dis/satisfaction. Fewer students reported that the streamed lectures triggered negative feelings of frustration, powerlessness, and inadequacy, and when this occurred, it often pertained to expectations of themselves in the forthcoming field experience in classrooms. Exceptions to this pattern of responses occurred in relation to the fifth streamed lecture presented in a non-interactive slideshow format that compressed a large amount of content. Many students responded to the content of the lecture rather than providing their emotional responses to this lecture, and one student felt “completely disengaged”. The social practice of online writing as blogs enabled the students to articulate their emotions. The findings primarily contribute new understanding about students' wide range of differing emotional states, both positive and negative, experienced in response to streamed live lectures and other learning activities in higher education external coursework. The is important because the majority of previous studies have focused on particular negative emotions, such as anxiety in test taking. The research also highlights the potentials of appraisal theory for studying human emotions in online learning and writing.