Student feedback systems in higher education: a focused literature review and environmental scan

In recent times, higher education institutions have paid increasing attention to the views of students to obtain feedback on their experience of learning and teaching through internal surveys. This article reviews research in the field and reports on practices in other Australian universities. Findings demonstrate that while student feedback is valued and used by all Australian universities, survey practices are idiosyncratic and in the majority of cases, questionnaires lack validity and reliability; data are used inadequately or inappropriately; and they offer limited potential for cross-sector benchmarking. The study confirms the need for institutions to develop an overarching framework for evaluation in which a valid, reliable, multidimensional and useful student feedback survey constitutes just one part. Given external expectations and internal requirements to collect feedback from students on their experience of learning and teaching, the pursuit of sound evaluation practices will continue to be of interest at local, national and international levels.

Finding your research voice : the application of bibliometrics as an analytic literature tool on the pedagogic field of higher education

This paper documents the use of bibliometrics as a methodology to bring forth a structured, systematic and rigorous way to analyse and evaluate a range of literature. When starting out and reading broadly for my doctoral studies, one article by Trigwell and Prosser (1996b) led me to reflect about my level of comprehension as the content, concepts and methodology did not resonate with my epistemology. A disconnection between our paradigms emerged. Further reading unveiled the work by Doyle (1987) who categorised research in teaching and teacher education by three main areas: teacher characteristics, methods research and teacher behaviour. My growing concerns that there were gaps in the knowledge also exposed the difficulties in documenting said gaps. As an early researcher who required support to locate myself in the field and to find my research voice, I identified bibliometrics (Budd, 1988; Yeoh & Kaur, 2007) as an appropriate methodology to add value and rigour in three ways. Firstly, the application of bibliometrics to analyse articles is systematic, builds a picture from the characteristics of the literature, and offers a way to elicit themes within the categories. Secondly, by systematic analysis there is occasion to identify gaps within the body of work, limitations in methodology or areas in need of further research. Finally, extension and adaptation of the bibliometrics methodology, beyond citation or content analysis, to investigate the merit of methodology, participants and instruments as a determinant for research worth allowed the researcher to build confidence and contribute new knowledge to the field. Therefore, this paper frames research in the pedagogic field of Higher Education through teacher characteristics, methods research and teacher behaviour, visually represents the literature analysis and locates my research self within methods research. Through my research voice I will present the bibliometrics methodology, the outcomes and document the landscape of pedagogy in the field of Higher Education.

The pedagogical balancing act : teaching reflection in higher education

Despite the common use of the term reflection in higher education assessment tasks, learners are not often taught how to communicate their disciplinary knowledge through reflection. This paper argues that students can and should be taught how to reflect in deep and transformative ways. It highlights the reflexive pedagogical balancing act of attending to different levels of reflection as a way to stimulate focused, thoughtful and reasoned reflections that show evidence of new ways of thinking and doing. The paper uses data from a current project to illustrate the effects of focusing on particular levels of reflection in the pedagogical strategies used, and argues that while the goal of academic or professional reflection is generally to move students to the highest level of reflection to transform their learning/practice, unless higher education teachers attend to every level of reflection, there are specific, observable gaps in the reflections that students produce.

Higher degree research at Australian Universities : Responding to diversity in engineering and information technology

BACKGROUND There is increasing enrolment of international students in the Engineering and Information Technology disciplines and anecdotal evidence of a need for additional understanding and support for these students and their supervisors due to differences both in academic and social cultures. While there is a growing literature on supervisory styles and guidelines on effective supervision, there is little on discipline-specific, cross-cultural supervision responding to the growing diversity. In this paper, we report findings from a study of Engineering and Information technology Higher Degree Research (HDR)students and supervision in three Australian universities. PURPOSE The aim was to assess perceptions of students and supervisors of factors influencing success that are particular to international or culturally and linguistically diverse (CaLD) HDR students in Engineering and Information technology. DESIGN/METHOD Online survey and qualitative data was collected from international and CaLD HDR students and supervisors at the three universities. Bayesian network analysis, inferential statistics, and qualitative analysis provided the main findings. RESULTS Survey results indicate that both students and supervisors are positive about their experiences, and do not see language or culture as particularly problematic. The survey results also reveal strong consistency between the perceptions of students and supervisors on most factors influencing success. Qualitative analysis of critical supervision incidents has provided rich data that could help improve support services. CONCLUSIONS In contrast with anecdotal evidence, HDR completion data from the three universities reveal that international students, on average, complete in shorter time periods than domestic students. The analysis suggests that success is linked to a complex set of factors involving the student, supervision, the institution and broader community.

The practice of quality assuring learning in higher education

There remains a lack of published empirical data on the substantive outcomes of higher learning and the establishment of quality processes for determining them. Studies that do exist are nationally focused with available rankings of institutions reflecting neither the quality of teaching and learning nor the diversity of institutions. This paper describes two studies in which Associate Deans from Australian higher education institutions and focus groups of management and academics identify current issues and practices in the design, development and implementation of processes for assuring the quality of learning and teaching. Results indicate that developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program necessitates knowledge creation and new inclusive processes. Common elements supporting consistently superior outcomes included: inclusivity; embedded graduate attributes; consistent and appropriate assessment; digital collection mechanisms; and systematic analysis of outcomes used in program review. Quality measures for assuring learning are proliferating nationally and changing the processes, systems and culture of higher education as a result.

Educating for digital futures : what the learning strategies of digital content professionals can teach higher education

Beginning in the second half of the 20th century, ICTs transformed many societies from industrial societies in which manufacturing was the central focus, into knowledge societies in which dealing effectively with data and information has become a central element of work (Anderson, 2008). To meet the needs of the knowledge society, universities must reinvent their structures and processes, their curricula and pedagogic practices. In addition to this, of course higher education is itself subject to the sweeping influence of ICTs. But what might effective higher education look like in the 21st century? In designing higher education systems and learning experiences which are responsive to the learning needs of the future and exploit the possibilities offered by ICTs, we can learn much from the existing professional development strategies of people who are already successful in 21st century fields, such as digital media. In this study, I ask: (1) what are the learning challenges faced by digital media professionals in the 21st century? (2) what are the various roles of formal and informal education in their professional learning strategies at present? (3) how do they prefer to acquire needed capabilities? In-depth interviews were undertaken with successful Australian digital media professionals working in micro businesses and SMEs to answer these questions. The strongest thematic grouping that emerged from the interviews related to the need for continual learning and relearning because of the sheer rate of change in the digital media industries. Four dialectical relationships became apparent from the interviewees’ commentaries around the learning imperatives arising out of the immense and continual changes occurring in the digital content industries: (1) currency vs best practice (2) diversification vs specialisation of products and services (3) creative outputs vs commercial outcomes (4) more learning opportunities vs less opportunity to learn. These findings point to the importance of ‘learning how to learn’ as a 21st century capability. The interviewees were ambivalent about university courses as preparation for professional life in their fields. Higher education was described by several interviewees as having relatively little value-add beyond what one described as “really expensive credentialling services.” For all interviewees in this study, informal learning strategies were the preferred methods of acquiring the majority of knowledge and skills, both for ongoing and initial professional development. Informal learning has no ‘curriculum’ per se, and tends to be opportunistic, unstructured, pedagogically agile and far more self-directed than formal learning (Eraut, 2004). In an industry impacted by constant change, informal learning is clearly both essential and ubiquitous. Inspired by the professional development strategies of the digital media professionals in this study, I propose a 21st century model of the university as a broad, open learning ecology, which also includes industry, professionals, users, and university researchers. If created and managed appropriately, the university learning network becomes the conduit and knowledge integrator for the latest research and industry trends, which students and professionals alike can access as needed.

12 principles for the effective supervision of creative practice higher research degrees, dispatches from the field

In late 2012 and early 2013 we interviewed 25 experienced and early career supervisors of creative practice higher research degrees. This journey spanned five universities and a broad range of disciplines including visual art, music, performing art, new media, creative writing, fashion, graphic design, interaction design and interior design. Some of the supervisors we interviewed were amongst the first to complete and supervise practice-led and practice-based PhDs; some have advocated for and defined this emergent field; and some belong to the next generation of supervisors who have confidently embarked on this exciting and challenging path. Their reflections have brought to light many insights gained over the past decade. Here we have drawn together common themes into a collection of principles and best practice examples. We present them as advice rather than rules, as one thing that the supervisors were unanimous about is the need to avoid proscriptive models and frameworks, and to foster creativity and innovation in what is still an emergent field of postgraduate supervision. It is with thanks to all of the supervisors who contributed to these conversations, and their generosity in sharing their practices, that we present their advice, exemplars and case studies.

E-motion diaries : blogging emotional responses to online learning in higher education

Emotions are inherently social, and are central to learning, online interaction and literacy practices (Shen, Wang, & Shen, 2009). Demonstrating the dynamic sociality of literacy practice, we used e-motion diaries or web logs to explore the emotional states of pre-service high school teachers’ experiences of online learning activities. This is because the methods of communication used by university educators in online learning and writing environments play an important role in fulfilling students’ need for social interaction and inclusion (McInnerney & Roberts, 2004). Feelings of isolation and frustration are common emotions experienced by students in many online learning environments, and are associated with the success or failure of online interactions and learning (Su, et al., 2005). The purpose of the study was to answer the research question: What are the trajectories of pre-service teachers’ emotional states during online learning experiences? This is important because emotions are central to learning, and the current trend toward Massive Open Online Courses (MOOCs) needs research about students’ emotional connections in online learning environments (Kop, 2011). The project was conducted with a graduate class of 64 high school science pre-service teachers in Science Education Curriculum Studies in a large Australian university, including males and females from a variety of cultural backgrounds, aged 22-55 years. Online activities involved the students watching a series of streamed live lectures for the first 5 weeks providing a varied set of learning experiences, such as viewing science demonstrations (e.g., modeling the use of discrepant events). Each week, students provided feedback on learning by writing and posting an e-motion diary or web log about their emotional response. Students answered the question: What emotions did you experience during this learning experience? The descriptive data set included 284 online posts, with students contributing multiple entries. Linguistic appraisal theory, following Martin and White (2005), was used to regroup the 22 different discrete emotions reported by students into the six main affect groups – three positive and three negative: unhappiness/happiness, insecurity/security, and dissatisfaction/satisfaction. The findings demonstrated that the pre-service teachers’ emotional responses to the streamed lectures tended towards happiness, security, and satisfaction within the typology of affect groups – un/happiness, in/security, and dis/satisfaction. Fewer students reported that the streamed lectures triggered negative feelings of frustration, powerlessness, and inadequacy, and when this occurred, it often pertained to expectations of themselves in the forthcoming field experience in classrooms. Exceptions to this pattern of responses occurred in relation to the fifth streamed lecture presented in a non-interactive slideshow format that compressed a large amount of content. Many students responded to the content of the lecture rather than providing their emotional responses to this lecture, and one student felt “completely disengaged”. The social practice of online writing as blogs enabled the students to articulate their emotions. The findings primarily contribute new understanding about students' wide range of differing emotional states, both positive and negative, experienced in response to streamed live lectures and other learning activities in higher education external coursework. The is important because the majority of previous studies have focused on particular negative emotions, such as anxiety in test taking. The research also highlights the potentials of appraisal theory for studying human emotions in online learning and writing.

Shaping lecturers’ beliefs about teaching and learning in higher education in the Pacific

The aim of this study was to describe the educational experiences shaping the teaching and learning beliefs held by a group of beginning lecturers in higher education at various tertiary institutions in the Pacific Island Countries (PICs). A total of sixty three essays written by participants in an online course on teaching in higher education comprised the data for the study. A modified version of narrative analysis was used. This is a powerful methodology in qualitative research that can provide remarkable insights into individuals’ beliefs. The critical experiences that were thought to shape their beliefs in teaching and learning were identified and discussed in the light of relevant literature. The participants described a range of influences that shaped their beliefs about teaching and learning including realisation about the need to work harder and know more, the importance of independence, support systems, curriculum, qualities of a teacher, teaching and learning process, teaching and learning strategies, and learning environments. This information was useful in teaching these students and for further courses.

‘I now know I can do this now’: Indigenous women and writing in the Australian higher education sector

This paper explores the challenges of writing and publishing faced by Indigenous women who work in the Australian higher education sector. It demonstrates that Indigenous women are under-represented in the academy and argues that Indigenous styles of writing are typically not valued for broader publication. The authors describe a writing mentoring and support program specifically developed for Indigenous academic women in Australia. The Tiddas Writin’ Up Workshop provided a safe and culturally-appropriate space for women to learn about academic writing and develop their writing skills. The workshop led to the publication of a special issue of the Journal of Australian Indigenous Issues – known as the Tiddas Collection. The authors highlight the power and strength of well-developed support programs to address skills development, confidence, inequities and under-representation of Indigenous women within the higher education workforce.

Hunting and gathering : new imperatives in mapping and collecting student learning data to assure quality outcomes

Assurance of learning (AOL) is a quality enhancement and quality assurance process used in higher education. It involves a process of determining programme learning outcomes and standards, and systematically gathering evidence to measure students' performance on these. The systematic assessment of whole-of-programme outcomes provides a basis for curriculum development and management, continuous improvement, and accreditation. To better understand how AOL processes operate, a national study of university practices across one discipline area, business and management, was undertaken. To solicit data on AOL practice, interviews were undertaken with a sample of business school representatives (n = 25). Two key processes emerged: (1) mapping of graduate attributes and (2) collection of assurance data. External drivers such as professional accreditation and government legislation were the primary reasons for undertaking AOL outcomes but intrinsic motivators in relation to continuous improvement were also evident. The facilitation of academic commitment was achieved through an embedded approach to AOL by the majority of universities in the study. A sustainable and inclusive process of AOL was seen to support wider stakeholder engagement in the development of higher education learning outcomes.

Benchmarking potential factors leading to education quality : a study of Cambodian higher education

Purpose To study the quality in higher education in Cambodia and explore the potential factors leading to quality in Cambodian higher education. Design/methodology/approach Five main factors that were deemed relevant in providing quality in Cambodian higher education were proposed: academic curriculum and extra-curricular activities, teachers' qualification and methods, funding and tuition, school facilities, and interactive network. These five propositions were used to compare Shu-Te University, Taiwan with the top five universities in Cambodia. The data came in the forms of questionnaire and desk research. Descriptive analytical approach is then carried out to describe these five factors. Findings Only 6 per cent of lecturers hold PhD degree and about 85 per cent never published any papers; some private universities charge as low as USD200 per academic year, there is almost no donation from international organizations, and annual government funding on higher education sector nationwide in 2005 was only about USD3.67 million; even though there is a library at each university, books, study materials etc. are not up-to-date and inadequate; 90 per cent of the lecturers never have technical discussion or meeting and about 60 per cent of students felt that their teachers did not have time for them to consult with. Originality/value A useful insight was gained into the perceived importance of quality in higher education that can stimulate debate and discussion on the role of government in building the standard quality in higher education. Also, the findings from this research can assist in the development of a framework of developing human resource.

Student interactivity and teacher participation: An application of legitimate peripheral participation in higher education online learning environments

Lave and Wenger’s legitimate peripheral participation is an important aspect of online learning environments. It is common for teachers to scaffold varying levels of online participation in Web 2.0 contexts, such as online discussion forums and blogs. This study argues that legitimate peripheral participation needs to be redefined in response to students’ decentralised multiple interactions and non-linear engagement in hyperlinked learning environments. The study examines students’ levels of participation in online learning through theories of interactivity, distinguishing between five levels of student participation in the context of a first-year university course delivered via a learning management system. The data collection was implemented through two instruments: i) a questionnaire about students’ interactivity perception in the online reflective learning (n = 238) and then ii) an open discussion on the reason for the diverse perceptions of interactivity (n = 34). The study findings indicate that student participants, other than those who were active, need high levels of teacher or moderator intervention, which better enables legitimate peripheral participation to occur in online learning contexts.

An analysis of digital storytelling as a documentation tool for research in the creative and performing arts

This paper examines the capacity of digital storytelling to document research activity in the creative and performing arts. In particular, it seeks to identify the thought processes and methods that underpin this research and to capture them using the digital storytelling medium. Interest in this issue was prompted by the author’s work with the creative and performing artists from the Queensland Conservatorium and the Queensland College of Art as part of the Federal government’s Research Quality Framework (RQF) in 2007. The RQF compelled artists to address what it means to undertake research in their disciplines, to describe this, measure it and quantify it; for many practitioners this represents a significant challenge. These issues continue to be pertinent in the context of the Excellence in Research for Australia (ERA) initiative. This research is significant because it seeks to identify, in layman’s terms, the research methods and thought processes used by artists in their research practice. It seeks to do so free of the encumbrances of the professional doctorate policies, the higher education research quality frameworks, and the dense philosophical debates that have to-date dominated discussions of this issue. The research involves qualitative data collection methods including a detailed literature review, interviews with key practitioners and academics involved in the creative and performing arts, and three case studies. The literature review focuses on publications that explore issues of research practice and method in the creative and performing arts. The case studies involve three Queensland-based artists. Digital stories will be developed (and presented) with Marcus and Mafe using their visual materials and drawing on the issues identified in the literature review and interviews. Emmerson’s DVD provided a point of comparison with the digital stories. (Brief bios are attached)

Creative labour and graduate outcomes : Implications for higher education and cultural policy

One of the principal ways that cultural and higher education policy and practice intersect is over a shared concern with the supply of talent and its employability and career sustainability. This article considers the multidisciplinary contributions to these debates, and then engages with these debates by drawing upon research from analyses of national Census data, and via granular empirical survey research into Australian creative arts graduates’ initial career trajectories. In so doing, it seeks to paint a more nuanced picture of graduate outcomes, the significance of creative skills and by extension creative education and training, and the various kinds of value that creative graduates add through their work. This evidence should assist in a closer affinity between the differing approaches to creative labour and the creative economy, and has implications for cultural and higher education policy.