Tracking new directions in the evaluation of postgraduate supervision

This article focuses on the evaluation of thesis supervision and highlights the vast range of problem areas presently documented as significant areas of concern for both graduate students and their supervisors. Additionally, the authors report on a study completed in 1995, which surveyed all Australian universities about current evaluative practices in postgraduate supervision. The conclusion of this study was that the conduct of such evaluations appears to be minimal and is primarily designed to obtain an indicator of the general “health” of a university's postgraduate supervision rather than to foster improved supervisory practices. As part of the same study, the authors conducted University faculty-based student and supervisor focus groups. Key issues emerging from these focus groups were: (1) the importance of relational aspects of supervision as the student communicates over the long term with one or more supervisors; (2) the importance of systematic feedback, monitoring, and evaluation to the supervisory process; and (3) the lack of strategies to facilitate this evaluative feedback process. On the basis of these findings, the authors designed evaluative strategies to facilitate regular ongoing feedback between students and supervisors.

Narrative on a doctoral narrative : reflections on postgraduate study and pedagogy

This paper explores the stages of one student’s intellectual journey through a Doctor of Philosophy program of study in an Australian university. It outlines the theoretical and methodological insights made as she came to understand that data was discourse, entailing a politics and position of power that ran contrary to the aims of the study that the student was undertaking in and on her own community. The article is a reflective narrative produced from the experience of having to come to terms with some of the personal and professional tensions and contradictions that postgraduate study can, and maybe should, engender if it is to be any of real value.

Building distributed leadership for effective supervision of creative practice higher research degrees : final report 2014

Building distributed leadership for effective supervision of creative practice higher research degrees is an Office for Learning and Teaching (OLT) funded project, conducted in partnership between Queensland University of Technology, The University of Melbourne, Auckland University of Technology, University of New South Wales and University of Western Sydney. The project was initiated to develop a cooperative approach to establishing an understanding of the contextual frameworks of the emergent field of creative practice higher degrees by research (HDRs); capturing early insights of administrators and supervisors; gathering exemplars of good practices; and establishing an in-common understanding of effective approaches to supervision. To this end, the project has produced: • A literature review, to provide a research foundation for creative practice higher research degree supervision (Chapter 3). • A contextual review of disciplinary frameworks for HDR programs, produced through surveys of postgraduate research administrators (Section 4.1), and an analysis of institutional materials and academic development programs for supervisors (Section 4.2). • A National Symposium, Effective Supervision of Creative Arts Research Degrees (ESCARD), at QUT in Brisbane in February 2013, with 62 delegates from 20 Australasian Universities, at which project findings were disseminated, and delegates presented case studies and position papers, and participated in discussions on key issues for supervisors (Appendix 1). • Resources, including a booklet for supervisors: 12 Principles for the Effective Supervision of Creative Practice Higher Research Degrees, which encapsulates attitudes, insights and good practices of experienced and new supervisors. It was produced through a content analysis of interviews with twenty-five supervisors in creative disciplines (visual and performing arts, music, new media, creative writing and design) (Printed booklet, PDF, Appendix 3). • A project website to disseminate project outcomes supervisioncreativeartsphd.net>, which holds project findings, relevant references, and a repository of case studies and position papers by supervisors and program administrators. • A call for papers for a special issue ‘Supervising Practice: Perspectives on the Supervision of Creative Practice Research Higher Degrees’ of ACCESS Journal: Critical Perspectives on Communication, Cultural & Policy Studies (ERA ranked A quality) in 2014 (Appendix 2). • A community of supervisory practice initiated through project partnerships, a national symposium where supervisors from across Australasia met in dialogue for the first time, resource sharing, and joint publishing opportunities. • A set of recommendations for supervision capacity building and academic development, produced through the triangulation of literature and contextual reviews, analysis of institutional frameworks, interviews with supervisors and national dialogues. It is anticipated that the project’s outcomes will support experienced and new supervisors in this emergent field, and so benefit HDR students, and will enable creative disciplines to build supervision capacity, and so to accommodate growth in postgraduate enrolments. Funded as a pilot project, the project set out to establish a robust research base to provide a foundation for future work involving sharing good practices, resource building, and designing effective approaches to academic development for supervisors. Recommendations that were produced out of this project include the need to extend beyond generic, formal training for supervisors to academic development that harnesses and extends distributed leadership; focuses on local, disciplinary contexts; has a strong emphasis on case studies; provides diverse resources; and facilitates dialogue between supervisors. Recommendations also include developing frameworks for mentoring new supervisors and building a national network to facilitate cross-institutional discourse, disseminate good practices, and share insights into the management of risk factors, ethical issues, and preparing candidates for examination. As a pilot investigation, the outcomes of this project lay the ground for this future work.

12 principles for the effective supervision of creative practice higher research degrees, dispatches from the field

In late 2012 and early 2013 we interviewed 25 experienced and early career supervisors of creative practice higher research degrees. This journey spanned five universities and a broad range of disciplines including visual art, music, performing art, new media, creative writing, fashion, graphic design, interaction design and interior design. Some of the supervisors we interviewed were amongst the first to complete and supervise practice-led and practice-based PhDs; some have advocated for and defined this emergent field; and some belong to the next generation of supervisors who have confidently embarked on this exciting and challenging path. Their reflections have brought to light many insights gained over the past decade. Here we have drawn together common themes into a collection of principles and best practice examples. We present them as advice rather than rules, as one thing that the supervisors were unanimous about is the need to avoid proscriptive models and frameworks, and to foster creativity and innovation in what is still an emergent field of postgraduate supervision. It is with thanks to all of the supervisors who contributed to these conversations, and their generosity in sharing their practices, that we present their advice, exemplars and case studies.

Supervising practice: Perspectives on the supervision of creative practice higher degrees by research

PhD supervision is a particularly complex form of pedagogical practice, and nowhere is its complexity more apparent than in new and emergent fields, such as creative practice Higher Degrees by Research (HDRs) where supervisors face the challenges of a unique, uncharted area of research training. While there is an increasing body of literature on postgraduate supervision, and another emerging body of research into what creative practice/practice-led/practice-based research is, so far little attention has been paid to matters associated with research education leadership and pedagogical aspects of supervision in creative practice disciplines.For this reason, this special issue brings together a range of perspectives on the supervision of creative practice PhDs in visual and performing arts, media production, creative writing, and design.

Speaking of supervision: A dialogic approach to building higher degree research supervision capacity in the creative arts

In the emergent field of creative practice higher degrees by research, first generation supervisors have developed new models of supervision for an unprecedented form of research that combines creative practice and written thesis. In a national research project, entitled 'Effective supervision of creative practice higher research degrees', we set out to capture and share early supervisors' insights, strategies and approaches to supporting their creative practice PhD students. From the insights we gained during the early interview process, we expanded our research methods in line with a distributed leadership model and developed a dialogic framework. This led us to unanticipated conclusions and unexpected recommendations. In this study, we primarily draw on philosopher and literary theorist Mikhail Bakhtin's dialogics to explain how giving precedence to the voices of supervisors not only facilitated the articulation of dispersed tacit knowledge, but also led to other 20 discoveries. These include the nature of supervisors' resistance to prescribed models, policies and central academic development programmes; the importance of polyvocality and responsive dialogue in enabling continued innovation in the field; the benefits to supervisors of reflecting, discussing and sharing practices with colleagues; and the value of distributed leadership and dialogue to academic development and supervision capacity building in research education.

Choreographing research : supervising the dancing thesis

The dancing doctorate is an interrogative endeavour which can but nurture the art form and forge a beneficial dynamism between those who seek and those who assess the emerging knowledges of dance’. (Vincs, 2009) From 2006-2008 three dance academics from Perth, Brisbane and Melbourne undertook a research project entitled Dancing between Diversity and Consistency: Refining Assessment in Postgraduate Degrees in Dance, funded by the ALTC Priority Projects Program. Although assessment rather than supervision was the primary focus of this research, interviews with 40 examiner/supervisors, 7 research deans and 32 candidates across Australia and across the creative arts, primarily in dance, provide an insight into what might be considered best practice in preparing students for higher research degrees, and the challenges that embodied and experiential knowledges present for supervision. The study also gained the industry perspectives of dance professionals in a series of national forums in 5 cities, based around the value of higher degrees in dance. The qualitative data gathered from these two primary sources was coded and analysed using the NVivo system. Further perspectives were drawn from international consultant and dance researcher Susan Melrose, as well as recent publications in the field. Dance is a young addition to academia and consequently there tends to be a close liaison between the academy and the industry, with a relational fluidity that is both beneficial and problematic. This partially explains why dance research higher degrees are predominantly practice-led (or multi-modal, referring to those theses where practice comprises the substantial examinable component). As a physical, embodied art form, dance engages with the contested territory of legitimising alternative forms of knowledge that do not sit comfortably with accepted norms of research. In supporting research students engaged with dance practice, supervisors traverse the tricky terrain of balancing university academic requirements with studies that are emergent, not only in the practice and attendant theory but in their methodologies and open-ended outcomes; and in an art form in which originality and new knowledge also arises from collaborative creative processes. Formal supervisor accreditation through training is now mandatory in most Australian universities, but it tends to be generic and not address supervisory specificity. This paper offers the kind of alternative proposed by Edwards (2002) that improving postgraduate supervision will be effective if supervisors are empowered to generate their own standards and share best practice; in this case, in ways appropriate to the needs of their discipline and alternative modes of thesis presentation. In order to frame the qualities and processes conducive to this goal, this paper will draw on both the experiences of interviewees and on philosophical premises which underpin the research findings of our study. These include the ongoing challenge of dissolving the binary oppositions of theory and practice, especially in creative arts practice where theory resides in and emerges from the doing as much as in articulating reflection about the doing through what Melrose (2003) terms ‘mixed mode disciplinary practices’. In guiding practitioners through research higher degrees, how do supervisors deal with not only different forms of knowledge but indeed differing modes of knowledge? How can they navigate tensions that occur between the ‘incompatible competencies’ (Candlin, 2000) of the ‘spectating’ academic experts with their ‘irrepressible drive ... to inscribe, interpret, and hence to practise temporal closure’, and practitioner experts who create emergent works of ‘residual unfinishedness’ (Melrose 2006) which are not only embodied but ephemeral, as in the case of live performance?

The effective and the evocative : reflecting on practice-led research approaches in art and design

Early in the practice-led research debate, Steven Scrivener (2000, 2002) identified some general differences in the approach of artists and designers undertaking postgraduate research. His distinctions centered on the role of the artefact in problem-based research (associated with design) and creative-production research (associated with artistic practice). Nonetheless, in broader discussions on practice-led research, 'art and design' often continues to be conflated within a single term. In particular, marked differences between art and design methodologies, theoretical framing, research goals and research claims have been underestimated. This paper revisits Scrivener's work and establishes further distinctions between art and design research. It is informed by our own experiences of postgraduate supervision and research methods training, and an empirical study of over sixty postgraduate, practice-led projects completed at the Creative Industries Faculty of QUT between 2002 and 2008. Our reflections have led us to propose that artists and designers work with differing research goals (the evocative and the effective, respectively), which are played out in the questions asked, the creative process, the role of the artefact and the way new knowledge is evidenced. Of course, research projects will have their own idiosyncrasies but, we argue, marking out the poles at each end of the spectrum of art and design provides useful insights for postgraduate candidates, supervisors and methodologists alike.

Six rules for practice-led research

Recent experience of practice-led postgraduate supervision has prompted me to conclude that the practice-led research method, as it is currently construed, produces good outcomes, especially in permitting practitioners in the creative arts, design and media into the research framework, but at the same time it also generates certain recurring difficulties. What are these difficulties? Practice-led candidates tend to rely on a narrow range of formulations with the result that they assume: (i) the innovative nature of practice-led research; (ii) that its novelty is based in opposition to other research methods; (iii) that practice is intrinsically research, often leading to tautological formulations; (iv) the hyper-self-reflexive nature of practice-led research. This set of guidelines was composed in order to circumvent the shortcomings that result from these recurring formulations. My belief is that, if these shortcomings are avoided, there is nothing to prevent practice-led from further developing as a research inquiry and thus achieving rewarding and successful research outcomes. Originally composed for the purposes of postgraduate supervision, these six rules are presented here in the context of a wider analysis of the emergence of practice-led research and its current conditions of possibility as a research method.

Implications of aligning full registration of doctors with medical school graduation:a qualitative study of stakeholder perspectives

OBJECTIVES: The Shape of Training report recommended that full registration is aligned with medical school graduation. As part of a General Medical Council-funded study about the preparedness for practice of UK medical graduates, we explored UK stakeholders' views about this proposal using qualitative interviews (30 group and 87 individual interviews) and Framework Analysis.

SETTING: Four UK study sites, one in each country.Save

PARTICIPANTS: 185 individuals from eight stakeholder groups: (1) foundation year 1 (F1) doctors (n=34); (2) fully registered trainee doctors (n=33); (3) clinical educators (n=32); (4) undergraduate/postgraduate Deans, and Foundation Programme Directors (n=30); (5) other healthcare professionals (n=13); (6) employers (n=7); (7) policy and government (n=11); (8) patient and public representatives (n=25).

RESULTS: We identified four main themes: (1) The F1 year as a safety net: patients were protected by close trainee supervision and 'sign off' to prevent errors; trainees were provided with a safe environment for learning on the job; (2) Implications for undergraduate medical education: if the proposal was accepted, a 'radical review' of undergraduate curricula would be needed; undergraduate education might need to be longer; (3) Implications for F1 work practice: steps to protect healthcare team integration and ensure that F1 doctors stay within competency limits would be required; (4) Financial, structural and political implications: there would be cost implications for trainees; clarification of responsibilities between undergraduate and postgraduate medical education would be needed. Typically, each theme comprised arguments for and against the proposal.

CONCLUSIONS: A policy change to align the timing of full registration with graduation would require considerable planning and preliminary work. These findings will inform policymakers' decision-making. Regardless of the decision, medical students should take on greater responsibility for patient care as undergraduates, assessment methods in clinical practice and professionalism domains need development, and good practice in postgraduate supervision and support must be shared.

Assessment of the influence of cultural barriers to HDR supervision of non-English speaking background (NESD) students in engineering and information technology (IT) disciplines

The paper details the results of the first phase of an on-going research into the sociocultural factors that influence the supervision of higher degrees research (HDR) engineering students in the Faculty of Built Environment and Engineering (BEE) and Faculty of Science and Technology (FaST) at Queensland University of Technology. A quantitative analysis was performed on the results from an online survey that was administered to 179 engineering students. The study reveals that cultural barriers impact their progression and developing confidence in their research programs. We argue that in order to assist international and non-English speaking background (NESB) research students to triumph over such culturally embedded challenges in engineering research, it is important for supervisors to understand this cohort's unique pedagogical needs and develop intercultural sensitivity in their pedagogical practice in postgraduate research supervision. To facilitate this, the governing body (Office of Research) can play a vital role in not only creating the required support structures but also their uniform implementation across the board.

Research supervision of international students in engineering and information technology disciplines (Resources)

This report provides an overview of the results of a collaborative research project titled "A model for research supervision of international students in engineering and information technology disciplines". This project aimed to identify factors influencing the success of culturally and linguistically diverse (CALD) higher degree research (HDR) students in the fields of Engineering and Information Technology at three Australian Universities: Queensland University of Technology, The University of Western Australia and Curtin University.

A model for research supervision of international students in engineering and information technology disciplines

A large proportion (over 12 per cent) of international and non-English speaking background (NESB) postgraduate research students enrol in engineering and information technology (IT) programs in Australian universities. They find themselves in an advanced research culture, and are technically and scientifically challenged early in their programs. This is in addition to cultural, social and religious isolation and linguistic barriers they have to contend with. The project team surveyed this cohort at QUT and UWA, on the hypothesis that they face challenges that are more discipline-specific. The results of the survey indicate that existing supervisory frameworks which are limited to linguistic contexts are not fully assisting these students and supervisors to achieve high quality research. The goal of this project is to extend these supervisory frameworks to a holistic model that will address the unique needs and supervisory issues these students face in engineering and IT disciplines. The model will be useable by all other Australian universities.

Developing learning cohorts for postgraduate research degrees

Development of researchers through higher degree research studies is a high priority in most universities. Yet, research about supervision as pedagogy and models of supervision is only recently gained increasing attention. Charged with producing good researchers within very limited resources, academics are constantly looking for more efficient models of supervision for higher degree research students. A cohort model of supervision promises several efficiencies, but we argue that its success lies importantly on how well the cohort is developed specifically for higher degree research studies. We drew on a growing body of literature on higher degree research supervision to design, implement and evaluate our approach to developing a cohort of seven students enrolled in the Master of Education (Research) degree. Our approach included four provisions: initial residential workshop, development of a learning community, nourishing scholarship, and ongoing learning opportunities. The four provisions resulted in gradually developing an environment and culture that students found very supportive and nurturing. This paper is based on the findings from data collected from student evaluations in the first year of studies, feedback from the cohort’s sponsor, and our reflective notes. The evaluation substantiated the value in investing time and resources for purposely developing a cohort for higher degree research studies. Whether the cohorts are sponsored or not, universities will still need to invest time and resources for cohort development if a cohort model is intended to gain wider efficiencies in supervision of higher degree research students.

Good Practice Report : Postgraduate Research and Coursework Degrees

Over the past decade, enrolments in postgraduate courses in Australian universities have risen by 34% (or from 258,164 in 2004 to 347,363 in 2013) (uCube, 2014). This substantial growth can be attributed to increased demand for postgraduate coursework as continuing professional education, the expansion of Higher Degrees Research (HDR) intakes, and the development of postgraduate research and coursework degrees in new fields. At the same time, the establishment of the Australian Qualification Framework (AQF) and national Tertiary Education Quality and Standards Agency (TEQSA), as well as the internationalisation of postgraduate education, have brought challenges and opportunities to the sector. During the past five years, the Australian Government Office for Learning and Teaching (OLT) and its predecessor bodies have funded a number of projects and Fellowships on postgraduate coursework and research degrees. They span diverse topics–from entry pathways and research training to supporting international and Indigenous students, examination, scoping studies of new and emergent programs, and effective supervision. In 2014 the OLT commissioned this good practice report to review the grants and fellowships conducted between 2009 and 2014. Encompassing twenty-seven learning and teaching projects and fellowships, the aims of this report include providing universities and academics with an overview of the current state of postgraduate study in Australia and the major influences upon it; a coherent overview of funded projects’ findings and outcomes; and a central point to access good practices, resources and tools in summary form. The objectives of this good practice report are to provide: • A literature review, which contextualises the projects within the Australian and international Higher Education environment, emphasises factors that currently influence postgraduate programs, and highlights challenges and opportunities for the sector. It also explains variations in postgraduate course types and definitions within the Australian Qualification Framework (AQF), and identifies key learning and teaching issues as well as good practices identified in scholarly research and position papers. • A collated overview of the twenty-seven national learning and teaching projects and fellowships on postgraduate coursework and research, including a summary of each project’s aims and objectives, methodologies, outcomes and resources. • A summative index of project characteristics (topics, themes and approaches) and inventory of scholarly research outcomes of the completed projects (publications, reports) as well as resources produced (tools, methods, good practice case studies), and their location (URL Links, references, etc.). • A summary of good practices that have been identified from the literature and the findings of completed projects. • A set of recommendations to address remaining gaps in the field and areas in which further work or development are appropriate. Bringing this work together will help enable university course teams to improve the delivery and development of existing postgraduate courses and to develop new ones, and it will provide academics with an overview of good practices and resources for teaching, supervising and supporting postgraduate students.