The practice of quality assuring learning in higher education

There remains a lack of published empirical data on the substantive outcomes of higher learning and the establishment of quality processes for determining them. Studies that do exist are nationally focused with available rankings of institutions reflecting neither the quality of teaching and learning nor the diversity of institutions. This paper describes two studies in which Associate Deans from Australian higher education institutions and focus groups of management and academics identify current issues and practices in the design, development and implementation of processes for assuring the quality of learning and teaching. Results indicate that developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program necessitates knowledge creation and new inclusive processes. Common elements supporting consistently superior outcomes included: inclusivity; embedded graduate attributes; consistent and appropriate assessment; digital collection mechanisms; and systematic analysis of outcomes used in program review. Quality measures for assuring learning are proliferating nationally and changing the processes, systems and culture of higher education as a result.

A stakeholder analysis of one institution’s assuring learning experience – is the promise and the practice worth the price?

Universities around the world are rushing to implement assurance of learning policies and practices with varying degrees of success. One School investigated its own policy and practice development through the eyes of its key stakeholders to identify whether the practice was worth the price. Findings indicate that although the key stakeholders considered different needs and viewed their experiences differently, value did abound and was in the eye of the beholder.

Hunters & gatherers : strategies for curriculum mapping and data collection for assurance of learning

Assurance of learning is a predominant feature in both quality enhancement and assurance in higher education. Assurance of learning is a process that articulates explicit program outcomes and standards, and systematically gathers evidence to determine the extent to which performance matches expectations. Benefits accrue to the institution through the systematic assessment of whole of program goals. Data may be used for continuous improvement, program development, and to inform external accreditation and evaluation bodies. Recent developments, including the introduction of the Tertiary Education and Quality Standards Agency (TEQSA) will require universities to review the methods they use to assure learning outcomes. This project investigates two critical elements of assurance of learning: 1. the mapping of graduate attributes throughout a program; and 2. the collection of assurance of learning data. An audit was conducted with 25 of the 39 Business Schools in Australian universities to identify current methods of mapping graduate attributes and for collecting assurance of learning data across degree programs, as well as a review of the key challenges faced in these areas. Our findings indicate that external drivers like professional body accreditation (for example: Association to Advance Collegiate Schools of Business (AACSB)) and TEQSA are important motivators for assuring learning, and those who were undertaking AACSB accreditation had more robust assurance of learning systems in place. It was reassuring to see that the majority of institutions (96%) had adopted an embedding approach to assuring learning rather than opting for independent standardised testing. The main challenges that were evident were the development of sustainable processes that were not considered a burden to academic staff, and obtainment of academic buy in to the benefits of assuring learning per se rather than assurance of learning being seen as a tick box exercise. This cultural change is the real challenge in assurance of learning practice.

Assurance of learning – are academics and senior management singing from the same hymn sheet?

Assurance of learning (AoL) is an important process in quality education, designed to measure the accomplishment of educational aims at the core of an institution’s programs, whilst encouraging faculty to continuously develop and improve the programs and courses. This paper reports on a study of Australian business schools to investigate current AoL practices through semi structured interviews with senior faculty leaders followed by focus group interviews with groups of senior program leaders and groups of academic teaching staff. Initial findings indicate there are significant challenges in encouraging academic staff to commit to the process and recognise the benefits of assuring learning. The differences in understanding between the various leaders and the academics were highlighted through the different focus groups. Leaders’ stressed strategic issues such as staff engagement and change, while academics focussed on process issues such as teaching graduate attributes and external accreditation. Understanding the differences in the perspectives of leaders and faculty is important, as without a shared understanding between the two groups, there is likely to be limited engagement, which creates difficulties in developing effective assurance of learning processes. Findings indicate that successful strategies developed to foster shared values on assurance of learning include: strong senior leaders’ commitment; developing champions among program and unit level staff; providing professional development opportunities; promoting and celebrating success and effectiveness; and ensuring an inclusive process with academics of all levels collaborating in the development and implementation of the process.

Building sustainable and effective assurance of learning process in a changing higher education environment

In a study of assuring learning in Australian Business Schools, 25 Teaching and Learning Associate Deans were interviewed to identify current issues in developing and measuring the quality of teaching and learning outcomes. Results indicate that for most institutions developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program required knowledge creation and the building of new inclusive processes. Common elements of effective practice, namely those which offered consistently superior outcomes, included: inclusive processes; embedded graduate attributes throughout a program; alongside consistent and appropriate assessment. Results indicate that assurance of learning processes are proliferating nationally while quality of teaching and learning outcomes and in the processes for assuring it is increasing as a result.

From mountaintop to corporate ladder: What new professionals really really want in a capstone experience!

Capstone subjects are increasingly used in Universities worldwide to complete the undergraduate program experience and to transition graduates into the workplace. As such, capstones fulfil a large role consolidating one experience and traversing the gap to another. Yet, little is known or understood about their design, their implementation or evaluation. In this study we investigate the final-year experience from the student's perspective. We surveyed graduates from five Business Schools in Australia to identify perceptions of their final-year experience. Findings indicate that the transition experience of the student to professional is unique. In their liminal or intermediate state between student and professional they recognise the value of process skills, in particular the development of generic business skills related to application and the importance of opportunities for experiencing the application of theory in practice. The findings add a new understanding to the current literature which has not previously acknowledged the insight of the transitioning professional.

Strategies to engage academics in assuring graduate attributes

Assurance of learning (AoL) is a predominant feature in both quality assurance and quality enhancement in higher education. The process may be used for program development, and to inform external accreditation and evaluation bodies. However, there is an obvious challenge in trying to get academic staff to buy into the benefits of the AoL process. This project conducted an audit across 25 Australian Business Schools. The majority of those interviewed stated that academic staff considered AoL to be extra work and viewed the process as a box ticking exercise for external bodies rather than sound educational practice. A change management process is required to promote the necessary cultural change to embed AoL into practice. This paper showcases some of the educational leadership strategies that have been successfully implemented across Australia to foster staff engagement in the AoL process. These include: strong senior management commitment and leadership demonstrating a constant and high level drive for staff engagement until AoL becomes an institutional norm; developing leadership and champions among unit and program level staff, to share practices and promote the benefits that come from engaging in the process; providing professional development opportunities to discuss and resolve difficulties and tensions around AoL; demonstrating success and effectiveness by selling staff on the evidence that AoL makes a difference; and making the process inclusive with academics collaborating in the development and implementation of the process.

Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.