The maturity model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention : new horizon or false dawn?

While the engagement, success and retention of first year students are ongoing issues in higher education, they are currently of considerable and increasing importance as the pressures on teaching and learning from the new standards framework and performance funding intensifies. This Nuts & Bolts presentation introduces the concept of a maturity model and its application to the assessment of the capability of higher education institutions to address student engagement, success and retention. Participants will be provided with (a) a concise description of the concept and features of a maturity model; and (b) the opportunity to explore the potential application of maturity models (i) to the management of student engagement and retention programs and strategies within an institution and (ii) to the improvement of these features by benchmarking across the sector.

Moving beyond transition pedagogy: Maturity models and student engagement

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). This paper proposes that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities. We will describe the Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012) we are currently investigating. We will discuss if our research may address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. The first aim of our research is to extend the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). The second aim of this research is to move beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Verifying a Student Engagement, Success and Retention Maturity Model (SESR-MM): A reality check of the model using data derived from three universities

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). We have proposed that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities and we are currently articulating a Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012). Our research aims to address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. Our research extends the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). Our research also moves beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. The issues explored in this workshop are (i) whether the maturity model concept can be usefully applied to provide a measure of institutional capability for SESR; (ii) whether the SESR-MM can be used to assess the maturity of a particular set of institutional practices; and (iii) whether a collective assessment of an institution’s SESR capabilities can provide an indication of the maturity of the institution’s SESR activities. The workshop will be approached in three stages. Firstly, participants will be introduced to the key characteristics of maturity models, followed by a discussion of the SESR-MM and the processes involved in its development. Secondly, participants will be provided with resources to facilitate the development of a maturity model and an assessment instrument for a range of institutional processes and related practices. In the final stage of the workshop, participants will “assess” the capability of these practices to provide a collective assessment of the maturity of these processes. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Assessing performance and capability in the work place : focusing on essential skills and knowledge in student work placements

The determination of performance standards and assessment practices in regard to student work placements is an essential and important task. Inappropriate, inadequate, or excessively complex assessment tasks can influence levels of student engagement and the quality of learning outcomes. Critical to determining appropriate standards and assessment tasks is an understanding and knowledge of key elements of the learning environment and the extent to which opportunities are provided for students to engage in critical reflection and judgement of their own performance in the contexts of the work environment. This paper focuses on the development of essential skills and knowledge (capabilities) that provide evidence of learning in work placements by describing an approach taken in the science and technology disciplines. Assessment matrices are presented to illustrate a method of assessment for use within the context of the learning environment centred on work placements in science and technology. This study contributes to the debate on the meaning of professional capability, performance standards and assessment practices in work placement programs by providing evidence of an approach that can be adapted by other programs to achieve similar benefits. The approach may also be valuable to other learning contexts where capability and performance are being judged in situations that are outside a controlled teaching and learning environment i.e. in other life-wide learning contexts.

Skill development and social inclusion via peer assisted learning in first-year higher education : an evaluation of student experiences across two law units

Because professions seek graduates who can 'collaborate, share skills and knowledge, and communicate' (Kruck and Reif, 2001, p 37), it is important that university graduates are not equipped solely with the content knowledge of their discipline, but also with prospective employment skills. Furthermore, when students 'interact more in positive ways with their teachers and peers, they gain more in terms of essential skills and competencies, such as critical thinking, problem~solving [and] effective communication' (NSSE, 2000, p 2)./n this way, peer assisted fellowing has the potential to enhance students' professional development, and provide the social inclusion and engagement necessary for effective learning. This session describes two peer assisted learning models embedded within first year QUT Faculty of Law units. Through a partnership between teaching staff, student mentors and mentees, the models aim to facilitate student socialisation whilst supplementing understanding of substantive law with the development of academic and work·related skills. Mentor and mentee perceptions, and program implications, are considered.

Changing relationships : Re-activating student and staff engagement in the design studio learning environment

QUT Teaching and Learning Support Services 'Revisiting University Teaching’program for mid-career academics. 'Innovations in Teaching at QUT' presentations. Presentations were part of a 2 day program that provides opportunities for experienced academic staff with responsibilities for teaching to review their current teaching practices and explore innovations in teaching that will assist them to enhance student learning and develop their own scholarship of teaching. The presenter responded to the following: 1.What is the innovation you have incorporated into your teaching? - give a brief overview/ description/ demonstration of the innovation 2.What challenges/issues prompted you to make changes in your approach? Were they discipline specific? Operational? Opportunistic? 3.What factors did you need to consider in implementing these changes? Which factors enabled success or hindered? 4.What has this innovation achieved so far? How have learners responded? How have the broader teaching team and academic staff from other units in your course responded? 5.How could this innovation be used by other academics in their teaching? What do you see as the possibilities for further expansion of this innovation? (NB. This question could be answered as part of a final sharing of group discussion). Presenter: Shannon Satherley

Supporting transition to law school and student well-being : the role of professional legal identity

The empirically established decline in law student well being during the first year of law school is a red-flagged imprimatur for first year curriculum change. This article suggests that by engaging law students with the concept of a positive professional identity, student engagement and intrinsic motivation will increase because they are working towards a career goal that has meaning and purpose. Law school is a time of professional transformation and the legal academy can take steps to ensure that this transformation is inculcated with positive messages. Literature from the fields of law and psychology is analysed in this article, to explain how a positive conception of the legal profession (and a student’s future role within it) can increase a student’s psychological well-being – at law school and beyond.

Valuing student and community voices in the university : action research as a framework for community service-learning

The Community Service-learning Lab (the Lab) was initiated as a university-wide service-learning experience at an Australian university. The Lab engages students, academics, and key community organisations in interdisciplinary action research projects to support student learning and to explore complex and ongoing problems nominated by the community partners. The current study uses feedback from the first offering of the Lab and focuses on exploring student experiences of the service learning project using an action research framework. Student reflections on this experience have revealed some positive outcomes of the Lab such as an appreciation for positive and strengths-based change. These outcomes are corroborated by collected reflections from community partners and academics. The students also identified challenges balancing the requirements for assessment and their goals to serve the community partner’s needs. This feedback has provided vital information for the academic team, highlighting the difficulties in balancing the agenda of the academic framework and the desire to give students authentic experiences.

Transition to law school and law student well-being : The importance of professional legal identity

The empirically established decline in law student well-being during the first year of law school is a red-flagged imprimatur for first year curriculum change. This article suggests that by engaging law students with the concept of a positive professional identity, student engagement and intrinsic motivation will increase because they are working towards a career goal that has meaning and purpose. Law school is a time of professional transformation and the legal academy can take steps to ensure that this transformation is inculcated with positive messages. Literature from the fields of law and psychology is analysed in this article, to explain how a positive conception of the legal profession (and a student’s future role within it) can increase a student’s psychological well-being – at law school and beyond.

Student self-assessment of performance : a useful learning tool for students and instructors?

Assurance of Learning is a focus for many Business Schools to meet AACSB requirements. Prior research has examined student self-assessment as a measure of learning in this context, but we propose instead that this is a useful ‘learning tool’ for students and instructors that can help improve student performance and curricula. We examined self-assessment in a third year undergraduate e-marketing course and found a positive, though weak relationship between student self-assessment and instructor ratings on intended learning goals and criteria for a written assignment. Students who self-assessed tended to perform better in the assessment item and course than those who did not self-assess. But less competent students were more inaccurate in their self-assessment, suggesting a need to enhance understanding of marking criteria and performance standards within this group. Implications for educators are discussed.

Development and implementation of the Australian universities radiation therapy student clinical assessment form

Purpose: Prior to 2009, one of the problems faced by radiation therapists who supervised and assessed students on placement in Australian clinical centres, was that each of the six Australian universities where Radiation Therapy (RT) programmes were conducted used different clinical assessment and reporting criteria. This paper describes the development of a unified national clinical assessment and reporting form that was implemented nationally by all six universities in 2009. Methods: A four phase methodology was used to develop the new assessment form and user guide. Phase 1 included university consensus around domains of student practice and assessment, and alignment with national competency standards; Phase 2 was a national consensus workshop attended by radiation therapists involved in student supervision and assessment; Phase 3 was an action research re-iterative Delphi technique involving two rounds of a mail-out to gain further expert consensus; and stage 4 was national piloting of the developed assessment form. Results: The new assessment form includes five main domains of practice and 19 sub-domain criteria which students are assessed against during placement. Feedback from the pilot centre participants was positive, with the new form being assessed to be comprehensive and complemented by the accompanying user guide. Conclusion: The new assessment form has improved both the formative and summative assessment of students on placement, as well as enhancing the quality of feedback to students and the universities. The new national form has high acceptance from the Australian universities and has been subject to wide review by the profession.

The ACEN Student Scholarship: a profile of financial hardship and Work Integrated Learning

An increasing emphasis on embedding Work-Integrated Learning (WIL) in the curriculum has impacted on teaching and learning approaches in Australian higher education institutions (Higher Education Base Funding Review: Final Report, 2011). Yet whilst the benefits and costs of these approaches have been identified (Bradley, Noonan, Nugent, & Scales, 2008; Patrick et al., 2009) insufficient attention has been paid to financial costs experienced by students studying subjects with a Work Integrated Learning component. In 2010 the Australian Collaborative Education Network (ACEN) responded to this issue by offering three modest student scholarships based on evidence of hardship. Data collected from over 1000 applicants between 2010 and 2012 indicate travel, accommodation, food, clothing, equipment and loss of income are of major concern especially for students on lengthy placements involving relocation. At the same time the Australian Federal Government’s review of base funding has recommended a detailed assessment of the costs of providing student placements across all disciplines - in particular health and education (DEEWR, 2011, p.94). This paper considers costs from the student perspective and highlights major concerns identified through ACEN scholarship applications over a three year period. The implications for ACEN are described and recommendations documented which outline ACEN’s role in ensuring that these issues are given greater consideration across the sector.

Establishing a framework for transforming student engagement, success and retention in higher education institutions [Final Report]

The perennial issues of student engagement, success and retention (SESR) in higher education continue to attract attention as key indicators of learning and teaching quality. This project aimed to establish and provide a holistic framework that would allow higher education institutions (HEIs) manage and improve their student engagement and retention strategies and programs. The framework and main project deliverable is a Maturity Model (MM) for Student Engagement, Success and Retention (SESR-MM). The project involved three Australian universities with experience and reputations in SESR activities: Queensland University of Technology (lead institution), the University of Queensland and Griffith University, working cooperatively to develop and trial the project deliverables. Project findings suggest that the SESR-MM has the potential to positively transform the holistic—academic, social and personal—engagement experiences of students in Australian universities, and that the SESR-MM is a useful mechanism for sharing good practice and improving programs designed to enhance the student experience.

Teacher and student readiness to use Facebook for learning in an adult vocational educational setting

The factors influencing both teacher and student readiness to use Facebook as part of their teaching and learning in a vocational educational institution were studied through a qualitative case study. Data included teacher and student questionnaire and focus group interviews. While it was found that the students demonstrated readiness and willingness to incorporate Facebook into their current learning, the teachers were more reluctant. Different perceptions around control of learning, time, and concerns around compartmentalisation of learning and social lives would need to be addressed before Facebook could be used as a formal learner engagement strategy.

Establishing a framework for transforming student engagement, success and retention in higher education institutions

This poster summarises the outcomes of a national project to develop and provide a holistic framework consisting of a series of sequential and increasingly sophisticated stages that will allow higher education institutions (HEIs) to manage and improve their student engagement and retention strategies/programs.