Verifying a Student Engagement, Success and Retention Maturity Model (SESR-MM): A reality check of the model using data derived from three universities

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). We have proposed that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities and we are currently articulating a Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012). Our research aims to address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. Our research extends the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). Our research also moves beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. The issues explored in this workshop are (i) whether the maturity model concept can be usefully applied to provide a measure of institutional capability for SESR; (ii) whether the SESR-MM can be used to assess the maturity of a particular set of institutional practices; and (iii) whether a collective assessment of an institution’s SESR capabilities can provide an indication of the maturity of the institution’s SESR activities. The workshop will be approached in three stages. Firstly, participants will be introduced to the key characteristics of maturity models, followed by a discussion of the SESR-MM and the processes involved in its development. Secondly, participants will be provided with resources to facilitate the development of a maturity model and an assessment instrument for a range of institutional processes and related practices. In the final stage of the workshop, participants will “assess” the capability of these practices to provide a collective assessment of the maturity of these processes. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Moving beyond transition pedagogy: Maturity models and student engagement

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). This paper proposes that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities. We will describe the Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012) we are currently investigating. We will discuss if our research may address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. The first aim of our research is to extend the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). The second aim of this research is to move beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Student engagement with an ePortfolio : a case study of pre-service education students

The emergence of ePortfolios is relatively recent in the university sector as a way to engage students in their learning and assessment, and to produce records of their accomplishments. An ePortfolio is an online tool that students can utilise to record, catalogue, retrieve and present reflections and artefacts that support and demonstrate the development of graduate students’ capabilities and professional standards across university courses. The ePortfolio is therefore considered as both process and product. Although ePortfolios show promise as a useful tool and their uptake has grown, they are not yet a mainstream higher education technology. To date, the emphasis has been on investigating their potential to support the multiple purposes of learning, assessment and employability, but less is known about whether and how students engage with ePortfolios in the university setting. This thesis investigates student engagement with an ePortfolio in one university. As the educational designer for the ePortfolio project at the University, I was uniquely positioned as a researching professional to undertake an inquiry into whether students were engaging with the ePortfolio. The participants in this study were a cohort (defined by enrolment in a unit of study) of second and third year education students (n=105) enrolled in a four year Bachelor of Education degree. The students were introduced to the ePortfolio in an introductory lecture and a hands-on workshop in a computer laboratory. They were subsequently required to complete a compulsory assessment task – a critical reflection - using the ePortfolio. Following that, engagement with the ePortfolio was voluntary. A single case study approach arising from an interpretivist paradigm directed the methodological approach and research design for this study. The study investigated the participants’ own accounts of their experiences with the ePortfolio, including how and when they engaged with the ePortfolio and the factors that impacted on their engagement. Data collection methods consisted of an attitude survey, student interviews, document collection, a researcher reflective journal and researcher observations. The findings of the study show that, while the students were encouraged to use the ePortfolio as a learning and employability tool, most students ultimately chose to disengage after completing the assessment task. Only six of the forty-five students (13%) who completed the research survey had used the ePortfolio in a sustained manner. The data obtained from the students during this research has provided insight into reasons why they disengaged from the ePortfolio. The findings add to the understandings and descriptions of student engagement with technology, and more broadly, advance the understanding of ePortfolios. These findings also contribute to the interdisciplinary field of technology implementation. There are three key outcomes from this study, a model of student engagement with technology, a set of criteria for the design of an ePortfolio, and a set of recommendations for effective practice for those implementing ePortfolios. The first, the Model of Student Engagement with Technology (MSET) (Version 2) explored student engagement with technology by highlighting key engagement decision points for students The model was initially conceptualised by building on work of previous research (Version 1), however, following data analysis a new model emerged, MSET (Version 2). The engagement decision points were identified as: • Prior Knowledge and Experience, leading to imagined usefulness and imagined ease of use; • Initial Supported Engagement, leading to supported experience of usefulness and supported ease of use; • Initial Independent Engagement, leading to actual experience of independent usefulness and actual ease of use; and • Ongoing Independent Engagement, leading to ongoing experience of usefulness and ongoing ease of use. The Model of Student Engagement with Technology (MSET) goes beyond numerical figures of usage to demonstrate student engagement with an ePortfolio. The explanatory power of the model is based on the identification of the types of decisions that students make and when they make them during the engagement process. This model presents a greater depth of understanding student engagement than was previously available and has implications for the direction and timing of future implementation, and academic and student development activities. The second key outcome from this study is a set of criteria for the re-conceptualisation of the University ePortfolio. The knowledge gained from this research has resulted in a new set of design criteria that focus on the student actions of writing reflections and adding artefacts. The process of using the ePortfolio is reconceptualised in terms of privileging student learning over administrative compliance. The focus of the ePortfolio is that the writing of critical reflections is the key function, not the selection of capabilities. The third key outcome from this research consists of five recommendations for university practice that have arisen from this study. They are that, sustainable implementation is more often achieved through small steps building on one another; that a clear definition of the purpose of an ePortfolio is crucial for students and staff; that ePortfolio pedagogy should be the driving force not the technology; that the merit of the ePortfolio is fostered in students and staff; and finally, that supporting delayed task performance is crucial. Students do not adopt an ePortfolio just because it is provided. While students must accept responsibility for their own engagement with the ePortfolio, the institution has to accept responsibility for providing the environment, and technical and pedagogical support to foster engagement. Ultimately, an ePortfolio should be considered as a joint venture between student and institution where strong returns on investment can be realised by both. It is acknowledged that the current implementation strategies for the ePortfolio are just the beginning of a much longer process. The real rewards for students, academics and the university lie in the future.

Transforming higher education and student engagement through collaborative review to inform educational design

This paper reports on staff perceptions arising from a review process designed to assist staff in making informed decisions regarding educational design, approaches to engage students in learning, and the technology to support engagement in the classroom and across multiple locations and delivery modes. The aim of the review process was to transform the level of student engagement in the business faculty of an Australian university. The process took a collaborative approach through consultation with academic staff involved in the design and delivery of the units under review, and included targeted professional development as necessary. An institutional framework that characterises engagement indicator contexts and their attributes facilitated dialog during the review process. This paper reports on a mixed method study that included a survey of participants, and purposeful interviews to evaluate the effectiveness of the process. Although the study identified factors that hindered implementation and operationalization of review recommendations in some instances, study participants were generally of the view that recommendations would enhance student engagement. It is demonstrated that the bottom-up approach described in this paper is consistent with theoretical frameworks for transformational change in teaching and learning and the adoption of innovations.

Intentional first year curriculum design as a means of facilitating student engagement: some exemplars

The importance of student engagement to higher education quality, making deep learning outcomes possible for students, and achieving student retention, is increasingly being understood. The issue of student engagement in the first year of tertiary study is of particular significance. This paper takes the position that the first year curriculum, and the pedagogical principles that inform its design, are critical influencers of student engagement in the first year learning environment. We use an analysis of case studies prepared for Kift’s ALTC Senior Fellowship to demonstrate ways in which student engagement in the first year of tertiary study can be successfully supported through intentional curriculum design that motivates students to learn, provides a positive learning climate, and encourages students to be active in their learning.

The maturity model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention : new horizon or false dawn?

While the engagement, success and retention of first year students are ongoing issues in higher education, they are currently of considerable and increasing importance as the pressures on teaching and learning from the new standards framework and performance funding intensifies. This Nuts & Bolts presentation introduces the concept of a maturity model and its application to the assessment of the capability of higher education institutions to address student engagement, success and retention. Participants will be provided with (a) a concise description of the concept and features of a maturity model; and (b) the opportunity to explore the potential application of maturity models (i) to the management of student engagement and retention programs and strategies within an institution and (ii) to the improvement of these features by benchmarking across the sector.

The place of higher education institutions in assessing student engagement, success and retention : a maturity model to guide practice

The perennial issues of student engagement, success and retention in higher education continue to attract attention as the salience of teaching and learning funding and performance measures has increased. This paper addresses the question of the responsibility or place of higher education institutions (HEIs) for initiating, planning, managing and evaluating their student engagement, success and retention programs and strategies. An evaluation of the current situation indicates the need for a sophisticated approach to assessing the ability of HEIs to proactively design programs and practices that enhance student engagement. An approach—the Student Engagement Success and Retention Maturity Model (SESR-MM)—is proposed and its development, current status, and relationship with and possible use in benchmarking are discussed.

The potentials of student initiated netspeak in a middle primary science-inspired mutliliteracies project

There is no denying that the information technology revolution of the late twentieth century has arrived. Whilst not equitably accessible for many, others hold high expectations for the contributions online activity will make to student learning outcomes. Concurrently, and not necessarily consequentially, the number of science and technology secondary school and university graduates throughout the world has declined substantially, as has their motivation and engagement with school science (OECD, 2006). The aim of this research paper is to explore one aspect of online activity, that of forum-based netspeak (Crystal, 2006), in relation to the possibilities and challenges it provides for forms of scientific learning. This paper reports findings from a study investigating student initiated netspeak in a science inspired multiliteracies (New London Group, 2000) project in one middle primary (aged 7-10 years) multi-age Australian classroom. Drawing on the theoretical description of the Five phases of enquiry proposed by Bybee (1997), an analytic framework is proffered that allows identification of student engagement, exploration, explanation, elaboration and evaluation of scientific enquiry. The findings provide insight into online forums for advancing learning in and motivation for science in the middle primary years.

Engaging marketing students : student operated businesses in a simulated world

Engaged students are committed and more likely to continue their university studies. Subsequently, they are less resource intensive from a university’s perspective. This article details an experiential second-year marketing course that requires students to develop real products and services to sell on two organized market days. In the course, students participate as both consumers and marketers in a simulated world. The current article explores the effectiveness of this experiential assessment in terms of its ability to engage students. Comparing student engagement to a traditional lecture course and National Survey of Student Engagement benchmarks, the results suggest that the use of a simulated marketplace is capable of engaging students. Specifically, the assessment reported encourages more active learning and collaboration, is more academically challenging, and permits more student–faculty interaction than a traditional lecture-based course. The course structure outlined in this article permits the dynamics of a live marketing environment to be introduced into the classroom. The authors provide practical advice for educators seeking to design and implement engaging pedagogy.

Establishing a framework for transforming student engagement, success and retention in higher education institutions [Final Report]

The perennial issues of student engagement, success and retention (SESR) in higher education continue to attract attention as key indicators of learning and teaching quality. This project aimed to establish and provide a holistic framework that would allow higher education institutions (HEIs) manage and improve their student engagement and retention strategies and programs. The framework and main project deliverable is a Maturity Model (MM) for Student Engagement, Success and Retention (SESR-MM). The project involved three Australian universities with experience and reputations in SESR activities: Queensland University of Technology (lead institution), the University of Queensland and Griffith University, working cooperatively to develop and trial the project deliverables. Project findings suggest that the SESR-MM has the potential to positively transform the holistic—academic, social and personal—engagement experiences of students in Australian universities, and that the SESR-MM is a useful mechanism for sharing good practice and improving programs designed to enhance the student experience.

A comparative study of marketing student engagement in online social networks and virtual worlds : Facebook versus second life

The study evaluated two student online contemporary learning environments; Second Life and Facebook, student learning experiences and student knowledge outcomes. A case study methodology was used to gain rich exploratory knowledge of student learning when integrating online social networks (OSN) and virtual worlds (VW) platforms. Findings indicated students must perceive relevance in the activities when using such platforms, even though online environments create an interesting learning space for students and educators, the novelty can diminish quickly and these online environments dilute traditional authority boundaries.

Acknowledging social justice and equity through good practice for monitoring student learning engagement in FYE

As higher education institutions respond to government targets to widen participation, their student populations will become increasingly diverse, and the issues around student success and retention will be more closely scrutinised. The concept of student engagement is a key factor in student achievement and retention and Australasian institutions have a range of initiatives aimed at monitoring and intervening with students who are at risk of disengaging. Within the widening participation agenda, it is absolutely critical that these initiatives are designed to enable success for all students, particularly those for whom social and cultural disadvantage have been a barrier. Consequently, for the sector, initiatives of this type must be consistent with the concept of social justice and a set of principles would provide this foundation. This session will provide an opportunity for participants to examine a draft set of principles and to discuss their potential value for the participants’ institutional contexts.

An exploration of the maturity model concept as a vehicle for higher education institutions to assess their capability to address student engagement : a work in progress

Gaining a competitive edge in the area of the engagement, success and retention of commencing students is a significant issue in higher education, made more so currently because of the considerable and increasing pressure on teaching and learning from the new standards framework and performance funding. This paper introduces the concept of maturity models (MMs) and their application to assessing the capability of higher education institutions (HEIs) to address student engagement, success and retention (SESR). A concise description of the features of maturity models is presented with reference to an SESR-MM currently being developed. The SESR-MM is proposed as a viable instrument for assisting HEIs in the management and improvement of their SESR activities.

Using a maturity model to move student engagement practices beyond the generational approach

This paper proposes that the generational approach to conceptualising first year student learning behaviour, while it has made a very useful contribution to understanding that behaviour, can be expanded upon. The generational approach has an explicit focus on student behaviour and it is suggested that a capability maturity model interpretation may provide a complementary extension of that as it allows an assessment of institutional capability to initiate, plan, manage and evaluate institutional student engagement practices. The development of a Student Engagement, Success and Retention Maturity Model (SESR-MM) is discussed along with Australasian FYE generational data and Australian SESR-MM data.