Verifying a Student Engagement, Success and Retention Maturity Model (SESR-MM): A reality check of the model using data derived from three universities

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). We have proposed that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities and we are currently articulating a Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012). Our research aims to address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. Our research extends the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). Our research also moves beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. The issues explored in this workshop are (i) whether the maturity model concept can be usefully applied to provide a measure of institutional capability for SESR; (ii) whether the SESR-MM can be used to assess the maturity of a particular set of institutional practices; and (iii) whether a collective assessment of an institution’s SESR capabilities can provide an indication of the maturity of the institution’s SESR activities. The workshop will be approached in three stages. Firstly, participants will be introduced to the key characteristics of maturity models, followed by a discussion of the SESR-MM and the processes involved in its development. Secondly, participants will be provided with resources to facilitate the development of a maturity model and an assessment instrument for a range of institutional processes and related practices. In the final stage of the workshop, participants will “assess” the capability of these practices to provide a collective assessment of the maturity of these processes. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Building sustainable and effective assurance of learning process in a changing higher education environment

In a study of assuring learning in Australian Business Schools, 25 Teaching and Learning Associate Deans were interviewed to identify current issues in developing and measuring the quality of teaching and learning outcomes. Results indicate that for most institutions developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program required knowledge creation and the building of new inclusive processes. Common elements of effective practice, namely those which offered consistently superior outcomes, included: inclusive processes; embedded graduate attributes throughout a program; alongside consistent and appropriate assessment. Results indicate that assurance of learning processes are proliferating nationally while quality of teaching and learning outcomes and in the processes for assuring it is increasing as a result.

"More tick-the-box": the challenge of promoting interdisciplinary learning in the middle years through the Australian history curriculum

The Australian Curriculum marks national reforms in social science education, first with the return to the disciplines of history and geography and second, through a new approach to interdisciplinary learning. This paper raises the question of whether the promise of interdisciplinary learning can be realised in the middle years of schooling if teachers have to teach history as a discipline rather than within an over-arching integrated curriculum framework. The paper explores the national blueprints and considers the national history curriculum in light of theories of teachers’ knowledge and middle school education. Evidence from teacher interviews indicates that historical understanding can be achieved through integrated frameworks to meet the goals of middle schooling.

Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.

The role of knowledge intermediaries in developing firm learning capabilities

Regional innovation systems (RISs) literature has emphasized the critical role of interactive learning and knowledge exchange amongst firms and a variety of spatially connected innovation institutions as the foundation of regional innovation. Knowledge intermediaries have been analysed in terms of the technology-transaction services they provide firms and/or knowledge producers such as universities and therefore the role they play in facilitating interaction within the RIS. However, innovation also depends on the capability of the firm to learn. Some studies have suggested that intermediaries also play a role in that regard as participation in intermediary knowledge transfer programmes can contribute to the development of firm capabilities for problem-solving and learning. Our research is based on two case study intermediary programmes involving interviews with facilitators and participants. Our data show that knowledge intermediaries affect organizational learning capabilities by impacting on firms' network relationships, internal and external communication channels and internal learning processes which in turn affect the ability to interpret and use knowledge within the firm. This suggests that the role of knowledge intermediaries might be greater than facilitating interactions in the innovation system, as knowledge intermediation may affect the ability of firms to learn and absorb knowledge from their environment.

So what do Design students really think? Exploring perceptions of Learning Landscapes

This paper investigates learning environments from the view of the key users - students. Recent literature on designing Learning Landscapes indicates a near absence of the student voice, assuming that the majority of students are either uninterested or unable to express what they want or need, in a learning environment. The focus of this research is to reveal Architecture and Fashion Design students’ perceptions of their learning environments. Furthermore, this study questions the appropriateness of usual design of learning spaces for Design students, or if the environment needs to be specifically catered for the learning of different disciplines of Design, such as Architecture and Fashion Design. Senior Architecture and Fashion Design students were invited to participate in a qualitative mixed method study, including investigation into existing literature, questionnaires, focus groups and spontaneous participatory research. Through the analysis of data it was found that students’ perceptions validate discipline specific learning environments and contribute towards the development of a framework for the design of future Learning Landscapes, for Design education.

Evaluation of the "reconceptualising early mathematics learning" project

The Pattern and Structure Mathematics Awareness Project (PASMAP) has investigated the development of patterning and early algebraic reasoning among 4 to 8 year olds over a series of related studies. We assert that an awareness of mathematical pattern and structure enables mathematical thinking and simple forms of generalisation from an early age. The project aims to promote a strong foundation for mathematical development by focusing on critical, underlying features of mathematics learning. This paper provides an overview of key aspects of the assessment and intervention, and analyses of the impact of PASMAP on students’ representation, abstraction and generalisation of mathematical ideas. A purposive sample of four large primary schools, two in Sydney and two in Brisbane, representing 316 students from diverse socio-economic and cultural contexts, participated in the evaluation throughout the 2009 school year and a follow-up assessment in 2010. Two different mathematics programs were implemented: in each school, two Kindergarten teachers implemented the PASMAP and another two implemented their regular program. The study shows that both groups of students made substantial gains on the ‘I Can Do Maths’ assessment and a Pattern and Structure Assessment (PASA) interview, but highly significant differences were found on the latter with PASMAP students outperforming the regular group on PASA scores. Qualitative analysis of students’ responses for structural development showed increased levels for the PASMAP students; those categorised as low ability developed improved structural responses over a relatively short period of time.

Governance of learning mechanisms : evidence from construction firms

The focus of knowledge management (KM) in the construction industry is moving towards capability building for value creation. The study reported by this paper is motivated by recent assertions about the genesis and evolution of knowledge management capability (KMC) in the strategic management field. It attempts to shed light on the governance of learning mechanisms that develop KMC within the context of construction firms. A questionnaire survey was administered to a sample of construction contractors operating in the very dynamic Hong Kong market to elicit opinions on the learning mechanisms and business outcomes of targeted firms. On the basis of a total of 149 usable responses, structural equation modeling (SEM) analysis identified relationships among knowledge-governance mechanisms, knowledge processes, and business performance, thereby supporting the existence of strategic learning loops. The study findings provide evidence from the construction context for capability assertions that knowledge-governance mechanisms and processes form learning mechanisms that carry out strategic learning to create value, effect performance outcomes, and ultimately drive the evolution of KMC. The findings imply that it is feasible for managing construction firms to govern learning mechanisms through managing the capability-based holistic KM system, thereby reconfiguring KMC to match needs in the dynamic market environment over time.

Peer assisted learning, skills development and Generation Y : a case study of a first year undergraduate law unit

The majority of current first year university students belong to Generation Y. Consequently, research suggests that, in order to more effectively engage them, their particular learning preferences should be acknowledged in the organisation of their learning environments and in the support provided. These preferences are reflected in the Torts Student Peer Mentor Program, which, as part of the undergraduate law degree at the Queensland University of Technology, utilises active learning, structured sessions and teamwork to supplement student understanding of the substantive law of Torts with the development of life-long skills. This article outlines the Program, and its relevance to the learning styles and experiences of Generation Y first year law students transitioning to university, in order to investigate student perceptions of its effectiveness – both generally and, more specifically, in terms of the Program’s capacity to assist students to develop academic and work-related skills.

Critical success elements for the design and implementation of organisational e-learning

Organisations are engaging in e-learning as a mechanism for delivering flexible learning to meet the needs of individuals and organisations. In light of the increasing use and organisational investment in e-learning, the need for methods to evaluate the success of its design and implementation seems more important than ever. To date, developing a standard for the evaluation of e-learning appears to have eluded both academics and practitioners. The currently accepted evaluation methods for e-learning are traditional learning and development models, such as Kirkpatrick’s model (1976). Due to the technical nature of e-learning it is important to broaden the scope and consider other evaluation models or techniques, such as the DeLone and McLean Information Success Model, that may be applicable to the e-learning domain. Research into the use of e-learning courses has largely avoided considering the applicability of information systems research. Given this observation, it is reasonable to conclude that e-learning implementation decisions and practice could be overlooking useful or additional viewpoints. This research investigated how existing evaluation models apply in the context of organisational e-learning, and resulted in an Organisational E-learning success Framework, which identifies the critical elements for success in an e-learning environment. In particular this thesis highlights the critical importance of three e-learning system creation elements; system quality, information quality, and support quality. These elements were explored in depth and the nature of each element is described in detail. In addition, two further elements were identified as factors integral to the success of an e-learning system; learner preferences and change management. Overall, this research has demonstrated the need for a holistic approach to e-learning evaluation. Furthermore, it has shown that the application of both traditional training evaluation approaches and the D&M IS Success Model are appropriate to the organisational e-learning context, and when combined can provide this holistic approach. Practically, this thesis has reported the need for organisations to consider evaluation at all stages of e-learning from design through to implementation.

Unmasking workshop ecology in applied performance

This practice-led research enquiry identifies, develops and illustrates workshop ecology in Applied Performance. It explores how Applied Performance forms are applied in and transformed through action in two distinct community-learning settings. The research is undertaken in two performance sites. The first, involving an executive leadership program addressing complex project management for Australia's Defence Materiel Organisation in Canberra, Australia. The second, a sexual health, HIV and AIDS education program to raise awareness and encourage the prevention of transmission of sexual diseases within Karkar Island, Papua New Guinea. The research strategies draw upon a mixed method approach involving practice-led research participant observation. The findings from each performance site show how the workshop ecology shapes and transforms performance forms as they are applied and influences the degree to which they are effective. It is anticipated that the findings from this research will assist Applied Performance practitioners to more carefully consider workshop ecology in the design and delivery of Applied Performances.

Effects of introducing an interactive teaching and learning activity (TLA) in the engineering classroom

This research study examines qualitatively and quantitatively the influence of introducing an activity in the traditional engineering classroom. It studies instances of active learning and its relationship with the student learning outcomes. The primary purpose of this study was to compare the learning outcomes of students who were involved in an active TLA with those students who were not, instead they learned under traditional teaching and studying approaches. I present the argument that the introduction of a TLA in class stimulates student engagement bringing enormous benefits to student learning. The outcomes of this study were measured using qualitative and quantitative data to evaluate the levels of student engagement, achievement and satisfaction in the terms of Intended Learning Outcomes (ILOs). Results indicate that students held positive attitude towards the activities in class and also, that a positive link between TLA, learning approach and learning outcome exist. It also provides insights about the potential benefits of active learning when compared with traditional, passive and teacher-centred methods of teaching & learning.

Connected learning in the library as a product of hacking, making, social diversity and messiness

Learning is most effective when intrinsically motivated through personal interest, and situated in a supportive socio-cultural context. This paper reports on findings from a study that explored implications for design of interactive learning environments through 18 months of ethnographic observations of people’s interactions at “Hack The Evening” (HTE). HTE is a meetup group initiated at the State Library of Queensland in Brisbane, Australia, and dedicated to provide visitors with opportunities for connected learning in relation to hacking, making and do-it-yourself technology. The results provide insights into factors that contributed to HTE as a social, interactive and participatory environment for learning – knowledge is created and co-created through uncoordinated interactions among participants that come from a diversity of backgrounds, skills and areas of expertise. The insights also reveal challenges and barriers that the HTE group faced in regards to connected learning. Four dimensions of design opportunities are presented to overcome those challenges and barriers towards improving connected learning in library buildings and other free-choice learning environments that seek to embody a more interactive and participatory culture among their users. The insights are relevant for librarians as well as designers, managers and decision makers of other interactive and free-choice learning environments.

Dancing Diseases : an applied theatre response to the challenge of conveying emotionally contradictory messages in HIV education

Health educators face an unusual challenge in relation to HIV: the need to convey two emotionally contradictory messages. On the one hand, there is currently no cure for HIV, which eventually leads to death (emotionally negative message). On the other hand, people with HIV can live long, healthy and productive lives (emotionally positive message). In developing countries where HIV prevalence is high, it is imperative that both messages are conveyed effectively. This article reports on a specific form, Dancing Diseases, implemented as one component of the Life Drama pilot study on Karkar Island, Papua New Guinea. Life Drama is an applied theatre and performance approach to HIV education. The article discusses Dancing Diseases as an example of applied theatre and performance practice, reflects on the participant group’s engagement with the form, and offers some ways in which the form could be refined and used in other health education contexts.

The practice of quality assuring learning in higher education

There remains a lack of published empirical data on the substantive outcomes of higher learning and the establishment of quality processes for determining them. Studies that do exist are nationally focused with available rankings of institutions reflecting neither the quality of teaching and learning nor the diversity of institutions. This paper describes two studies in which Associate Deans from Australian higher education institutions and focus groups of management and academics identify current issues and practices in the design, development and implementation of processes for assuring the quality of learning and teaching. Results indicate that developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program necessitates knowledge creation and new inclusive processes. Common elements supporting consistently superior outcomes included: inclusivity; embedded graduate attributes; consistent and appropriate assessment; digital collection mechanisms; and systematic analysis of outcomes used in program review. Quality measures for assuring learning are proliferating nationally and changing the processes, systems and culture of higher education as a result.