Peer mentoring works! How peer mentoring enhances student success in Higher Education:evaluation toolkit

This toolkit, published by the HEA, provides colleagues across the Sector with the practical and methodological tools to empirically evaluate peer mentoring and tutoring. This evaluation kit provides two data collection tools that may be adopted and adapted to meet institutional requirements. The first of these is a survey, developed out of the original survey used in the Peer Mentoring Works Project. Some questions have been added as a result of reflexive application of colleagues and students input as the project has progressed. The second part of the toolkit comprises a qualitative interview guide. This guide is similar to the one used in the research, but again it has been further developed as a result of the project. It may be adapted for use in focus groups or one-to-one interviews. The final document within the kit is a sample consent form.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

Evaluation of the quality of diverse teaching practice in higher education research methods

The purpose of this paper is to report on a methods research project investigating the evaluation of diverse teaching practice in Higher Education. The research method is a single site case study of an Australian university with data collected through published documents, surveys, interviews and focus groups. This project provides evidence of the wide variety of evaluation practice and diverse teaching practice across the university. This breadth identifies the need for greater flexibility of evaluation processes, tools and support to assist teaching staff to evaluate their diverse teaching practice. The employment opportunities for academics benchmark the university nationally and position the case study in the field. Finally this reaffirms the institutional responsibility for services to support teaching staff in an ongoing manner.

REFRAME : a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

From policy borrowing to implementation : an illuminative evaluation of learning and teaching in higher education in Australia (2002 to 2008)

This study documents and theorises the consequences of the 2003 Australian Government Reform Package focussed on learning and teaching in Higher Education during the period 2002 to 2008. This is achieved through the perspective of program evaluation and the methodology of illuminative evaluation. The findings suggest that the three national initiatives of that time, Learning and Teaching Performance Fund (LTPF), Australian Learning and Teaching Council (ALTC), and Australian Universities Quality Agency (AUQA), were successful in repositioning learning and teaching as a core activity in universities. However, there were unintended consequences brought about by international policy borrowing, when the short-lived nature of LTPF suggests a legacy of quality compliance rather than one of quality enrichment.

Gender differences in internet usage habits: a case study in higher education

The usage of information and communication technologies has been growing among students and teachers. In order to improve the use of the Internet as a tool to support teaching and learning it is necessary to understand the Internet usage habits of students. Thus, a study was conducted with 1397 students from five schools of the Polytechnic of Porto. The data was collected through an online questionnaire abd was analized by age range, gender and scientific field. In this paper,gender differences are analyzed and presented in 3 dimensions: type of Internet usage, communication tools and the role of the Internet tools in education.

Quality in Blended Learning in Higher Education. A proposal for an Evaluation Model.

This paper presents an analysis of different models used to assess the quality of formative actions, considering classroom learning and distance education courses. Taking as starting point one of the analyzed models, the paper sets out the necessity of developing a new model that could measure the quality of a blended formation process, by selecting the applicable indicators and proposing some new. The model is composed of seven different categories, which include a sum of thirty five indicators. They will be used to represent courses quality level in Kiviat?s diagrams. This model is currently being put into practice in a real university environment.

A Soft Computing Approach to Quality Evaluation of General Chemistry Learning in Higher Education

In contemporary societies higher education must shape individuals able to solve problems in a workable and simpler manner and, therefore, a multidisciplinary view of the problems, with insights in disciplines like psychology, mathematics or computer science becomes mandatory. Undeniably, the great challenge for teachers is to provide a comprehensive training in General Chemistry with high standards of quality, and aiming not only at the promotion of the student’s academic success, but also at the understanding of the competences/skills required to their future doings. Thus, this work will be focused on the development of an intelligent system to assess the Quality-of-General-Chemistry-Learning, based on factors related with subject, teachers and students.

Australian higher education evaluation through assurance of learning

A collaborative research project conducted by five Australian universities inquired into the philosophy and motivation for Assurance of Learning (AoL) as a process of education evaluation. Associate Deans Teaching and Learning representing Business schools from twenty-five universities across Australia participated in telephone interviews. Data was analysed using NVIVO9. Results indicated that articulated rationale for AoL was both ensuring that students had acquired the attributes and skills the universities claimed they had, and the philosophy of continuous improvement. AoL was motivated both by ritualistic objectives to satisfy accreditation requirements and virtuous agendas for quality improvement. Closing-the-loop was emphasised, but was mostly wishful thinking for next steps beyond data collection and reporting. AoL was conceptualised as one element within the larger context of quality review, but there was no evidence of comprehensive frameworks or strategic plans.

Student feedback systems in higher education: a focused literature review and environmental scan

In recent times, higher education institutions have paid increasing attention to the views of students to obtain feedback on their experience of learning and teaching through internal surveys. This article reviews research in the field and reports on practices in other Australian universities. Findings demonstrate that while student feedback is valued and used by all Australian universities, survey practices are idiosyncratic and in the majority of cases, questionnaires lack validity and reliability; data are used inadequately or inappropriately; and they offer limited potential for cross-sector benchmarking. The study confirms the need for institutions to develop an overarching framework for evaluation in which a valid, reliable, multidimensional and useful student feedback survey constitutes just one part. Given external expectations and internal requirements to collect feedback from students on their experience of learning and teaching, the pursuit of sound evaluation practices will continue to be of interest at local, national and international levels.

Towards success in higher education in engineering and technology : a coaching approach to develop holistic graduates

There is a growing gap between engineering practice and engineering education that may be contributing to less engineers practicing in industry. Coaching approach to learning and teaching has been proven to be an effective way to develop people in the workplace. A pilot coaching program is offered to Engineering and Technology students in Queensland University of Technology to enable holistic growth in order to better integrate them to the work force and society at large. The results and findings of this program will be published once the program has been completed

Identifying discourses of moderation in higher education

Moderation of student assessment is a critical component of teaching and learning in contemporary universities. Yet, despite this, it tends to be marked by idiosyncratic and sporadic processes informed by liminal understanding. This paper, in the light of forthcoming radical national requirements for the declaration of moderation processes in tertiary curricula in Australia, will present four discourses of moderation we identified in a recent study in a Faculty of Education in a large metropolitan university. The discourses are equity, justification, community building and accountability. Together, they will act as a starting point for academics to review their beliefs and attitudes towards the moderation of student assessment.

Establishing a framework for transforming student engagement, success and retention in higher education institutions [Final Report]

The perennial issues of student engagement, success and retention (SESR) in higher education continue to attract attention as key indicators of learning and teaching quality. This project aimed to establish and provide a holistic framework that would allow higher education institutions (HEIs) manage and improve their student engagement and retention strategies and programs. The framework and main project deliverable is a Maturity Model (MM) for Student Engagement, Success and Retention (SESR-MM). The project involved three Australian universities with experience and reputations in SESR activities: Queensland University of Technology (lead institution), the University of Queensland and Griffith University, working cooperatively to develop and trial the project deliverables. Project findings suggest that the SESR-MM has the potential to positively transform the holistic—academic, social and personal—engagement experiences of students in Australian universities, and that the SESR-MM is a useful mechanism for sharing good practice and improving programs designed to enhance the student experience.