Australian higher education evaluation through assurance of learning

A collaborative research project conducted by five Australian universities inquired into the philosophy and motivation for Assurance of Learning (AoL) as a process of education evaluation. Associate Deans Teaching and Learning representing Business schools from twenty-five universities across Australia participated in telephone interviews. Data was analysed using NVIVO9. Results indicated that articulated rationale for AoL was both ensuring that students had acquired the attributes and skills the universities claimed they had, and the philosophy of continuous improvement. AoL was motivated both by ritualistic objectives to satisfy accreditation requirements and virtuous agendas for quality improvement. Closing-the-loop was emphasised, but was mostly wishful thinking for next steps beyond data collection and reporting. AoL was conceptualised as one element within the larger context of quality review, but there was no evidence of comprehensive frameworks or strategic plans.

Patient deterioration education: Evaluation of face-to-face simulation and e-simulation approaches

Background: Simulation-based education is one strategy that may be used to teach nursing students to recognize and manage patient deterioration. Method: Final-year preregistration nursing students (n=97) completed three face-to-face laboratory-based team simulations with a simulated patient (actor) and 330 students individually completed a three-scenario Web-based simulation program: FIRST2ACTWeb™. Results: Both groups achieved moderate performance scores (means: face to face, 49%; e-simulation, 69%). Course evaluations were positive, skill gain showing a greater effect size in the face-to-face program than for e-simulation, and higher satisfaction and more positive appraisal. Conclusion: Face-to-face simulation and e-simulation are effective educational strategies with e-simulation offering greater feasibility. Either strategy is likely to add value to the learning experience.

Promoting a functional physical self-concept in physical education: Evaluation of a 10-week intervention

Most physical education intervention studies on the positive effect of sports on self-concept development have attempted to increase schoolchildren’s self-concept without taking the veridicality of the self-concept into account. The present study investigated whether a 10-week intervention in physical education would lead to an increase not only in the general level of self-concept of endurance and self-concept of strength but also in its veridicality in those who had previously under- or overestimated their abilities. A total of 464 primary schoolchildren (246 boys, 218 girls, Mage = 11.9) either participated in the intervention or served as controls. The intervention group received endurance and strength training during physical education lessons carried out with a consistent individualized teacher frame of reference (iTFR). Results showed that this specific intervention was associated with increases not only in the general level of self-concept but also in its veridicality in under- and overestimators. Results are discussed in terms of didactic methods to promote functional self-concepts in physical education.

An electronic record system in nursing education: evaluation and utilization

The main objective of the present work is to analyze the results of the utilization and evaluation of the LORETO Record System (LRS), providing improvement areas in the teaching-learning process and technology, in second year nursing students. A descriptive, prospective, cross sectional study using inferential statics has been carried out on all electronic records reported by 55 nursing students during clinical internships (April 1º-June 26º, 2013). Electronic record average rated 7.22 points (s=0.6; CV=0.083), with differences based on the clinical practice units (p<0,05). Three items assessed did not exceed the quality threshold set at 0.7 (p<0.05). Record Rate exceeds the quality threshold set at 80% for the overall sample, with differences based on the practice units. Only two clinical practice units rated above the minimum threshold (p <0.05). Record of care provision every 3 days did not reach the estimated quality threshold (p <0.05). There is a dichotomy between qualitative and quantitative results of LRS. Improvement areas in theoretical education have been identified. The LRS seems an appropriate learning and assessment tool, although the development of a new APP version and the application of principles of gamification should be explored.

The Driver Education Evaluation Program (DEEP) study. A report to the Congress.

Mode of access: Internet.

The Driver Education Evaluation Program (DEEP) study. Report to Congress. Second.

Mode of access: Internet.

Peer mentoring works! How peer mentoring enhances student success in Higher Education:evaluation toolkit

This toolkit, published by the HEA, provides colleagues across the Sector with the practical and methodological tools to empirically evaluate peer mentoring and tutoring. This evaluation kit provides two data collection tools that may be adopted and adapted to meet institutional requirements. The first of these is a survey, developed out of the original survey used in the Peer Mentoring Works Project. Some questions have been added as a result of reflexive application of colleagues and students input as the project has progressed. The second part of the toolkit comprises a qualitative interview guide. This guide is similar to the one used in the research, but again it has been further developed as a result of the project. It may be adapted for use in focus groups or one-to-one interviews. The final document within the kit is a sample consent form.

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

An evaluation of a project-based learning initiative in engineering education

Project-based learning (PBL) is a well-known methodology for engineering design education due to a number of benefits it is claimed to offer. This paper presents the initial offering of a first-year engineering PBL unit at Griffith University in Australia. An evaluation of student perceptions of the unit revealed that students generally enjoyed the experience, with the oral presentation aspect receiving the lowest satisfaction rating. There was no significant difference in the ratings between any demographic grouping, suggesting that all students were able to participate in, and experience, the unit in essentially the same way. The best aspects of the unit and those aspects needing improvement were similar to the findings of other investigations documented in the literature. It is proposed that future offerings of the unit will reduce the number of design projects from three to two per semester and will attempt more sophisticated individualisation of marks for group work activities.

Reframing evaluation of learning and teaching: An approach to change

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.