Video-stimulated recall in cross-cultural research in education: A case study in Vietnam

This paper examines incorporating video-stimulated recall (VSR) as a data collection technique in cross-cultural research. With VSR, participants are invited to watch video-recordings of particular events that they are involved in; they then recall their thoughts in relation to their observations of their behaviour in relation the event. The research draws on a larger PhD project completed at an Australian university that explored Vietnamese lecturers’ beliefs about learner autonomy. In cross-cultural research using the VSR technique provided significant challenges including time constraints of participants, misunderstandings of the VSR protocol and the possibility of participants’ losing face when reflecting on their teaching episodes. Adaptations to the VSR technique were required to meet the cultural challenges specific to this population, indicating a need for flexibility and awareness of the cultural context for research.

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

REFRAME : a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

Illuminative evaluation as a method applied to Australian Government policy borrowing and implementation in higher education

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

Personal epistemology in pre-service teachers : belief changes throughout a teacher education course

Classrooms of the 21st century are complex systems. They support diverse learners from varied contexts and function in a “messy” bricolage of policy contexts. This complexity is also evident in the nature of teaching and learning deployed in these classrooms. There is also, in current contexts, a general expectation that teachers will support students to construct, rather than simply receive knowledge. This process of constructing knowledge requires a focus on critical thinking in complex social and real world contexts (see also Elen & Clarebout, 2001; Yang, Chang & Hsu 2008). Critical thinking, which involves the identification and evaluation of multiple perspectives when making decisions, is a process of knowing – a tool of wisdom (Kuhn & Udell, 2001). Schommer-Aikens, Bird and Bakken (2010) refer to classrooms that encourage critical thinking as “epistemologically based” in which “the teacher encourages his/her students to look for connections among concepts within the text, with their prior knowledge, and with concepts found in the world beyond themselves” (p. 48).

Technological talespinning : new media and higher education in the USA : an examination of technological determinism

This thesis examines the theory of technological determinism, which espouses the view that technological change drives social change, through an analysis of the impact of new media on higher education models in the United States of America. In so doing, it explores the impacts of new media technologies on higher education, in particular, and society in general. The thesis reviews the theoretical shape of the discourse surrounding new media technologies before narrowing in on utopian claims about the impact of new media technologies on education. It tests these claims through a specific case study of higher education in the USA. The study investigates whether 'new' media technologies (eg the Internet) are resulting in new forms of higher education in the USA and whether the blurring of information and entertainment technologies has caused a similar blurring in education and entertainment providers. It uses primary data gathered by the author in a series of interviews with key education, industry and media representatives in North America in 1997. Chapter 2 looks at the literature and history surrounding several topics central to the thesis - the discourses of technological determinism, the history of technology use and adoption in education, and impacts of new media technologies on education. Chapter 3 presents the findings of the American case study on the relationship between media and higher education and Chapter 4 concludes and synthesises the investigation.

Sing & Grow : the co-existence of evaluation research and clinical practice in an early intervention music therapy project

Sing & Grow is a short term early intervention music therapy program for at risk families. Sing & Grow uses music to strengthen parent-child relationships by increasing positive parent-child interactions, assisting parents to bond with their children, and extending the repertoire of parents’ skills in relating to their child through interactive . Both the Australian and New Zealand governments are looking for evidence based research to highlight the effectiveness of funded programs in early childhood. As a government funded program, independent evaluation is a requirement of the delivery of the service. This paper explains the process involved in setting up and managing this large scale evaluation from engaging the evaluators and designing the project, to the data gathering stage. It describes the various challenges encountered and concludes that a highly collaborative and communicative partnership bet en researchers and clinicians is essential to ensure data can be gathered with minimal disturbance to clinical music therapy practice.

Critical attributes of public-private partnerships : a case study in vocational education

The term ‘partnership’ is increasingly used by governments, industry, community organisations and schools in supporting their daily businesses. Similar to the terms ‘ICT’ and ‘learning’, ‘partnerships’ are now ubiquitous in policy discourse. Yet, the term remains ill-defined and ambiguous. This study reviews and reflects on a government led industry-school partnership initiative in the state of Queensland, Australia, to understand how the term is used in this initiative. Given the frequent use of Public Private Partnership (PPP) language, PPP was used as a framework to review this initiative. The methodology of this qualitative case study involved consultations with stakeholders and an analysis of Gateway schools documents, policy documents, and literature. The review suggests that despite the use of terminology akin to PPP projects in Gateway school and policy documents, the implicit suggestion that this initiative is a public-private partnership is untenable. The majority of principles shaping a PPP have not been considered to a significant extent in the Gateway project. Although the review recognises the legitimate and sincere purpose of the Gateway schools initiative, the adoption of a PPP framework during the design, monitoring, or evaluation stages could have strengthened the initiative in terms of outcomes, benefits, and sustainability.

Evaluation of a community-based intervention to promote physical activity in youth : lessons from active winners

Purpose To test the effects of a community-based physical activity intervention designed to increase physical activity and to conduct an extensive process evaluation of the intervention. Design Quasi-experimental. Setting Two rural communities in South Carolina. One community received the intervention, and the other served as the comparison. Subjects Public school students who were in fifth grade at the start of the study (558 at baseline) were eligible to participate. A total of 436 students participated over the course of the study. Intervention The intervention included after-school and summer physical activity programs and home, school, and community components designed to increase physical activity in youth. The intervention took place over an 18-month period. Measures. Students reported after-school physical activity at three data collection points (prior to, during, and following the intervention) using the Previous Day Physical Activity Recall (PDPAR). They also completed a questionnaire designed to measure hypothesized psychosocial and environmental determinants of physical activity behavior The process evaluation used meeting records, documentation of program activities, interviews, focus groups, and heart rate monitoring to evaluate the planning and implementation of the intervention. Results There were no significant differences in the physical activity variables and few significant differences in the psychosocial variables between the intervention and comparison groups. The process evaluation indicated that the after-school and summer physical activity component of the intervention was implemented as planned, but because of resource and time limitations, the home, school, and community components were not implemented as planned. Conclusions The intervention did not have a significant effect on physical activity in the target population of children in the intervention community. This outcome is similar to that reported in other studies of community-based physical activity intervention.

Changes in teachers’ behaviour in secondary science education : implementing a standards-referenced national curriculum

There is substantial attention worldwide to the quality of secondary school teaching in STEM in Education. This paper reports on the use of Outcome Mapping (OM) as an approach to guide and monitor change in teacher practice and a visual tool, shaped as a Star, to benchmark and monitor this behaviour. OM and the visual tool were employed to guide and document three secondary teachers’ behaviour as they planned, implemented and assessed a science unit in the new Australian standards-referenced curriculum. Five key outcome markers in the teachers’ behaviour were identified together with progress markers — cumulative qualitative indicators — leading to these outcomes. The use of a Star to benchmark and track teachers’ behaviours was particularly useful because it showed teacher behaviour on multiple dimensions simultaneously at various points in time. It also highlighted priorities in need of further attention and provided a pathway to achievement. Hence, OM and the Star representation provide both theoretical and pragmatic approaches to enhancing quality in STEM teaching.

Taking an evaluative stance to decision-making about professional development options in early childhood education and care

This article builds on our ongoing work in conceptualising an ‘evaluative stance’ framework to assist in understanding how leaders in the field of early childhood education and care (ECEC) make decisions about the selection of professional development options for themselves and their staff. It introduces the notion that evaluative mindsets can be considered in terms of attitudes towards decision-making that are based on personal epistemologies. Drawing on data from semi-structured interviews, it explores the mindsets of six experienced leaders in two long-established ECEC organisations in Australia with respect to their decision-making about professional development. The article concludes with a consideration of the potential utility of the framework and the coding template used in this exploratory study.