An evaluation framework for sustaining the impact of educational development

Notwithstanding significant efforts by international aid agencies, aid ineffectiveness became apparent in 1990s as the impact of continued development intervention did not endure the expected outcomes. Conventional monitoring and evaluation by those agencies is critiqued for focusing on measuring project outcomes and giving little attention to aspects of sustainability. As a result, devising a rigorous evaluation framework for educational development has been sought in light of recent paradigm shifts in international development. This paper reports on a case study of an Egyptian educational development project highlighting the importance of transforming the evaluation procedures to process evaluation so as to enhance project impact and longevity. This requires building evaluation capacity of the aid recipient country.

Student engagement with an ePortfolio : a case study of pre-service education students

The emergence of ePortfolios is relatively recent in the university sector as a way to engage students in their learning and assessment, and to produce records of their accomplishments. An ePortfolio is an online tool that students can utilise to record, catalogue, retrieve and present reflections and artefacts that support and demonstrate the development of graduate students’ capabilities and professional standards across university courses. The ePortfolio is therefore considered as both process and product. Although ePortfolios show promise as a useful tool and their uptake has grown, they are not yet a mainstream higher education technology. To date, the emphasis has been on investigating their potential to support the multiple purposes of learning, assessment and employability, but less is known about whether and how students engage with ePortfolios in the university setting. This thesis investigates student engagement with an ePortfolio in one university. As the educational designer for the ePortfolio project at the University, I was uniquely positioned as a researching professional to undertake an inquiry into whether students were engaging with the ePortfolio. The participants in this study were a cohort (defined by enrolment in a unit of study) of second and third year education students (n=105) enrolled in a four year Bachelor of Education degree. The students were introduced to the ePortfolio in an introductory lecture and a hands-on workshop in a computer laboratory. They were subsequently required to complete a compulsory assessment task – a critical reflection - using the ePortfolio. Following that, engagement with the ePortfolio was voluntary. A single case study approach arising from an interpretivist paradigm directed the methodological approach and research design for this study. The study investigated the participants’ own accounts of their experiences with the ePortfolio, including how and when they engaged with the ePortfolio and the factors that impacted on their engagement. Data collection methods consisted of an attitude survey, student interviews, document collection, a researcher reflective journal and researcher observations. The findings of the study show that, while the students were encouraged to use the ePortfolio as a learning and employability tool, most students ultimately chose to disengage after completing the assessment task. Only six of the forty-five students (13%) who completed the research survey had used the ePortfolio in a sustained manner. The data obtained from the students during this research has provided insight into reasons why they disengaged from the ePortfolio. The findings add to the understandings and descriptions of student engagement with technology, and more broadly, advance the understanding of ePortfolios. These findings also contribute to the interdisciplinary field of technology implementation. There are three key outcomes from this study, a model of student engagement with technology, a set of criteria for the design of an ePortfolio, and a set of recommendations for effective practice for those implementing ePortfolios. The first, the Model of Student Engagement with Technology (MSET) (Version 2) explored student engagement with technology by highlighting key engagement decision points for students The model was initially conceptualised by building on work of previous research (Version 1), however, following data analysis a new model emerged, MSET (Version 2). The engagement decision points were identified as: • Prior Knowledge and Experience, leading to imagined usefulness and imagined ease of use; • Initial Supported Engagement, leading to supported experience of usefulness and supported ease of use; • Initial Independent Engagement, leading to actual experience of independent usefulness and actual ease of use; and • Ongoing Independent Engagement, leading to ongoing experience of usefulness and ongoing ease of use. The Model of Student Engagement with Technology (MSET) goes beyond numerical figures of usage to demonstrate student engagement with an ePortfolio. The explanatory power of the model is based on the identification of the types of decisions that students make and when they make them during the engagement process. This model presents a greater depth of understanding student engagement than was previously available and has implications for the direction and timing of future implementation, and academic and student development activities. The second key outcome from this study is a set of criteria for the re-conceptualisation of the University ePortfolio. The knowledge gained from this research has resulted in a new set of design criteria that focus on the student actions of writing reflections and adding artefacts. The process of using the ePortfolio is reconceptualised in terms of privileging student learning over administrative compliance. The focus of the ePortfolio is that the writing of critical reflections is the key function, not the selection of capabilities. The third key outcome from this research consists of five recommendations for university practice that have arisen from this study. They are that, sustainable implementation is more often achieved through small steps building on one another; that a clear definition of the purpose of an ePortfolio is crucial for students and staff; that ePortfolio pedagogy should be the driving force not the technology; that the merit of the ePortfolio is fostered in students and staff; and finally, that supporting delayed task performance is crucial. Students do not adopt an ePortfolio just because it is provided. While students must accept responsibility for their own engagement with the ePortfolio, the institution has to accept responsibility for providing the environment, and technical and pedagogical support to foster engagement. Ultimately, an ePortfolio should be considered as a joint venture between student and institution where strong returns on investment can be realised by both. It is acknowledged that the current implementation strategies for the ePortfolio are just the beginning of a much longer process. The real rewards for students, academics and the university lie in the future.

Exporting educational services : an Australian university’s experience in delivering offshore project management programs

Increasingly, Australian universities are facing the challenges of global education. While overseas students studying in Australia provide the primary source of export earnings for educational institutions, a number of institutions, including QUT, are also involved in international trade in educational services by providing services offshore. This paper discusses driving forces behind moves by Australian universities to enter the international education market. It then briefly describes Queensland University of Technology’s internationalisation strategy. The paper concludes with a case study describing how the School of Construction Management has pioneered the development of offshore courses at QUT. The introduction of the Master of Project Management and Graduate Diploma of Project Management programs in Singapore in November 1993 represented QUT’s first experience in this area. With the experience of 18 months of operation, the School now has the opportunity to reflect on the outcomes of this venture and consider future options.

Generalising across borders : policy and the limits of educational science.

AERA Distinguished Lecture, Annual Meetings of the American Educational Research Association, New Orleans, 8 April 2011. How well does educational policy, innovation and science cross borders? What are the parameters of a generalisable cultural science of education? What constitutes a principled policy 'borrowing'?

Design for inspiration : children, personal connections and educational technology

The project is working towards building an understanding of the personal interests and experiences of children with the aim of designing appropriate, usable and, most importantly, inspirational educational technology. kidprobe, an adaptation of the technology probe concept, has been used as a lightweight method of gaining contextual information about children's interactions with 'fun' technology. kidprobe has produced design inspiration which focuses primarily on the social and emotional connections children made. The use of kidprobe has generated some important ideas for improving the use of probes with children. It is an important first step in understanding how to effectively adapt probing techniques to inspire the design of technology for children.

Assurance of learning : the role of work integrated learning and industry partners

In the partnering with students and industry it is important for universities to recognize and value the nature of knowledge and learning that emanates from work integrated learning experiences is different to formal university based learning. Learning is not a by-product of work rather learning is fundamental to engaging in work practice. Work integrated learning experiences provide unique opportunities for students to integrate theory and practice through the solving of real world problems. This paper reports findings to date of a project that sought to identify key issues and practices faced by academics, industry partners and students engaged in the provision and experience of work integrated learning within an undergraduate creative industries program at a major metropolitan university. In this paper, those findings are focused on some of the particular qualities and issues related to the assessment of learning at and through the work integrated experience. The findings suggest that the assessment strategies needed to better value the knowledges and practices of the Creative Industries. The paper also makes recommendations about how industry partners might best contribute to the assessment of students’ developing capabilities and to continuous reflection on courses and the assurance of learning agenda.

Educational policy for citizenship in the early years in Australia

This paper explores how young children are constructed in educational policy for citizenship in Australia, investigating tensions between early childhood educational discourses that construct young children as active citizens and the broader discourses of citizenship in Australian educational policy. There is a widespread discourse within early childhood education that regards young children as citizens and democratic participants in their own lives. This view is a reflection of the oft cited Article 12 in the UN Convention on the Rights of the Child (UNCRC 1989). However, educational policy and curriculum for citizenship in Australia, by and large, adheres to age and stage understandings of children, implicitly deeming young children unable to conceptualise abstract ideas of what it means to ‘be a good citizen’. This paper is located in the borders and intersections between discourses of early childhood education, young children as active participants in their own lives and what it means to be an active citizen in Australia. We are concerned with the interweaving of these ideas and how they are played out in educational policy making. This is an important perspective to take for governing and policy making are exercises in harnessing existing ideas and discourses, thereby rendering strategies and tactics for managing populations thinkable and sayable (Rose 1999). The ‘views from the margins’ (Burman 2008, p. 7) can provide alternative perspectives on policymaking, illuminating discursive tactics and strategies.

Assessment reform and educational change in Australia

This chapter addresses the changing climate of assessment policy and practice in Australia in response to global trends in education and the mounting accountability demands of standards-driven reform. Queensland, a State of Australia, has a tradition of respecting and trusting teacher judgment through the practice of, and policy commitment to, externally moderated school-based assessment. This chapter outlines the global trends in curriculum and assessment reform, and then analyzes the impact of international comparisons on national policy. The creation of the Australian Curriculum, Assessment and Reporting Authority (ACARA) together with the intent of establishing a standards-referenced framework raises tensions and challenges for teachers’ practice. The argument for sustaining confidence in teacher-based assessment is developed with reference to research evidence pertaining to the use of more authentic assessments and moderation practices for the purposes of improving learning, equity and accountability. Evidence is drawn from local studies of teacher judgment practice and used to demonstrate these developments and in so doing illuminate the complex issues of engaging the demands of policy while sustaining confidence in teacher assessment.

International regional planning educational experience : learnings from Malaysia, Korea and Turkey study trips

In an era of rapidly changing economic, social and environmental conditions, urban and regional planning education must be resilient, innovative and able to deal with the complex political and socio-economic fabric of post-modern cities. As a consequence, urban and regional planning education plays a fundamental role in educating and forming planning practitioners that will be able to tackle such complexity. However, not many tertiary education institutions provide a trans-cultural engagement opportunity for students, where the need to internationalise planning education has been widely recognised worldwide. The aim of this paper is to communicate the findings of three overseas study trips (Kuala Lumpur-Malaysia, Daejeon-Korea, Istanbul and Gallipoli-Turkey) that students of Queensland University of Technology are taken to where these study trips trailed the provision of an innovative tertiary education experience of teaching regional planning in an international context. The findings of the pedagogic analyses of the study reveal that the exposure of students to different planning processes and practices give them a new outlook on what they knew from their own country and provide them with useful insights on international planning issues and cultural differences and barriers.

An examination of the relationship between emotional intelligence, leadership style and perceived leadership outcomes in Australian educational institutions

In the field of leadership studies transformational leadership theory (e.g., Bass, 1985; Avolio, Bass, & Jung, 1995) has received much attention from researchers in recent years (Hughes, Ginnet, & Curphy, 2009; Hunt, 1999). Many previous studies have found that transformational leadership is related to positive outcomes such as the satisfaction, motivation and performance of followers in organisations (Judge & Piccolo, 2004; Lowe, Kroeck, & Sivasubramaniam, 1996), including in educational institutions (Chin, 2007; Leithwoood & Jantzi, 2005). Hence, it is important to explore constructs that may predict leadership style in order to identify potential transformational leaders in leadership assessment and selection procedures. Several researchers have proposed that emotional intelligence (EI) is one construct that may account for hitherto unexplained variance in transformational leadership (Mayer, 2001; Watkin, 2000). Different models of EI exist (e.g., Goleman, 1995, 2001; Bar-On, 1997; Mayer & Salovey, 1997) but momentum is growing for the Mayer and Salovey (1997) model to be considered the most useful (Ashkanasy & Daus, 2005; Daus & Ashkanasy, 2005). Studies in non-educational settings claim to have found that EI is a useful predictor of leadership style and leader effectiveness (Harms & Crede, 2010; Mills, 2009) but there is a paucity of studies which have examined the Mayer and Salovey (1997) model of EI in educational settings. Furthermore, other predictor variables have rarely been controlled in previous studies and only self-ratings of leadership behaviours, rather than multiple ratings, have usually been obtained. Therefore, more research is required in educational settings to answer the question: to what extent is the Mayer and Salovey (1997) model of EI a useful predictor of leadership style and leadership outcomes? This project, set in Australian educational institutions, was designed to move research in the field forward by: using valid and reliable instruments, controlling for other predictors, obtaining an adequately sized sample of real leaders as participants and obtaining multiple ratings of leadership behaviours. Other variables commonly used to predict leadership behaviours (personality factors and general mental ability) were assessed and controlled in the project. Additionally, integrity was included as another potential predictor of leadership behaviours as it has previously been found to be related to transformational leadership (Parry & Proctor-Thomson, 2002). Multiple ratings of leadership behaviours were obtained from each leader and their supervisors, peers and followers. The following valid and reliable psychological tests were used to operationalise the variables of interest: leadership styles and perceived leadership outcomes (Multifactor Leadership Questionnaire, Avolio et al., 1995), EI (Mayer–Salovey–Caruso Emotional Intelligence Test, Mayer, Salovey, & Caruso, 2002), personality factors (The Big Five Inventory, John, Donahue, & Kentle, 1991), general mental ability (Wonderlic Personnel Test-Quicktest, Wonderlic, 2003) and integrity (Integrity Express, Vangent, 2002). A Pilot Study (N = 25 leaders and 75 raters) made a preliminary examination of the relationship between the variables included in the project. Total EI, the experiential area, and the managing emotions and perceiving emotions branches of EI, were found to be related to transformational leadership which indicated that further research was warranted. In the Main Study, 144 leaders and 432 raters were recruited as participants to assess the discriminant validity of the instruments and examine the usefulness of EI as a predictor of leadership style and perceived leadership outcomes. Scores for each leadership scale across the four rating levels (leaders, supervisors, peers and followers) were aggregated with the exception of the management-by-exception active scale of transactional leadership which had an inadequate level of interrater agreement. In the descriptive and measurement component of the Main Study, the instruments were found to demonstrate adequate discriminant validity. The impact of role and gender on leadership style and EI were also examined, and females were found to be more transformational as leaders than males. Females also engaged in more contingent reward (transactional leadership) behaviours than males, whilst males engaged in more passive/avoidant leadership behaviours than females. In the inferential component of the Main Study, multiple regression procedures were used to examine the usefulness of EI as a predictor of leadership style and perceived leadership outcomes. None of the EI branches were found to be related to transformational leadership or the perceived leadership outcomes variables included in the study. Openness, emotional stability (the inverse of neuroticism) and general mental ability (inversely) each predicted a small amount of variance in transformational leadership. Passive/avoidant leadership was inversely predicted by the understanding emotions branch of EI. Overall, EI was not found to be a useful predictor of leadership style and leadership outcomes in the Main Study of this project. Implications for researchers and human resource practitioners are discussed.

The power of educational podcasting : using short-format podcasts to reinforce tertiary student learning experiences in science

In 2007 I introduced short-format educational podcast resources that reinforced conceptual teaching and learning in an interdisciplinary tertiary science study area (biochemistry). This study aims to determine student attitudes to the perceived usefulness and benefit of short-format educational podcasts, and presents the findings (qualitative and quantitative) from surveys obtained from three offerings of the science teaching unit (2007, 2008 and 2009). Podcasts were recorded (MP3 audio files) separately from the instructive lecture sessions, and subsequent to the weekly lecture, short-format podcasts summarising the key learning objectives were integrated within the resources presented through the students learning management system (Blackboard). The vast majority (>88%) of students utilised the podcast resources, indicating a high level of acceptance and uptake for this portable educational technology. The respondents reported that podcasts focused their attention to core learning concepts and supported their understanding and learning of the lecture material. Furthermore, the data showed that respondents agreed strongly that podcasts assisted with study and revision for examinations and, somewhat surprisingly, there was a perception that podcasts positively impacted on examination performance. Overall, student users perceived that podcasting is as an effective and valuable educational tool that offers convenience and flexibility for their learning and understanding of a tertiary science study area, such as biochemistry.

Inciting the social imagination to realising the dream : An unfolding story of reform in a low-socioeconomic school

This paper presentation addresses design-based research that became a catalyst for social change among a disadvantaged school community. The aim of the longitudinal research was to protoype an evidence-based model for whole school digital and print literacy pedagogy renewal among students from low socioeconomic, Indigenous, and migrant backgrounds. Applying Anthony Gidden’s principle of the “duality of structure”, the paper presentation interprets how the collective agency of researchers and the school community began to transform the structural properties of the institution in a two-way dynamism, so that the structural properties of the school were not outside of individual action, but were implicated in its reproduction and transformation.

Supporting refugee students in schools : what constitutes inclusive education?

The worldwide rise in numbers of refugees and asylum seekers suggests the need to examine the practices of those institutions charged with their resettlement in host countries. In this paper we investigate the role of one important institution – schooling – and its contribution to the successful resettlement of refugee children. We begin with an examination of forced migration and its links with globalisation, and the barriers to inclusion confronting refugees. A discussion of the educational challenges confronting individual refugee youth and schools is followed by case studies of four schools and the approaches they had developed to meet the needs of young people from a refugee background. Using our findings and other research, we outline a model of good practice in refugee education. We conclude by discussing how educational institutions might play a more active role in facilitating transitions to citizenship for refugee youth through an inclusive approach.

Aspiration levels and educational choices: An experimental study

The explanation of social inequalities in education is still a debated issue in economics. Recent empirical studies tend to downplay the potential role of credit constraint. This article tests a different potential explanation of social inequalities in education, specifically that social differences in aspiration level result in different educational choices. Having existed for a long time in the sociology of education, this explanation can be justified if aspiration levels are seen as reference points in a prospect theory framework. In order to test this explanation, this article applies the method of experimental economics to the issue of education choice and behaviour. One hundred and twenty-nine individuals participated in an experiment in which they had to perform a task over 15 stages grouped in three blocks or levels. In order to continue through the experiment, a minimum level of success was required at the end of each level. Rewards were dependent on the final level successfully reached. At the end of each level, participants could either choose to stop and take their reward or to pay a cost to continue further in order to possibly receive higher rewards. To test the impact of aspiration levels, outcomes were either presented as gains or losses relative to an initial sum. In accordance with the theoretical predictions, participants in the loss framing group choose to go further in the experiment. There was also a significant and interesting gender effect in the loss framing treatment, such that males performed better and reached higher levels.