Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Enabling access to and re-use of publicly funded research data as Open Educational Resources : a strategy for overcoming the legal barriers to data access and re-use

Open Educational Resources (OER) are teaching, learning and research materials that have been released under an open licence that permits online access and re-use by others. The 2012 Paris OER Declaration encourages the open licensing of educational materials produced with public funds. Digital data and data sets produced as a result of scientific and non-scientific research are an increasingly important category of educational materials. This paper discusses the legal challenges presented when publicly funded research data is made available as OER, arising from intellectual property rights, confidentiality and information privacy laws, and the lack of a legal duty to ensure data quality. If these legal challenges are not understood, addressed and effectively managed, they may impede and restrict access to and re-use of research data. This paper identifies some of the legal challenges that need to be addressed and describes 10 proposed best practices which are recommended for adoption to so that publicly funded research data can be made available for access and re-use as OER.

Reflection in Integrated Problem Based Learning : 20 years of continuing application to the teaching of architecture

In 1984 the School of Architecture and Built Environment within the University of Newcastle, Australia introduced an integrated program based on real design projects and using Integrated Problem Based Learning (IPBL) as the teaching method. Since 1984 there have been multiple changes arising from the expectations of the architectural fraternity, enrolling students, lecturers, available facilities, accreditation authorities and many others. These challenges have been successfully accommodated whilst maintaining the original purposes and principles of IPBL. The Architecture program has a combined two-degree structure consisting of a first degree, Bachelor of Science (Architecture), followed by a second degree, Bachelor of Architecture. The program is designed to simulate the problem-solving situations that face a working architect in every day practice. This paper will present the degree structure where each student is enrolled in a single course per semester incorporating design integration and study areas in design studies, professional studies, historical studies, technical studies, environmental studies and communication skills. Each year the design problems increase in complexity and duration set around an annual theme. With 20 years of successful delivery of any program there are highlights and challenges along the way and this paper will discuss some of the successes and barriers experienced within the School of Architecture and Built Environment in delivering IPBL. In addition, the reflective process investigates the currency of IPBL as an appropriate vehicle for delivering the curriculum in 2004 and any additional administrative or staff considerations required to enhance the continuing application of IPBL.

Self-regulation and mastery motivation in individuals with developmental disabilities : barriers, supports, and strategies

The development of the capacity for self-regulation represents an important achievement of childhood and is associated with social, behavioral, and academic competence (Bronson, 2001; Cleary & Zimmerman, 2004). Self-regulation evolves as individuals mature, with its final form integrating emotional, cognitive, and behavioral elements working together to achieve self-selected goals. This evolution is closely intertwined with the innate press to master the environment, labeled mastery motivation (Morgan, Harmon, & Maslin-Cole, 1990), as competence is the aim that underpins mastery motivation.

Widening participation in university learning

This paper reports how one Australian university and the Queensland Department of Education and Training (DET) are working together to increase the number of school students from low socio-economic backgrounds enrolling in undergraduate university degrees. This innovative program involves university lecturers and school teachers working together in the delivery and assessment of four Bachelor of Education units (or subjects) to a cohort of Year eleven and twelve students at a secondary school. Focus group interviews collected data from 26 students, 7 parents, 4 school and 3 university staff to assess the effectiveness of the program. All stakeholders viewed the program as a highly valuable opportunity to experience university learning with 31 high school graduating students being made offers to enter full-time university in the 2010 and 2011. This positive result has particular significance in the current drive in Australia and elsewhere to increase the participation in higher education of young people from under-represented groups.

Work integrated research higher degree studies : experiences, benefits, barriers and coping strategies

Attributed to the changing social, political and economic landscape of the ‘knowledge economy’, Australian universities are under pressure to produce researchers that have a variety of skills which meet the demands of an increasingly diverse job market. As a consequence, the Australian PhD now includes a range of doctoral degrees. This paper reports on the experiences of two PhD students engaged in an informally managed research higher degree program described in this paper as a Work Integrated Research Higher Degree (WIRHD). Their learning process shares the attributes from both the traditional PhD program and professional doctorates. However, because of the blended nature of the learning contexts, what students need to manage within the WIRHD is much more complicated than the established RHD programs. An exploratory case study approach exploring experiences, benefits, barriers and coping strategies was conducted with the view to develop a preliminary integrative framework that attempts to explain the various contexts that influence the learning experience of WIRHD candidates. The paper concludes with some recommended strategies for helping WIRHD candidates to manage the challenges associated with their learning process.

Mid-Career professionals in STEM transiting to school teaching : barriers at the border

In many countries there is a shortage of quality teachers in areas of science, technology, engineering and mathematics (STEM). Additional to the low levels of recruitment is an extraordinary high attrition rate with some 50% of beginning teachers leaving the profession within five years. One solution implemented in several countries has been to encourage mid-career professionals in the area of STEM to become school teachers. These professionals are said to bring to teaching enthusiasm, knowledge and a passion for their subject which will impact engagement and learning by students. However, these career-changers have constructed professional identities and are accustomed to working within a culture of collaboration and inquiry. In contrast, school cultures are quite different and often teaching is a lonely solitary affair with little opportunity for collegial relationships aimed at knowledge building in the context of teaching. Crossing from a culture of STEM to a culture of schools and teaching can be challenging. This study was conducted with 13 teachers who were followed for three years. However, this paper reports on the experiences of one teacher with an engineering background crossing the boundaries from practising STEM to Teaching STEM.

Australian adolescents’ perceived school-based barriers and facilitators to engagement in physical activity

OBJECTIVE: School-aged youth spend a significant amount of time either in transit to and from school, or within school settings performing a range of varying learning-based activities. Adolescent physical activity has also been shown to increase the likelihood of maintaining physical activity throughout adulthood. The purpose of this study is to investigate adolescents’ perceived school-based barriers and facilitators to engagement in physical activity. METHODS: One-hundred and twenty four participants (38 males and 86 females) were recruited from two non-denominational same-sex private schools, in Brisbane, Australia. The mean age and standard deviation (SD) was 13.83 (0.56) and 14.40 (2.33) for males and females respectively. Participants responded to a series questions regarding perceived barriers and facilitators to engagement in physical activity. Quantitative data was analysed using descriptive statistics and frequency distributions, and qualitative data with thematic analysis. RESULTS: A total of 121 (97.6%) participants had complete data sets and were included in the analysis. School timetable (44.6%), homework (81.8%), and assessment (81.0%) were identified as the most prominent perceived factors, increasing the difficulty of physical activity engagement. Physical Education classes (71.9%) and school sport programs (80.2%) were identified as the most prominent perceived factors that facilitate engagement in physical activity. There was no significant gender effect. CONCLUSIONS: Each of the identified factors perceived by adolescent's as either barriers or facilitators to engagement in physical activity may be addressed by administrators at a school and government policy level. These may include strategies such as; increasing the assigned hours to physical education classes, providing additional extra-curricular sporting opportunities, and reviewing the time allocated to homework and assessment items. This may provide a simpler, low-cost solution to increasing youth physical activity, as opposed to contemporary higher-cost strategies utilising increased staff commitment, mass media, provision of equipment and counsellors and other health professionals.

Perceptions and expectations for learning and development for older workers within Queensland local government councils : a case study

Current literature warns organisations about a global ageing phenomenon. Workplace ageing is causing a diminishing work pool which has consequences for a sustainable workforce in the future. This phenomenon continues to impact on local government councils in Australia. Australia has one of the world’s most rapidly ageing populations, and there is evidence that Australian local government councils are already resulting in an unsustainable workforce. Consequently, this research program investigated the role of older workers in the Queensland local government workplace in enabling them to extend their working lives towards transitional employment and a sustainable workforce in the future. Transitional Employment is intended as a strategy for enabling individuals to have greater control over their employment options and their employability during the period leading to their final exit from the workforce. There was no evidence of corporate support for older workers in Queensland local government councils other than tokenistic government campaigns encouraging organisations to "better value their older workers". (Queensland Government, 2007d, p.6). TE is investigated as a possible intervention for older workers in the future. The international and national literature review reflected a range of matters impacting on current older workers in the workforce and barriers preventing them from accessing services towards extending their employment beyond the traditional retirement age (60 years) as defined by the Australian Government; an age when individuals can access their superannuation. Learning and development services were identified as one of those barriers. There was little evidence of investment in or consistent approaches to supporting older workers by organisations. Learning and development services appeared at best to be ad hoc, reactive to corporate productivity and outputs with little recognition of the ageing phenomenon (OECD, 2006, p.23) and looming skills and labour shortages (ALGA, 2006, p. 19). Themes from the literature review led to the establishment of three key research questions: 1. What are the current local government workforce issues impacting on skills and labour retention? 2. What are perceptions about the current workplace environment? And, 3. What are the expectations about learning and development towards extending employability of older workers within the local government sector? The research questions were explored by utilising three qualitative empirical studies, using some numerical data for reporting and comparative analysis. Empirical Study One investigated common themes for accessing transitional employment and comprised two phases. A literature review and Study One data analysis enabled the construction of an initial Transitional Employment Model which includes most frequent themes. Empirical Study Two comprised focus groups to further consider those themes. This led to identification of issues impacting the most on access to learning and development by older workers and towards a revised TEM. Findings presented majority support for transitional employment as a strategy for supporting older workers to work beyond their traditional retirement age. Those findings are presented as significant issues impacting on access to transitional employment within the final 3-dimensionsal TEM. The model is intended as a guide for responding to an ageing workforce by local government councils in the future. This study argued for increased and improved corporate support, particularly for learning and development services for older workers. Such support will enable older workers to maintain their employability and extend their working lives; a sustainable workforce in the future.

Connected learning in the library as a product of hacking, making, social diversity and messiness

Learning is most effective when intrinsically motivated through personal interest, and situated in a supportive socio-cultural context. This paper reports on findings from a study that explored implications for design of interactive learning environments through 18 months of ethnographic observations of people’s interactions at “Hack The Evening” (HTE). HTE is a meetup group initiated at the State Library of Queensland in Brisbane, Australia, and dedicated to provide visitors with opportunities for connected learning in relation to hacking, making and do-it-yourself technology. The results provide insights into factors that contributed to HTE as a social, interactive and participatory environment for learning – knowledge is created and co-created through uncoordinated interactions among participants that come from a diversity of backgrounds, skills and areas of expertise. The insights also reveal challenges and barriers that the HTE group faced in regards to connected learning. Four dimensions of design opportunities are presented to overcome those challenges and barriers towards improving connected learning in library buildings and other free-choice learning environments that seek to embody a more interactive and participatory culture among their users. The insights are relevant for librarians as well as designers, managers and decision makers of other interactive and free-choice learning environments.

Learning without boundaries

Discipline boundaries of science and technology education are inevitable. Often, such barriers are an obstacle to industry-based learning leading to preventable complexities. Industry-based learning is a complex scenario, rather than conventional learning, leading to the study of liquid learning, which is a timely concept to investigate learning without boundaries. Liquid learning consists of accountability, expectations and driven by outcomes with different learning choices. Liquid learning is a significant phenomenon requiring awareness in the science and technology education. This paper aims to discuss some practical issues when designing industry-based learning without boundaries. A case study approach is reviewed and presented.

The QUT innovation space : a trans-disciplinary learning environment for entrepreneurship education : final report

In the current climate of global economic volatility, there are increasing calls for training in enterprising skills and entrepreneurship to underpin the systemic innovation required for even medium-term business sustainability. The skills long-recognised as the essential for entrepreneurship now appear on the list of employability skills demanded by industry. The QUT Innovation Space (QIS) was an experiment aimed at delivering entrepreneurship education (EE), as an extra-curricular platform across the university, to the undergraduate students of an Australian higher education institute. It was an ambitious project that built on overseas models of EE studied during an Australian Learning and Teaching Council (ALTC) Teaching Fellowship (Collet, 2011) and implemented those approaches across an institute. Such EE approaches have not been attempted in an Australian university. The project tested resonance not only with the student population, from the perspective of what worked and what didn’t work, but also with every level of university operations. Such information is needed to inform the development of EE in the Australian university landscape. The QIS comprised a physical co-working space, virtual sites (web, Twitter and Facebook) and a network of entrepreneurial mentors, colleagues, and students. All facets of the QIS enabled connection between like-minded individuals that underpins the momentum needed for a project of this nature. The QIS became an innovation community within QUT. This report serves two purposes. First, as an account of the QIS project and its evolution, the report serves to identify the student demand for skills and training as well as barriers and facilitators of the activities that promote EE in an Australian university context. Second, the report serves as a how-to manual, in the tradition of many tomes on EE, outlining the QIS activities that worked as well as those that failed. The activities represent one measure of QIS outcomes and are described herein to facilitate implementation in other institutes. The QIS initially aimed to adopt an incubation model for training in EE. The ‘learning by doing’ model for new venture creation is a highly successful and high profile training approach commonly found in overseas contexts. However, the greatest demand of the QUT student population was not for incubation and progression of a developed entrepreneurial intent, but rather for training that instilled enterprising skills in the individual. These two scenarios require different training approaches (Fayolle and Gailly, 2008). The activities of the QIS evolved to meet that student demand. In addressing enterprising skills, the QIS developed the antecedents of entrepreneurialism (i.e., entrepreneurial attitudes, motivation and behaviours) including high-level skills around risk-taking, effective communication, opportunity recognition and action-orientation. In focusing on the would-be entrepreneur and not on the (initial) idea per se, the QIS also fostered entrepreneurial outcomes that would never have gained entry to the rigid stage-gated incubation model proposed for the original QIS framework. Important lessons learned from the project for development of an innovation community include the need to: 1. Evaluate the context of the type of EE program to be delivered and the student demand for the skills training (as noted above). 2. Create a community that builds on three dimensions: a physical space, a virtual environment and a network of mentors and partners. 3. Supplement the community with external partnerships that aid in delivery of skills training materials. 4. Ensure discovery of the community through the use of external IT services to deliver advertising and networking outlets. 5. Manage unrealistic student expectations of billion dollar products. 6. Continuously renew and rebuild simple activities to maintain student engagement. 7. Accommodate the non-university end-user group within the community. 8. Recognise and address the skills bottlenecks that serve as barriers to concept progression; in this case, externally provided IT and programming skills. 9. Use available on-line and published resources rather than engage in constructing project-specific resources that quickly become obsolete. 10. Avoid perceptions of faculty ownership and operate in an increasingly competitive environment. 11. Recognise that the continuum between creativity/innovation and entrepreneurship is complex, non-linear and requires different training regimes during the different phases of the pipeline. One small entity, such as the QIS, cannot address them all. The QIS successfully designed, implemented and delivered activities that included events, workshops, seminars and services to QUT students in the extra-curricular space. That the QIS project can be considered successful derives directly from the outcomes. First, the QIS project changed the lives of emerging QUT student entrepreneurs. Also, the QIS activities developed enterprising skills in students who did not necessarily have a business proposition, at the time. Second, successful outcomes of the QIS project are evidenced as the embedding of most, perhaps all, of the QIS activities in a new Chancellery-sponsored initiative: the Leadership Development and Innovation Program hosted by QUT Student Support Services. During the course of the QIS project, the Brisbane-based innovation ecosystem underwent substantial change. From a dearth of opportunities for the entrepreneurially inclined, there is now a plethora of entities that cater for a diversity of innovation-related activities. While the QIS evolved with the landscape, the demand endpoint of the QIS activities still highlights a gap in the local and national innovation ecosystems. The freedom to experiment and to fail is not catered for by the many new entities seeking to build viable businesses on the back of the innovation push. The onus of teaching the enterprising skills, which are the employability skills now demanded by industry, remains the domain of the higher education sector.

Low-cost railway level crossings - breaking down the barriers

Level crossing risk continues to be a significant safety concern for the security of rail operations around the world. Over the last decade or so, a third of railway related fatalities occurred as a direct result of collisions between road and rail vehicles in Australia. Importantly, nearly half of these collisions occurred at railway level crossings with no active protection, such as flashing lights or boom barriers. Current practice is to upgrade level crossings that have no active protection. However, the total number of level crossings found across Australia exceed 23,500, and targeting the proportion of these that are considered high risk (e.g. public crossings with passive controls) would cost in excess of AU$3.25 billion based on equipment, installation and commissioning costs of warning devices that are currently type approved. Level crossing warning devices that are low-cost provide a potentially effective control for reducing risk; however, over the last decade, there have been significant barriers and legal issues in both Australia and the US that have foreshadowed their adoption. These devices are designed to have significantly lower lifecycle costs compared with traditional warning devices. They often make use of use of alternative technologies for train detection, wireless connectivity and solar energy supply. This paper describes the barriers that have been encountered for the adoption of these devices in Australia, including the challenges associated with: (1) determining requisite safety levels for such devices; (2) legal issues relating to duty of care obligations of railway operators; and (3) issues of Tort liability around the use of less than fail-safe equipment. This paper provides an overview of a comprehensive safety justification that was developed as part of a project funded by a collaborative rail research initiative established by the Australian government, and describes the conceptual framework and processes being used to justify its adoption. The paper provides a summary of key points from peer review and discusses prospective barriers that may need to be overcome for future adoption. A successful outcome from this process would result in the development of a guideline for decision-making, providing a precedence for adopting low-cost level crossing warning devices in other parts of the world. The framework described in this paper also provides relevance to the review and adoption of analogous technologies in rail and other safety critical industries.

Blended learning : meeting the needs of distance education students

This thesis involved research into the barriers and enablers that existed for a cohort of mature-aged education support students engaging with blended learning through distance education. The findings that emerged from this research indicated that a flexible model of blended learning is possible in this context. The findings shed light on the experiences of novice technology users' participation in blended learning. The study highlighted the significance of factors such as isolation, technology, communication, connectivity, prior learning, and the growth of self-efficacy that influenced learner engagement.