Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

There has been a greater focus on strengthening evaluation capacity building (ECB) within development organisations in recent years. This can be attributed in part to the growing appreciation of the value of participatory and collaborative forms of evaluation. Evaluation is increasingly seen as an ongoing learning process and an important means of strengthening capacity and improving organisational performance (Horton et al., 2003:7). While there are many benefits of using participatory methodologies in ECB projects, our experiences and a review of the literature in this area highlight the many challenges, issues and contradictions that can affect the success of such ECB efforts. We discuss these issues, drawing on our learnings from the ongoing participatory action research (PAR) project 'Assessing Communication for Social Change’ (AC4SC). This four year project, which began in 2007, is a collaboration between communication and development academics and evaluation specialists from two Australian universities and communication for development practitioners and monitoring and evaluation (M&E) staff in the NGO Equal Access Nepal (EAN). The aim is to develop, implement, and evaluate a participatory methodology for assessing the social change impacts of community radio programs produced by EAN. It builds on previous projects that used ethnographic action research (EAR) methodology (Tacchi et al., 2007).

Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

"In this chapter the authors present a critique of Participatory Evaluation as worked in development projects, in this case, in Nepal. The article works between established claims that Participatory Evaluation builds capacity at programmatic and organisational levels, and the specific experiences of these claims in the authors’ current work. They highlight the need to address key difficulties such as high turn-over of staff and resulting loss of capacity to engage in Participatory Evaluation, and the difficulty of communication between academic as compared with local practical wisdoms. A key issue is the challenge of addressing the inevitable issues of power inequities that such approaches encounter. While Participatory Evaluation has been around for some time, it has only enjoyed more widespread recognition of its value in comparatively recent times, with its uptake in international development environments. To this extent, the practice is still in its early stages of development, and Jo, June and Michael’s work contributes to strengthening and more comprehensively understanding it. With regard to the meta-theme of this publication, this chapter is an example of how context not only influences the methodology to be used and the praxis of how it is to be used, but contributes to early explication of the core nature of an emerging methodology."

The why dimension - opening frontiers for digital learning

Digital learning has come a long way from the days of simple 'if-then' queries. It is now enabled by countless innovations that support knowledge sharing, openness, flexibility, and independent inquiry. Set against an evolutionary context this study investigated innovations that directly support human inquiry. Specifically, it identified five activities that together are defined as the 'why dimension' – asking, learning, understanding, knowing, and explaining why. Findings highlight deficiencies in mainstream search-based approaches to inquiry, which tend to privilege the retrieval of information as distinct from explanation. Instrumental to sense-making, the 'why dimension' provides a conceptual framework for development of 'sense-making technologies'.

An exploration into retrofitting cities with natural design elements - learning from the city of Berlin, Germany

Natural design features in the built environment or biophilic elements are emerging as a potential response to the challenges of climate change, urbanisation and population pressures which have invited issues such as rising urban heat island effect, rising pollution, increased congestion, among others. This concept of living cities was made popular by Professor Tim Beatley in his book titled ‘Biophilic Urbanism’. Evidence of biophilic urbanism can be seen in some cities from around the globe since decoupling environmental pressures from future development is a priority on many agendas. Berlin is an example of a modern economy that has adopted an ecological sustainable development approach to reduce environmental degradation while driving innovation and employment.

Developing learning cohorts for postgraduate research degrees

Development of researchers through higher degree research studies is a high priority in most universities. Yet, research about supervision as pedagogy and models of supervision is only recently gained increasing attention. Charged with producing good researchers within very limited resources, academics are constantly looking for more efficient models of supervision for higher degree research students. A cohort model of supervision promises several efficiencies, but we argue that its success lies importantly on how well the cohort is developed specifically for higher degree research studies. We drew on a growing body of literature on higher degree research supervision to design, implement and evaluate our approach to developing a cohort of seven students enrolled in the Master of Education (Research) degree. Our approach included four provisions: initial residential workshop, development of a learning community, nourishing scholarship, and ongoing learning opportunities. The four provisions resulted in gradually developing an environment and culture that students found very supportive and nurturing. This paper is based on the findings from data collected from student evaluations in the first year of studies, feedback from the cohort’s sponsor, and our reflective notes. The evaluation substantiated the value in investing time and resources for purposely developing a cohort for higher degree research studies. Whether the cohorts are sponsored or not, universities will still need to invest time and resources for cohort development if a cohort model is intended to gain wider efficiencies in supervision of higher degree research students.

Innovations in dietetic training in foodservice management : can early exposure to practice enhance final year competency development?

Student engagement in the delivery of theoretical course materials is a current challenge in the tertiary sector including for dietetic training. In 2011 with the creation of a new nutritionist position for Queensland Meals on Wheels (QMOW), a service learning approach to support this organisation was used with third year dietetic students undertaking two days of structured activities at various QMOW sites in South East Queensland, aligned with coursework in Foodservice Management (FSM). This cohort of students was then followed in their final year post successful completion of five weeks professional practice in FSM to see if this experience supported readiness for placement and competency development. Evaluation was undertaken of eligible students (n = 50) via paper based survey (response rate 94%) with all participating in targeted focus groups. Findings showed that students acknowledged the QMOW experience (on reflection 14 months later) providing opportunity for participation and/or observation in 5 of 12 FSM areas taught in third year, including food safety, meal production, assembly, delivery and dishwashing. Over half the students identified good exposure to FSM competency areas during the QMOW experience, with 83% satisfied with their competency exposure and subsequent practice during final year placements. A consistent theme emerged from focus groups supporting inclusion of practical opportunities with the theoretical component of the FSM subject to highlight relevance to learning. These findings highlight the importance of such teaching initiatives to met student learning preferences, linking theory with practice and supporting competency development in the final year of training programs.

Applying threshold learning theory to teach sustainable business practice in post-graduate engineering education

This paper presents the results of a qualitative action-research inquiry into how a highly diverse cohort of post-graduate students could develop significant capacity in sustainable development within a single unit (course), in this case a compulsory component of four built environment masters programs. The method comprised applying threshold learning theory within the technical discipline of sustainable development, to transform student understanding of sustainable business practice in the built environment. This involved identifying a number of key threshold concepts, which once learned would provide a pathway to having a transformational learning experience. Curriculum was then revised, to focus on stepping through these targeted concepts using a scaffolded, problem-based-learning approach. Challenges included a large class size of 120 students, a majority of international students, and a wide span of disciplinary backgrounds across the spectrum of built environment professionals. Five ‘key’ threshold learning concepts were identified and the renewed curriculum was piloted in Semester 2 of 2011. The paper presents details of the study and findings from a mixed-method evaluation approach through the semester. The outcomes of this study will be used to inform further review of the course in 2012, including further consideration of the threshold concepts. In future, it is anticipated that this case study will inform a framework for rapidly embedding sustainability within curriculum.

Enhancing student learning in the workplace through developing the leadership capabilities of clinical supervisors in the nursing discipline

The purpose of this project was to build the leadership capacity of clinical supervisors in the nursing discipline by developing, implementing and systematically embedding a leadership model into the structure and practice of student supervision. The University worked in partnership with three major metropolitan hospitals in Queensland to develop a framework and professional development program incorporating leadership and clinical supervision. The Leadership and Clinical Education (LaCE) program consisted of two structured workshops complemented by individual personal development projects undertaken by participants. Participants were supported in these activities with a purpose-built website that provides access to a wide variety of information and other learning resources. Quantitative and qualitative evaluations indicated that the approach was highly valued by participants, as it promoted useful peer dialogue, sharing of experiences and personal development in relation to assisting leadership development and student learning in the workplace. The LaCE program provides an ideal springboard for introducing the development of welltrained leaders into the clinical workplace. The resources developed have the potential to provide ongoing support for clinical supervisors to improve the learning of undergraduate nursing student. The challenge will be to achieve continued innovation within clinical education through sustainable leadership programs.

The role of metacognition in the development of EFL learners’ listening skill

This study investigated the impact of metacognitive instruction on the listening skill, and metacognitive knowledge of a group of male students (N = 30) who were learning English as a Foreign Language (EFL) in Iran. The study participants spoke Persian as a first language and were high-intermediate EFL learners. The participants received a guided lesson plan in metacognition (planning, monitoring, and evaluation) through a pedagogical cycle approach over a semester (eight weeks). International English Language Testing System listening tests (practice) were used to track the participants’ listening performance. Participants also completed a Metacognitive Awareness Listening Questionnaire, which examined their use of metacognitive awareness when engaged in listening tasks. Results indicated that the students improved their listening skill after being taught about metacognition; however, no significant use of metacognitive awareness was reported. This study concludes with a discussion of some potential implications, and provides scope for future research.

Educating for digital futures : what the learning strategies of digital media professionals can teach higher education

This article explores how universities might engage more effectively with the imperative to develop students’ 21st century skills for the information society, by examining learning challenges and professional learning strategies of successful digital media professionals. The findings of qualitative interviews with professionals from Australian games, online publishing, apps and software development companies reinforce an increasing body of literature that suggests that legacy university structures and pedagogical approaches are not conducive to learning for professional capability in the digital age. Study participants were ambivalent about the value of higher education to digital careers, in general preferring a range of situated online and face-to-face social learning strategies for professional currency. This article draws upon the learning preferences of the professionals in this study to present a model of 21st century learning, as linked with extant theory relating to informal, self-determined learning and communities of practice.

A psychology student’s reflections on learning psychodynamic psychotherapy in the classroom

A graduate psychology student reflects upon the experience of learning psychodynamic psychotherapy exclusively in the classroom. The majority of undergraduate psychology students have scant, and frequently inaccurate, exposure to psychodynamic psychotherapy. This appears to heavily influence students’ choice of postgraduate programs, and to reduce the likelihood that they will expose themselves to psychodynamic therapy at any stage of their careers. It is hoped that the original insights provided by this reflection will inform the development of psychodynamic psychotherapy teaching material that can be imparted effectively in undergraduate programs, even when access to patients and supervisors is not possible, so that more students are inspired to study psychodynamic psychotherapy in postgraduate programs.

Knowledge ecosystems of early career academics : a grounded theory of experiencing information use for learning in developmental networks

This study explored early career academics' experiences in using information to learn while building their networks for professional development. A 'knowledge ecosystem' model was developed consisting of informal learning interactions such as relating to information to create knowledge and engaging in mutually supportive relationships. Findings from this study present an alternative interpretation of information use for learning that is focused on processes manifesting as human interactions with informing entities revolving around the contexts of reciprocal human relationships.

Professional development and social networks in digital content industry micro businesses

This study explores the professional development strategies of digital content professionals in Australian micro businesses. This thesis presents the argument that as these professionals are working in cutting edge creative fields where digital technology drives ongoing change, formal education experiences may be less important than for other professionals, and that specific types of online and face-to-face socially mediated informal learning strategies may be critical to currency. This thesis documents the findings of a broad survey of industry professionals' learning needs and development strategies, in conjunction with rich data from in-depth interviews and social network analyses.

Peer assisted learning and generation Y : a case study

In recent years Australian Law Schools have implemented various forms of peer assisted learning or mentoring, including career mentoring by former students of final year students and orientation mentoring or tutoring by later year students of incoming first year students. The focus of these programs therefore is on the transition into or out of law school. There is not always as great an emphasis however, as part of this transition, on the use of law students belonging to the same unit cohort as a learning resource for each other within their degree. This is despite the claimed preference of Generation Y students for collaborative learning environments, authentic learning experiences and the development of marketable workplace skills. In the workplace, be it professional legal practice or otherwise, colleagues rely heavily on each other for information, support and guidance. In the undergraduate law degree at the Queensland University of Technology (‘QUT’) the Torts Student Peer Mentor Program aims to supplement a student’s understanding of the substantive law of torts with the development of life-long skills. As such it has the primary objective, albeit through discussion facilitated by more senior students, of encouraging first year students to develop for themselves the skills they need to be successful both as law students and as legal practitioners. Examples of such skills include those relevant to: preparation for assessment tasks; group work; problem solving, cognition and critical thinking; independent learning; and communication. Significantly, in this way, not only do the mentees benefit from involvement in the program, but the peer mentors, or program facilitators, themselves also benefit from their participation in the real world learning environment the program provides. This paper outlines the development and implementation of the above program, the pedagogy which influenced it, and its impact on student learning experiences

Liminal moments : designing, thinking and learning

This paper provides a contextual reflection for understanding best practice teaching to first year design students. The outcome (job) focussed approach to higher education has lead to some unanticipated collateral damage for students, and in the case we discuss, has altered the students’ expectations of course delivery with specific implications and challenges for design educators. This tendency in educational delivery systems is further compounded by the distinct characteristics of Generation Y students within a classroom context. It is our belief that foundational design education must focus more on process than outcomes, and through this research with first year design students we analyse and raise questions relative to the curriculum for a Design and Creative Thinking course—in which students not only benefit from learning the theories and processes of design thinking, conceptualisation and creativity, but also are encouraged to see it as an essential tool for their education and development as designers. This study considers the challenges within a design environment; specifically, we address the need for process based learning in contrast to the outcome-focused approach taken by most students. With this approach, students simultaneously learn to be a designer and rethink their approach to “doing design”.