Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

A framework for developing evaluation capacity in health care settings.

The purpose of this paper is to provide a framework for developing an effective evaluation practice within health care settings. Three features are reviewed; capacity building, the application of evaluation to program activities and the utilization of evaluation recommendations. First, the organizational elements required to establish effective evaluation practice are reviewed emphasizing that an organization's capacity for evaluation develops over time and in stages. Second, a comprehensive evaluation framework is presented which demonstrates how evaluation practice can be applied to all aspects of a program's life cycle, thus promoting the scope of evidence-based decision making within an organization. Finally, factors which influence the adoption of evaluation recommendations by decision makers are reviewed accompanied by strategies to promote the utilization of evaluation recommendations in organization decision making.

Leader-organization fit: comparing the effectiveness of paternalistic and transformational leadership in different organizational cultures

Research on paternalistic leadership (PL) has been based exclusively on national cultures´ differences. However there are cues that other contextual variables can add to the explanation of this construct. Due to its capacity to influence expectations of individuals in organizations, organizational culture can contribute to fill this gap. To test if organizational culture influences the effectiveness of leadership style, we conducted two experimental studies using Amazon’s Mechanical Turk, comparing effects of paternalistic and transformational leadership on followers’ outcomes. Using video clips and vignettes, we found that PL is better related to followers´ outcomes in cultures oriented to people than outcome, and that TL has a better relationship in cultures oriented to innovation than stability. The results suggest that organizational culture helps in explaining PL endorsement, and that further analysis of the influence of this variable to PL can provide a better understanding of the expression of this leadership style in organizations.

Information and communication technology for disaster risk management in the Caribbean: subregional solutions to the challenge of limited human resource capacity

This document was adapted from a paper originally presented to the 8th Annual Caribbean Conference of Comprehensive Disaster Management, held in Montego Bay, Jamaica in December, 2013. It summarizes several activities that ECLAC has undertaken to assess the current state of information and communications technology (ICT) in the field of disaster risk management (DRM) as practiced in the Caribbean. These activities included an in-depth study that encompassed a survey of disaster management organizations in the region, an Expert Group Meeting attended by the heads of several national disaster offices, and a training workshop for professionals working in DRM in the Caribbean. One of the notable conclusions of ECLAC’s investigation on this topic is that the lack of human capacity is the single largest constraint that is faced in the implementation of ICT projects for DRM in the Caribbean. In considering strategies to address the challenge of limited human capacity at a regional level, two separate issues are recognized – the need to increase the ICT capabilities of disaster management professionals, and the need to make ICT specialists available to disaster management organizations to advise and assist in the implementation of technology-focused projects. To that end, two models are proposed to engage with this issue at a regional level. The first entails the establishment of a network of ICT trainers in the Caribbean to help DRM staff develop a strategic understanding of how technology can be used to further their organizational goals. The second is the development of “Centres of Excellence” for ICT in the Caribbean, which would enable the deployment of specialized ICT expertise to national disaster management offices on a project-by-project basis.

Relationship between composition and organizational levels of nanostructured systems formed by Oleth 10 and PPG-5-Ceteth-20 for potential drug delivery

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

The firm. Techno-organizational changes, industrial relations and performances. An enquiry on Reggio Emilia local industrial system

It is not unknown that the evolution of firm theories has been developed along a path paved by an increasing awareness of the organizational structure importance. From the early “neoclassical” conceptualizations that intended the firm as a rational actor whose aim is to produce that amount of output, given the inputs at its disposal and in accordance to technological or environmental constraints, which maximizes the revenue (see Boulding, 1942 for a past mid century state of the art discussion) to the knowledge based theory of the firm (Nonaka & Takeuchi, 1995; Nonaka & Toyama, 2005), which recognizes in the firm a knnowledge creating entity, with specific organizational capabilities (Teece, 1996; Teece & Pisano, 1998) that allow to sustaine competitive advantages. Tracing back a map of the theory of the firm evolution, taking into account the several perspectives adopted in the history of thought, would take the length of many books. Because of that a more fruitful strategy is circumscribing the focus of the description of the literature evolution to one flow connected to a crucial question about the nature of firm’s behaviour and about the determinants of competitive advantages. In so doing I adopt a perspective that allows me to consider the organizational structure of the firm as an element according to which the different theories can be discriminated. The approach adopted starts by considering the drawbacks of the standard neoclassical theory of the firm. Discussing the most influential theoretical approaches I end up with a close examination of the knowledge based perspective of the firm. Within this perspective the firm is considered as a knowledge creating entity that produce and mange knowledge (Nonaka, Toyama, & Nagata, 2000; Nonaka & Toyama, 2005). In a knowledge intensive organization, knowledge is clearly embedded for the most part in the human capital of the individuals that compose such an organization. In a knowledge based organization, the management, in order to cope with knowledge intensive productions, ought to develop and accumulate capabilities that shape the organizational forms in a way that relies on “cross-functional processes, extensive delayering and empowerment” (Foss 2005, p.12). This mechanism contributes to determine the absorptive capacity of the firm towards specific technologies and, in so doing, it also shape the technological trajectories along which the firm moves. After having recognized the growing importance of the firm’s organizational structure in the theoretical literature concerning the firm theory, the subsequent point of the analysis is that of providing an overview of the changes that have been occurred at micro level to the firm’s organization of production. The economic actors have to deal with challenges posed by processes of internationalisation and globalization, increased and increasing competitive pressure of less developed countries on low value added production activities, changes in technologies and increased environmental turbulence and volatility. As a consequence, it has been widely recognized that the main organizational models of production that fitted well in the 20th century are now partially inadequate and processes aiming to reorganize production activities have been widespread across several economies in recent years. Recently, the emergence of a “new” form of production organization has been proposed both by scholars, practitioners and institutions: the most prominent characteristic of such a model is its recognition of the importance of employees commitment and involvement. As a consequence it is characterized by a strong accent on the human resource management and on those practices that aim to widen the autonomy and responsibility of the workers as well as increasing their commitment to the organization (Osterman, 1994; 2000; Lynch, 2007). This “model” of production organization is by many defined as High Performance Work System (HPWS). Despite the increasing diffusion of workplace practices that may be inscribed within the concept of HPWS in western countries’ companies, it is an hazard, to some extent, to speak about the emergence of a “new organizational paradigm”. The discussion about organizational changes and the diffusion of HPWP the focus cannot abstract from a discussion about the industrial relations systems, with a particular accent on the employment relationships, because of their relevance, in the same way as production organization, in determining two major outcomes of the firm: innovation and economic performances. The argument is treated starting from the issue of the Social Dialogue at macro level, both in an European perspective and Italian perspective. The model of interaction between the social parties has repercussions, at micro level, on the employment relationships, that is to say on the relations between union delegates and management or workers and management. Finding economic and social policies capable of sustaining growth and employment within a knowledge based scenario is likely to constitute the major challenge for the next generation of social pacts, which are the main social dialogue outcomes. As Acocella and Leoni (2007) put forward the social pacts may constitute an instrument to trade wage moderation for high intensity in ICT, organizational and human capital investments. Empirical evidence, especially focused on the micro level, about the positive relation between economic growth and new organizational designs coupled with ICT adoption and non adversarial industrial relations is growing. Partnership among social parties may become an instrument to enhance firm competitiveness. The outcome of the discussion is the integration of organizational changes and industrial relations elements within a unified framework: the HPWS. Such a choice may help in disentangling the potential existence of complementarities between these two aspects of the firm internal structure on economic and innovative performance. With the third chapter starts the more original part of the thesis. The data utilized in order to disentangle the relations between HPWS practices, innovation and economic performance refer to the manufacturing firms of the Reggio Emilia province with more than 50 employees. The data have been collected through face to face interviews both to management (199 respondents) and to union representatives (181 respondents). Coupled with the cross section datasets a further data source is constituted by longitudinal balance sheets (1994-2004). Collecting reliable data that in turn provide reliable results needs always a great effort to which are connected uncertain results. Data at micro level are often subjected to a trade off: the wider is the geographical context to which the population surveyed belong the lesser is the amount of information usually collected (low level of resolution); the narrower is the focus on specific geographical context, the higher is the amount of information usually collected (high level of resolution). For the Italian case the evidence about the diffusion of HPWP and their effects on firm performances is still scanty and usually limited to local level studies (Cristini, et al., 2003). The thesis is also devoted to the deepening of an argument of particular interest: the existence of complementarities between the HPWS practices. It has been widely shown by empirical evidence that when HPWP are adopted in bundles they are more likely to impact on firm’s performances than when adopted in isolation (Ichniowski, Prennushi, Shaw, 1997). Is it true also for the local production system of Reggio Emilia? The empirical analysis has the precise aim of providing evidence on the relations between the HPWS dimensions and the innovative and economic performances of the firm. As far as the first line of analysis is concerned it must to be stressed the fundamental role that innovation plays in the economy (Geroski & Machin, 1993; Stoneman & Kwoon 1994, 1996; OECD, 2005; EC, 2002). On this point the evidence goes from the traditional innovations, usually approximated by R&D investment expenditure or number of patents, to the introduction and adoption of ICT, in the recent years (Brynjolfsson & Hitt, 2000). If innovation is important then it is critical to analyse its determinants. In this work it is hypothesised that organizational changes and firm level industrial relations/employment relations aspects that can be put under the heading of HPWS, influence the propensity to innovate in product, process and quality of the firm. The general argument may goes as follow: changes in production management and work organization reconfigure the absorptive capacity of the firm towards specific technologies and, in so doing, they shape the technological trajectories along which the firm moves; cooperative industrial relations may lead to smother adoption of innovations, because not contrasted by unions. From the first empirical chapter emerges that the different types of innovations seem to respond in different ways to the HPWS variables. The underlying processes of product, process and quality innovations are likely to answer to different firm’s strategies and needs. Nevertheless, it is possible to extract some general results in terms of the most influencing HPWS factors on innovative performance. The main three aspects are training coverage, employees involvement and the diffusion of bonuses. These variables show persistent and significant relations with all the three innovation types. The same do the components having such variables at their inside. In sum the aspects of the HPWS influence the propensity to innovate of the firm. At the same time, emerges a quite neat (although not always strong) evidence of complementarities presence between HPWS practices. In terns of the complementarity issue it can be said that some specific complementarities exist. Training activities, when adopted and managed in bundles, are related to the propensity to innovate. Having a sound skill base may be an element that enhances the firm’s capacity to innovate. It may enhance both the capacity to absorbe exogenous innovation and the capacity to endogenously develop innovations. The presence and diffusion of bonuses and the employees involvement also spur innovative propensity. The former because of their incentive nature and the latter because direct workers participation may increase workers commitment to the organizationa and thus their willingness to support and suggest inovations. The other line of analysis provides results on the relation between HPWS and economic performances of the firm. There have been a bulk of international empirical studies on the relation between organizational changes and economic performance (Black & Lynch 2001; Zwick 2004; Janod & Saint-Martin 2004; Huselid 1995; Huselid & Becker 1996; Cappelli & Neumark 2001), while the works aiming to capture the relations between economic performance and unions or industrial relations aspects are quite scant (Addison & Belfield, 2001; Pencavel, 2003; Machin & Stewart, 1990; Addison, 2005). In the empirical analysis the integration of the two main areas of the HPWS represent a scarcely exploited approach in the panorama of both national and international empirical studies. As remarked by Addison “although most analysis of workers representation and employee involvement/high performance work practices have been conducted in isolation – while sometimes including the other as controls – research is beginning to consider their interactions” (Addison, 2005, p.407). The analysis conducted exploiting temporal lags between dependent and covariates, possibility given by the merger of cross section and panel data, provides evidence in favour of the existence of HPWS practices impact on firm’s economic performance, differently measured. Although it does not seem to emerge robust evidence on the existence of complementarities among HPWS aspects on performances there is evidence of a general positive influence of the single practices. The results are quite sensible to the time lags, inducing to hypothesize that time varying heterogeneity is an important factor in determining the impact of organizational changes on economic performance. The implications of the analysis can be of help both to management and local level policy makers. Although the results are not simply extendible to other local production systems it may be argued that for contexts similar to the Reggio Emilia province, characterized by the presence of small and medium enterprises organized in districts and by a deep rooted unionism, with strong supporting institutions, the results and the implications here obtained can also fit well. However, a hope for future researches on the subject treated in the present work is that of collecting good quality information over wider geographical areas, possibly at national level, and repeated in time. Only in this way it is possible to solve the Gordian knot about the linkages between innovation, performance, high performance work practices and industrial relations.

"The need obliged us" : culture as capacity during the Hurricane Stan emergency response. A case study from Tectitan, Huehuetenango, Guatemala

A major deficiency in disaster management plans is the assumption that pre-disaster civil-society does not have the capacity to respond effectively during crises. Following from this assumption a dominant emergency management strategy is to replace weak civil-society organizations with specialized disaster organizations that are often either military or Para-military and seek to centralize decision-making. Many criticisms have been made of this approach, but few specifically addresses disasters in the developing world. Disasters in the developing world present unique problems not seen in the developed world because they often occur in the context of compromised governments, and marginalized populations. In this context it is often community members themselves who possess the greatest capacity to respond to disasters. This paper focuses on the capacity of community groups to respond to disaster in a small town in rural Guatemala. Key informant interviews and ethnographic observations are used to reconstruct the community response to the disaster instigated by Hurricane Stan (2005) in the municipality of Tectitán in the Huehuetenango department. The interviews were analyzed using techniques adapted from grounded theory to construct a narrative of the events, and identify themes in the community’s disaster behavior. These themes are used to critique the emergency management plans advocated by the Guatemalan National Coordination for the Reduction of Disasters (CONRED). This paper argues that CONRED uncritically adopts emergency management strategies that do not account for the local realities in communities throughout Guatemala. The response in Tectitán was characterized by the formation of new organizations, whose actions and leadership structure were derived from “normal” or routine life. It was found that pre-existing social networks were resilient and easily re-oriented meet the novel needs of a crisis. New or emergent groups that formed during the disaster utilized social capital accrued by routine collective behavior, and employed organizational strategies derived from “normal” community relations. Based on the effectiveness of this response CONRED could improve its emergency planning on the local-level by utilizing the pre-existing community organizations rather than insisting that new disaster-specific organizations be formed.

Building the capacity for learning and change through reflective conversation

Conversation is central to the process of organizational learning and change. Drawing on the notion of reflective conversation, we describe an action research project, "learning through listening" in Omega, a residential healthcare organization. In this project, service users, staff, members of management committees, trustees, managers, and central office staff participated in listening to each other and in working together towards building capacity for creating their own vision of how the organization could move into the future, according to its values and ethos. In doing so they developed ways of engaging in reflective conversation that enabled progress towards a strategic direction.

Organizational Capability of Local Societies in Rural Development: A Comparative Study of Microfinance Organizations in Thailand and the Philippines

The importance of organizing local people for development work is widely recognized. Both governmental and non-governmental agencies have implemented various projects that have needed and encouraged collective action by people. Often, however, such projects malfunction after the outside agencies retreat from the project site, suggesting that making organizations is not the same as making a system of making organizations. The latter is essential to make rural organizations self-reliant and sustainable. This paper assumes that such a system exists in local societies and focuses on the capacity of local societies for creating and managing organizations for development. It reveals that (1) such capability differs according to the locality, (2) the difference depends on the structure of the organizations that coordinate people's social relations, and (3) the local administrative bodies define, at least partly, the organizational capability of local societies. We compare two rural societies, one in Thailand and the other in the Philippines, which show clear contrasts in both the form of microfinance organizations and the way of making these organizations.

Capacity building in development projects

Development projects of different types mainly aim to alleviate poverty and ameliorate the livelihoods of local people. One of the strategies commonly used is to focus on organizations and build from their existing capacities in order to improve their living standards or try to build new organizations to work in a common project. Social and human capitals are two key components of these organizations and they might be crucial to the success of the actions that they accomplish. Both can be considered as part of the social capacity of the local organization. This capacity can be enforced with development projects through capacity building. This term means much more than training activities as it includes not only human resource development but also organizational and institutional development (UNESCO, 2010). Capacity and capacity building concepts, as well as capacity measurements in this context are explored to build a framework to evaluate the social capacity generated with the interventions and better plan the actions to be undertaken by the projects to succeed. The focus is set on rural development projects.

Capacity-building through English language training to improve long-term resettlement success: A case study of the refugees along the Thailand-Burma border

Achieving long-term resettlement success is a challenge for many refugees seeking to restart their lives after displacement and being uprooted from their lives. Refugees must deal with finding employment, integrating into a society immensely different from what they have known their whole lives, and starting over from scratch. Learning a new language enables refugees to progress towards integration and long-term resettlement success, however, resettled refugees face a multitude of barriers in the U.S. to accessing language classes and attaining English proficiency. This study seeks to bridge this problem by exploring the possibilities of implementing a standardized language training program in the refugee camps to better prepare refugees for resettlement. A case study of the refugees along the Thai-Burma border demonstrated the significance of learning English in the camps on eventual English proficiency as well as the need for increased partnerships to overcome the barriers of lack of motivation and lack of funding. The author explores the possibilities of implementing a language training program in the camps by determining need, interest, barriers, and perceptions through the use of interviews, surveys, and focus groups of camp refugees, resettled refugees, and key organizational representatives. The significance of these results offers the possibility of leveraging and unlocking resettlement as a durable solution for more of the world's refugees in protracted situations.

Do organizational climate and competitive strategy moderate the relationship between human resource management and productivity?

This study examined whether the effectiveness of human resource management (HRM)practices is contingent on organizational climate and competitive strategy The concepts of internol and external fit suggest that the positive relationship between HRM and subsequent productivity will be stronger for firms with a positive organizational climate and for firms using differentiation strategies. Resource allocation theories of motivation, on the other hand, predict that the relationship between HRM and productivity will be stronger for firms with a poor climate because employees working in these firms should have the greatest amount of spare capacity. The results supported the resource allocation argument.

A victim of its own success: Internationalization, neoliberalism, and organizational involution at the Business Council of Australia

The focus of this article is on the Business Council of Australia (BCA), an association of the CEOs of the 100 or so largest companies operating in Australia. Since its inception the BCA has been an influential supporter of largely successful efforts to neoliberalize and internationalize the Australian economy. Running in parallel with these developments, however, the BCA has moved from being a "somewhat strong" to a relatively weak policy organization. This article argues these two trends are causally related. Neoliberal-inspired economic restructuring and economic internationalization have weakened the "logic of membership" and the "logic of influence" of the BCA, leading to a process of organizational involution. Furthermore, potential offsets to what I describe as the organizational predations of neoliberalism and internationalization - especially via a willingness or capacity to forge supportive or mutualistic relations with the state - have not been realized.

Do organizational climate and competitive strategy moderate the relationship between human resource management and productivity?

This study examined whether the effectiveness of human resource management (HRM) practices is contingent on organizational climate and competitive strategy. The concepts of internal and external fit suggest that the positive relationship between HRM and subsequent productivity will be stronger for firms with a positive organizational climate and for firms using differentiation strategies. Resource allocation theories of motivation, on the other hand, predict that the relationship between HRM and productivity will be stronger for firms with a poor climate because employees working in these firms should have the greatest amount of spare capacity. The results supported the resource allocation argument. © 2005 Southern Management Association. All rights reserved.

The East-Central European new donors: mapping capacity building and remaining challenges

In the past decade, the East-Central European countries were provided significant external capacity building assistance in order to help their emergence as donors of foreign aid. This paper aims to map these capacity development programs and identify where they have helped and what challenges remain for the new donors. The main conclusion is that while capacity building has been instrumental in building organizational structures, working procedures and training staff, deeper underlying problems such as low levels of financing, lacking political will, the need for visibility and low staff numbers continue to hinder the new international development policies.