Mid-term evaluation of African Technology Policy Studies Network Phase VI Strategic Plan: for the period January 1, 2009 to December 31, 2010

The African Technology Policy Studies Network (ATPS) is a multidisciplinary network of researchers, private sector actors, policymakers and civil society. ATPS has the vision to become the leading international centre of excellence and reference in science, technology and innovation (STI) systems research, training and capacity building, communication and sensitization, knowledge brokerage, policy advocacy and outreach in Africa. It has a Regional Secretariat in Nairobi Kenya, and operates through national chapters in 29 countries (including 27 in Africa and two Chapters in the United Kingdom and USA for Africans in the Diaspora) with an expansion plan to cover the entire continent by 2015. The ATPS Phase VI Strategic Plan aims to improve the understanding and functioning of STI processes and systems to strengthen the learning capacity, social responses, and governance of STI for addressing Africa's development challenges, with a specific focus on the Millennium Development Goals (MDGs). A team of external evaluators carried out a midterm review to assess the effectiveness and efficiency of the implementation of the Strategic Plan for the period January 1, 2009 to December 31, 2010. The evaluation methodology involved multiple quantitative and qualitative methods to assess the qualitative and quantitative inputs (human resources, financial resources, time, etc.) into ATPS activities (both thematic and facilitative) and their tangible and intangible outputs, outcomes and impacts. Methods included a questionnaire survey of ATPS members and stakeholders, key informant interviews, and focus group discussions (FGDs) with members in six countries. Effectiveness of Programmes Under all six strategic goals, very good progress has been made towards planned outputs and outcomes. This is evidenced by key performance indicators (KPIs) generated from desk review, ratings from the survey respondents, and the themes that run through the FGDs. Institutional and Programme Cost Effectiveness Institutional Effectiveness: assessment of institutional effectiveness suggests that adequate management frameworks are in place and are being used effectively and transparently. Also technical and financial accounting mechanisms are being followed in accordance with grant agreements and with global good practice. This is evidenced by KPIs generated from desk review. Programme Cost Effectiveness: assessment of cost-effectiveness of execution of programmes shows that organisational structure is efficient, delivering high quality, relevant research at relatively low cost by international standards. The evidence includes KPIs from desk review: administrative costs to programme cost ratio has fallen steadily, to around 10%; average size of research grants is modest, without compromising quality. There is high level of pro bono input by ATPS members. ATPS Programmes Strategic Evaluation ATPS research and STI related activities are indeed unique and well aligned with STI issues and needs facing Africa and globally. The multi-disciplinary and trans-boundary nature of the research activities are creating a unique group of research scientists. The ATPS approach to research and STI issues is paving the way for the so called Third Generation University (3GU). Understanding this unique positioning, an increasing number of international multilateral agencies are seeking partnership with ATPS. ATPS is seeing an increasing level of funding commitments by Donor Partners. Recommendations for ATPS Continued Growth and Effectiveness On-going reform of ATPS administrative structure to continue The on-going reforms that have taken place within the Board, Regional Secretariat, and at the National Chapter coordination levels are welcomed. Such reform should continue until fully functional corporate governance policy and practices are fully established and implemented across the ATPS governance structures. This will further strengthen ATPS to achieve the vision of being the leading STI policy brokerage organization in Africa. Although training in corporate governance has been carried out for all sectors of ATPS leadership structure in recent time, there is some evidence that these systems have not yet been fully implemented effectively within all the governance structures of the organization, especially at the Board and National chapter levels. Future training should emphasize practical application with exercises relevant to ATPS leadership structure from the Board to the National Chapter levels. Training on Transformational Leadership - Leading a Change Though a subject of intense debate amongst economists and social scientists, it is generally agreed that cultural mindsets and attitudes could enhance and/or hinder organizational progress. ATPS’s vision demands transformational leadership skills amongst its leaders from the Board members to the National Chapter Coordinators. To lead such a change, ATPS leaders must understand and avoid personal and cultural mindsets and value systems that hinder change, while embracing those that enhance it. It requires deliberate assessment of cultural, behavioural patterns that could hinder progress and the willingness to be recast into cultural and personal habits that make for progress. Improvement of relationship amongst the Board, Secretariat, and National Chapters A large number of ATPS members and stakeholders feel they do not have effective communications and/or access to Board, National Chapter Coordinators and Regional Secretariat activities. Effort should be made to improve the implementation of ATPS communication strategy to improve on information flows amongst the ATPS management and the members. The results of the survey and the FGDs suggest that progress has been made during the past two years in this direction, but more could be done to ensure effective flow of pertinent information to members following ATPS communications channels. Strategies for Increased Funding for National Chapters There is a big gap between the fundraising skills of the Regional Secretariat and those of the National Coordinators. In some cases, funds successfully raised by the Secretariat and disbursed to national chapters were not followed up with timely progress and financial reports by some national chapters. Adequate training in relevant skills required for effective interactions with STI key policy players should be conducted regularly for National Chapter coordinators and ATPS members. The ongoing training in grant writing should continue and be made continent-wide if funding permits. Funding of National Chapters should be strategic such that capacity in a specific area of research is built which, with time, will not only lead to a strong research capacity in that area, but also strengthen academic programmes. For example, a strong climate change programme is emerging at University of Nigeria Nsukka (UNN), with strong collaborations with Universities from neighbouring States. Strategies to Increase National Government buy-in and support for STI Translating STI research outcomes into policies requires a great deal of emotional intelligence, skills which are often lacking in the first and second generation universities. In the epoch of the science-based or 2GUs, governments were content with universities carrying out scientific research and providing scientific education. Now they desire to see universities as incubators of new science- or technology-based commercial activities, whether by existing firms or start-ups. Hence, governments demand that universities take an active and leading role in the exploitation of their knowledge and they are willing to make funds available to support such activities. Thus, for universities to gain the attention of national leadership they must become centres of excellence and explicit instruments of economic development in the knowledge-based economy. The universities must do this while working collaboratively with government departments, parastatals, and institutions and dedicated research establishments. ATPS should anticipate these shifting changes and devise programmes to assist both government and universities to relate effectively. New administrative structures in member organizations to sustain and manage the emerging STI multidisciplinary teams Second Generation universities (2GUs) tend to focus on pure science and often do not regard the application of their know-how as their task. In contrast, Third Generation Universities (3GUs) objectively stimulate techno-starters – students or academics – to pursue the exploitation or commercialisation of the knowledge they generate. They view this as being equal in importance to the objectives of scientific research and education. Administratively, research in the 2GU era was mainly monodisciplinary and departments were structured along disciplines. The emerging interdisciplinary scientific teams with focus on specific research areas functionally work against the current mono-disciplinary faculty-based, administrative structure of 2GUs. For interdisciplinary teams, the current faculty system is an obstacle. There is a need for new organisational forms for university management that can create responsibilities for the task of know-how exploitation. ATPS must anticipate this and begin to strategize solutions for their member institutions to transition to 3Gus administrative structure, otherwise ATPS growth will plateau, and progress achieved so far may be stunted.

Uptake pathways: the potential of Bayesian belief networks to assist the management, monitoring and evaluation of development-orientated research

The effectiveness of development assistance has come under renewed scrutiny in recent years. In an era of growing economic liberalisation, research organisations are increasingly being asked to account for the use of public funds by demonstrating achievements. However, in the natural resources (NR) research field, conventional economic assessment techniques have focused on quantifying the impact achieved rather understanding the process that delivered it. As a result, they provide limited guidance for planners and researchers charged with selecting and implementing future research. In response, “pathways” or logic models have attracted increased interest in recent years as a remedy to this shortcoming. However, as commonly applied these suffer from two key limitations in their ability to incorporate risk and assess variance from plan. The paper reports the results of a case study that used a Bayesian belief network approach to address these limitations and outlines its potential value as a tool to assist the planning, monitoring and evaluation of development-orientated research.

Deconstruction plan assessment using multi-criteria decision-making methods

The purpose of this paper is to present two multi-criteria decision-making models, including an Analytic Hierarchy Process (AHP) model and an Analytic Network Process (ANP) model for the assessment of deconstruction plans and to make a comparison between the two models with an experimental case study. Deconstruction planning is under pressure to reduce operation costs, adverse environmental impacts and duration, in the meanwhile to improve productivity and safety in accordance with structure characteristics, site conditions and past experiences. To achieve these targets in deconstruction projects, there is an impending need to develop a formal procedure for contractors to select a most appropriate deconstruction plan. Because numbers of factors influence the selection of deconstruction techniques, engineers definitely need effective tools to conduct the selection process. In this regard, multi-criteria decision-making methods such as AHP have been adopted to effectively support deconstruction technique selection in previous researches. in which it has been proved that AHP method can help decision-makers to make informed decisions on deconstruction technique selection based on a sound technical framework. In this paper, the authors present the application and comparison of two decision-making models including the AHP model and the ANP model for deconstruction plan assessment. The paper concludes that both AHP and ANP are viable and capable tools for deconstruction plan assessment under the same set of evaluation criteria. However, although the ANP can measure relationship among selection criteria and their sub-criteria, which is normally ignored in the AHP, the authors also indicate that whether the ANP model can provide a more accurate result should be examined in further research.

The evaluation of CloudSat and CALIPSO ice microphysical products using ground-based cloud radar and lidar observations

In this paper, the statistical properties of tropical ice clouds (ice water content, visible extinction, effective radius, and total number concentration) derived from 3 yr of ground-based radar–lidar retrievals from the U.S. Department of Energy Atmospheric Radiation Measurement Climate Research Facility in Darwin, Australia, are compared with the same properties derived using the official CloudSat microphysical retrieval methods and from a simpler statistical method using radar reflectivity and air temperature. It is shown that the two official CloudSat microphysical products (2B-CWC-RO and 2B-CWC-RVOD) are statistically virtually identical. The comparison with the ground-based radar–lidar retrievals shows that all satellite methods produce ice water contents and extinctions in a much narrower range than the ground-based method and overestimate the mean vertical profiles of microphysical parameters below 10-km height by over a factor of 2. Better agreements are obtained above 10-km height. Ways to improve these estimates are suggested in this study. Effective radii retrievals from the standard CloudSat algorithms are characterized by a large positive bias of 8–12 μm. A sensitivity test shows that in response to such a bias the cloud longwave forcing is increased from 44.6 to 46.9 W m−2 (implying an error of about 5%), whereas the negative cloud shortwave forcing is increased from −81.6 to −82.8 W m−2. Further analysis reveals that these modest effects (although not insignificant) can be much larger for optically thick clouds. The statistical method using CloudSat reflectivities and air temperature was found to produce inaccurate mean vertical profiles and probability distribution functions of effective radius. This study also shows that the retrieval of the total number concentration needs to be improved in the official CloudSat microphysical methods prior to a quantitative use for the characterization of tropical ice clouds. Finally, the statistical relationship used to produce ice water content from extinction and air temperature obtained by the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite is evaluated for tropical ice clouds. It is suggested that the CALIPSO ice water content retrieval is robust for tropical ice clouds, but that the temperature dependence of the statistical relationship used should be slightly refined to better reproduce the radar–lidar retrievals.