Comparative evaluation of a new effective population size estimator based on approximate Bayesian computation

We describe and evaluate a new estimator of the effective population size (N-e), a critical parameter in evolutionary and conservation biology. This new "SummStat" N-e. estimator is based upon the use of summary statistics in an approximate Bayesian computation framework to infer N-e. Simulations of a Wright-Fisher population with known N-e show that the SummStat estimator is useful across a realistic range of individuals and loci sampled, generations between samples, and N-e values. We also address the paucity of information about the relative performance of N-e estimators by comparing the SUMMStat estimator to two recently developed likelihood-based estimators and a traditional moment-based estimator. The SummStat estimator is the least biased of the four estimators compared. In 32 of 36 parameter combinations investigated rising initial allele frequencies drawn from a Dirichlet distribution, it has the lowest bias. The relative mean square error (RMSE) of the SummStat estimator was generally intermediate to the others. All of the estimators had RMSE > 1 when small samples (n = 20, five loci) were collected a generation apart. In contrast, when samples were separated by three or more generations and Ne less than or equal to 50, the SummStat and likelihood-based estimators all had greatly reduced RMSE. Under the conditions simulated, SummStat confidence intervals were more conservative than the likelihood-based estimators and more likely to include true N-e. The greatest strength of the SummStat estimator is its flexible structure. This flexibility allows it to incorporate any, potentially informative summary statistic from Population genetic data.

Evaluation of a nucleoprotein-based enzyme-linked immunosorbent assay for the detection of antibodies against infectious bronchitis virus

As an immunogen of the coronavirus, the nucleoprotein (N) is a potential antigen for the serological monitoring of infectious bronchitis virus (IBV). In this report, recombinant N protein from the Beaudette strain of IBV was produced and purified from Escherichia coli as well as Sf9 ( insect) cells, and used for the coating of enzyme-linked immunosorbent assay ( ELISA) plates. The N protein produced in Sf9 cells was phosphorylated whereas N protein from E. coli was not. Our data indicated that N protein purified from E. coli was more sensitive to anti-IBV serum than the protein from Sf9 cells. The recombinant N protein did not react with the antisera to other avian pathogens, implying that it was specific in the recognition of IBV antibodies. In addition, the data from the detection of field samples and IBV strains indicated that using the recombinant protein as coating antigen could achieve an equivalent performance to an ELISA kit based on infected material extracts as a source of antigen(s). ELISAs based on recombinant proteins are safe ( no live virus), clean ( only virus antigens are present), specific ( single proteins can be used) and rapid ( to respond to new viral strains and strains that cannot necessarily be easily cultured).

Performance of asynchronous channel assignment scheme in non-uniform and dynamic topology WLANs

Dense deployments of wireless local area networks (WLANs) are fast becoming a permanent feature of all developed cities around the world. While this increases capacity and coverage, the problem of increased interference, which is exacerbated by the limited number of channels available, can severely degrade the performance of WLANs if an effective channel assignment scheme is not employed. In an earlier work, an asynchronous, distributed and dynamic channel assignment scheme has been proposed that (1) is simple to implement, (2) does not require any knowledge of the throughput function, and (3) allows asynchronous channel switching by each access point (AP). In this paper, we present extensive performance evaluation of this scheme when it is deployed in the more practical non-uniform and dynamic topology scenarios. Specifically, we investigate its effectiveness (1) when APs are deployed in a nonuniform fashion resulting in some APs suffering from higher levels of interference than others and (2) when APs are effectively switched `on/off' due to the availability/lack of traffic at different times, which creates a dynamically changing network topology. Simulation results based on actual WLAN topologies show that robust performance gains over other channel assignment schemes can still be achieved even in these realistic scenarios.

PETS2010 and PETS2009 evaluation of results using individual ground truthed single views

This paper presents the results of the crowd image analysis challenge of the PETS2010 workshop. The evaluation was carried out using a selection of the metrics developed in the Video Analysis and Content Extraction (VACE) program and the CLassification of Events, Activities, and Relationships (CLEAR) consortium. The PETS 2010 evaluation was performed using new ground truthing create from each independant two dimensional view. In addition, the performance of the submissions to the PETS 2009 and Winter-PETS 2009 were evaluated and included in the results. The evaluation highlights the detection and tracking performance of the authors’ systems in areas such as precision, accuracy and robustness.

Evaluation of the time resolved fluorescence immunoassay (TRFIA) for the detection of varicella zoster virus (VZV) antibodies following vaccination of healthcare workers

Determination of varicella zoster virus (VZV) immunity in healthcare workers without a history of chickenpox is important for identifying those in need of vOka vaccination. Post immunisation, healthcare workers in the UK who work with high risk patients are tested for seroconversion. To assess the performance of the time-resolved fluorescence immunoassay (TRFIA) for the detection of antibody in vaccinated as well as unvaccinated individuals, a cut-off was first calculated. VZV-IgG specific avidity and titres six weeks after the first dose of vaccine were used to identify subjects with pre-existing immunity among a cohort of 110 healthcare workers. Those with high avidity (≥60%) were considered to have previous immunity to VZV and those with low or equivocal avidity (<60%) were considered naive. The former had antibody levels ≥400mIU/mL and latter had levels <400mIU/mL. Comparison of the baseline values of the naive and immune groups allowed the estimation of a TRFIA cut-off value of >130mIU/mL which best discriminated between the two groups and this was confirmed by ROC analysis. Using this value, the sensitivity and specificity of TRFIA cut-off were 90% (95% CI 79-96), and 78% (95% CI 61-90) respectively in this population. A subset of samples tested by the gold standard Fluorescence Antibody to Membrane Antigen (FAMA) test showed 84% (54/64) agreement with TRFIA.

Quantitative evaluation of three reconfiguration strategies on FPGAs: a case study

Reconfigurable computing is becoming an important new alternative for implementing computations. Field programmable gate arrays (FPGAs) are the ideal integrated circuit technology to experiment with the potential benefits of using different strategies of circuit specialization by reconfiguration. The final form of the reconfiguration strategy is often non-trivial to determine. Consequently, in this paper, we examine strategies for reconfiguration and, based on our experience, propose general guidelines for the tradeoffs using an area-time metric called functional density. Three experiments are set up to explore different reconfiguration strategies for FPGAs applied to a systolic implementation of a scalar quantizer used as a case study. Quantitative results for each experiment are given. The regular nature of the example means that the results can be generalized to a wide class of industry-relevant problems based on arrays.

Towards a method for the economic evaluation of environmental indicators in UK Integrated Arable Farming Systems

Integrated Arable Farming Systems (IAFS), which involve a reduction in the use of off-farm inputs, are attracting considerable research interest in the UK. The objectives of these systems experiments are to compare their financial performance with that from conventional or current farming practices. To date, this comparison has taken little account of any environmental benefits (or disbenefits) of the two systems. The objective of this paper is to review the assessment methodologies available for the analysis of environmental impacts. To illustrate the results of this exercise, the methodology and environmental indicators chosen are then applied to data from one of the LINK - Integrated Farming Systems experimental sites. Data from the Pathhead site in Southern Scotland are used to evaluate the use of invertebrates and nitrate loss as environmental indicators within IAFS. The results suggest that between 1992 and 1995 the biomass of earthworms fell by 28 kg per hectare on the integrated rotation and rose by 31 kg per hectare on the conventional system. This led to environmental costs ranging between £2.24 and £13.44 per hectare for the integrated system and gains of between £2.48 and £14.88 for the conventional system. In terms of nitrate, the integrated system had an estimated loss of £72.21 per hectare in comparison to £149.40 per hectare on the conventional system. Conclusions are drawn about the advantages and disadvantages of this type of analytical framework. Keywords: Farming systems; IAFS; Environmental valuation; Economics; Earthworms; Nitrates; Soil fauna

Mid-term evaluation of African Technology Policy Studies Network Phase VI Strategic Plan: for the period January 1, 2009 to December 31, 2010

The African Technology Policy Studies Network (ATPS) is a multidisciplinary network of researchers, private sector actors, policymakers and civil society. ATPS has the vision to become the leading international centre of excellence and reference in science, technology and innovation (STI) systems research, training and capacity building, communication and sensitization, knowledge brokerage, policy advocacy and outreach in Africa. It has a Regional Secretariat in Nairobi Kenya, and operates through national chapters in 29 countries (including 27 in Africa and two Chapters in the United Kingdom and USA for Africans in the Diaspora) with an expansion plan to cover the entire continent by 2015. The ATPS Phase VI Strategic Plan aims to improve the understanding and functioning of STI processes and systems to strengthen the learning capacity, social responses, and governance of STI for addressing Africa's development challenges, with a specific focus on the Millennium Development Goals (MDGs). A team of external evaluators carried out a midterm review to assess the effectiveness and efficiency of the implementation of the Strategic Plan for the period January 1, 2009 to December 31, 2010. The evaluation methodology involved multiple quantitative and qualitative methods to assess the qualitative and quantitative inputs (human resources, financial resources, time, etc.) into ATPS activities (both thematic and facilitative) and their tangible and intangible outputs, outcomes and impacts. Methods included a questionnaire survey of ATPS members and stakeholders, key informant interviews, and focus group discussions (FGDs) with members in six countries. Effectiveness of Programmes Under all six strategic goals, very good progress has been made towards planned outputs and outcomes. This is evidenced by key performance indicators (KPIs) generated from desk review, ratings from the survey respondents, and the themes that run through the FGDs. Institutional and Programme Cost Effectiveness Institutional Effectiveness: assessment of institutional effectiveness suggests that adequate management frameworks are in place and are being used effectively and transparently. Also technical and financial accounting mechanisms are being followed in accordance with grant agreements and with global good practice. This is evidenced by KPIs generated from desk review. Programme Cost Effectiveness: assessment of cost-effectiveness of execution of programmes shows that organisational structure is efficient, delivering high quality, relevant research at relatively low cost by international standards. The evidence includes KPIs from desk review: administrative costs to programme cost ratio has fallen steadily, to around 10%; average size of research grants is modest, without compromising quality. There is high level of pro bono input by ATPS members. ATPS Programmes Strategic Evaluation ATPS research and STI related activities are indeed unique and well aligned with STI issues and needs facing Africa and globally. The multi-disciplinary and trans-boundary nature of the research activities are creating a unique group of research scientists. The ATPS approach to research and STI issues is paving the way for the so called Third Generation University (3GU). Understanding this unique positioning, an increasing number of international multilateral agencies are seeking partnership with ATPS. ATPS is seeing an increasing level of funding commitments by Donor Partners. Recommendations for ATPS Continued Growth and Effectiveness On-going reform of ATPS administrative structure to continue The on-going reforms that have taken place within the Board, Regional Secretariat, and at the National Chapter coordination levels are welcomed. Such reform should continue until fully functional corporate governance policy and practices are fully established and implemented across the ATPS governance structures. This will further strengthen ATPS to achieve the vision of being the leading STI policy brokerage organization in Africa. Although training in corporate governance has been carried out for all sectors of ATPS leadership structure in recent time, there is some evidence that these systems have not yet been fully implemented effectively within all the governance structures of the organization, especially at the Board and National chapter levels. Future training should emphasize practical application with exercises relevant to ATPS leadership structure from the Board to the National Chapter levels. Training on Transformational Leadership - Leading a Change Though a subject of intense debate amongst economists and social scientists, it is generally agreed that cultural mindsets and attitudes could enhance and/or hinder organizational progress. ATPS’s vision demands transformational leadership skills amongst its leaders from the Board members to the National Chapter Coordinators. To lead such a change, ATPS leaders must understand and avoid personal and cultural mindsets and value systems that hinder change, while embracing those that enhance it. It requires deliberate assessment of cultural, behavioural patterns that could hinder progress and the willingness to be recast into cultural and personal habits that make for progress. Improvement of relationship amongst the Board, Secretariat, and National Chapters A large number of ATPS members and stakeholders feel they do not have effective communications and/or access to Board, National Chapter Coordinators and Regional Secretariat activities. Effort should be made to improve the implementation of ATPS communication strategy to improve on information flows amongst the ATPS management and the members. The results of the survey and the FGDs suggest that progress has been made during the past two years in this direction, but more could be done to ensure effective flow of pertinent information to members following ATPS communications channels. Strategies for Increased Funding for National Chapters There is a big gap between the fundraising skills of the Regional Secretariat and those of the National Coordinators. In some cases, funds successfully raised by the Secretariat and disbursed to national chapters were not followed up with timely progress and financial reports by some national chapters. Adequate training in relevant skills required for effective interactions with STI key policy players should be conducted regularly for National Chapter coordinators and ATPS members. The ongoing training in grant writing should continue and be made continent-wide if funding permits. Funding of National Chapters should be strategic such that capacity in a specific area of research is built which, with time, will not only lead to a strong research capacity in that area, but also strengthen academic programmes. For example, a strong climate change programme is emerging at University of Nigeria Nsukka (UNN), with strong collaborations with Universities from neighbouring States. Strategies to Increase National Government buy-in and support for STI Translating STI research outcomes into policies requires a great deal of emotional intelligence, skills which are often lacking in the first and second generation universities. In the epoch of the science-based or 2GUs, governments were content with universities carrying out scientific research and providing scientific education. Now they desire to see universities as incubators of new science- or technology-based commercial activities, whether by existing firms or start-ups. Hence, governments demand that universities take an active and leading role in the exploitation of their knowledge and they are willing to make funds available to support such activities. Thus, for universities to gain the attention of national leadership they must become centres of excellence and explicit instruments of economic development in the knowledge-based economy. The universities must do this while working collaboratively with government departments, parastatals, and institutions and dedicated research establishments. ATPS should anticipate these shifting changes and devise programmes to assist both government and universities to relate effectively. New administrative structures in member organizations to sustain and manage the emerging STI multidisciplinary teams Second Generation universities (2GUs) tend to focus on pure science and often do not regard the application of their know-how as their task. In contrast, Third Generation Universities (3GUs) objectively stimulate techno-starters – students or academics – to pursue the exploitation or commercialisation of the knowledge they generate. They view this as being equal in importance to the objectives of scientific research and education. Administratively, research in the 2GU era was mainly monodisciplinary and departments were structured along disciplines. The emerging interdisciplinary scientific teams with focus on specific research areas functionally work against the current mono-disciplinary faculty-based, administrative structure of 2GUs. For interdisciplinary teams, the current faculty system is an obstacle. There is a need for new organisational forms for university management that can create responsibilities for the task of know-how exploitation. ATPS must anticipate this and begin to strategize solutions for their member institutions to transition to 3Gus administrative structure, otherwise ATPS growth will plateau, and progress achieved so far may be stunted.

In vitro evaluation of the fermentation properties and potential prebiotic activity of caprine cheese whey oligosaccharides in batch culture systems

The prebiotic effect of oligosaccharides recovered and purified from caprine whey, was evaluated by in vitro fermentation under anaerobic conditions using batch cultures at 37ºC with human faeces. Effects on key gut bacterial groups were monitored over 24h by fluorescence in situ hybridisation (FISH), which was used to determine a quantitative prebiotic index score. Production of short-chain fatty acids (SCFAs) as fermentation end products was analysed by high-performance liquid chromatography (HPLC). Growth of Bifidobacterium spp was significantly higher (p ≥ 0.05) with the purified oligosaccharides compared to the negative control. Lactic and propionic acids were the main SCFAs produced. Antimicrobial activity of the oligosaccharides was also tested, revealing no inhibition though a decrease in Staphylococcus aureus and Escherichia coli growth. These findings indicate that naturally extracted oligosaccharides from caprine whey could be used as new and valuable source of prebiotics.

Probabilistic evaluation of time series models : a comparison of several approaches

Several methods are examined which allow to produce forecasts for time series in the form of probability assignments. The necessary concepts are presented, addressing questions such as how to assess the performance of a probabilistic forecast. A particular class of models, cluster weighted models (CWMs), is given particular attention. CWMs, originally proposed for deterministic forecasts, can be employed for probabilistic forecasting with little modification. Two examples are presented. The first involves estimating the state of (numerically simulated) dynamical systems from noise corrupted measurements, a problem also known as filtering. There is an optimal solution to this problem, called the optimal filter, to which the considered time series models are compared. (The optimal filter requires the dynamical equations to be known.) In the second example, we aim at forecasting the chaotic oscillations of an experimental bronze spring system. Both examples demonstrate that the considered time series models, and especially the CWMs, provide useful probabilistic information about the underlying dynamical relations. In particular, they provide more than just an approximation to the conditional mean.

Evaluation of climate models using palaeoclimatic data.

There is large uncertainty about the magnitude of warming and how rainfall patterns will change in response to any given scenario of future changes in atmospheric composition and land use. The models used for future climate projections were developed and calibrated using climate observations from the past 40 years. The geologic record of environmental responses to climate changes provides a unique opportunity to test model performance outside this limited climate range. Evaluation of model simulations against palaeodata shows that models reproduce the direction and large-scale patterns of past changes in climate, but tend to underestimate the magnitude of regional changes. As part of the effort to reduce model-related uncertainty and produce more reliable estimates of twenty-first century climate, the Palaeoclimate Modelling Intercomparison Project is systematically applying palaeoevaluation techniques to simulations of the past run with the models used to make future projections. This evaluation will provide assessments of model performance, including whether a model is sufficiently sensitive to changes in atmospheric composition, as well as providing estimates of the strength of biosphere and other feedbacks that could amplify the model response to these changes and modify the characteristics of climate variability.

Multi-site evaluation of an urban land-surface model: intra-urban heterogeneity, seasonality and parameter complexity requirements

An extensive off-line evaluation of the Noah/Single Layer Urban Canopy Model (Noah/SLUCM) urban land-surface model is presented using data from 15 sites to assess (1) the ability of the scheme to reproduce the surface energy balance observed in a range of urban environments, including seasonal changes, and (2) the impact of increasing complexity of input parameter information. Model performance is found to be most dependent on representation of vegetated surface area cover; refinement of other parameter values leads to smaller improvements. Model biases in net all-wave radiation and trade-offs between turbulent heat fluxes are highlighted using an optimization algorithm. Here we use the Urban Zones to characterize Energy partitioning (UZE) as the basis to assign default SLUCM parameter values. A methodology (FRAISE) to assign sites (or areas) to one of these categories based on surface characteristics is evaluated. Using three urban sites from the Basel Urban Boundary Layer Experiment (BUBBLE) dataset, an independent evaluation of the model performance with the parameter values representative of each class is performed. The scheme copes well with both seasonal changes in the surface characteristics and intra-urban heterogeneities in energy flux partitioning, with RMSE performance comparable to similar state-of-the-art models for all fluxes, sites and seasons. The potential of the methodology for high-resolution atmospheric modelling application using the Weather Research and Forecasting (WRF) model is highlighted. This analysis supports the recommendations that (1) three classes are appropriate to characterize the urban environment, and (2) that the parameter values identified should be adopted as default values in WRF.

Development and evaluation of an Earth-System model – HadGEM2

We describe here the development and evaluation of an Earth system model suitable for centennial-scale climate prediction. The principal new components added to the physical climate model are the terrestrial and ocean ecosystems and gas-phase tropospheric chemistry, along with their coupled interactions. The individual Earth system components are described briefly and the relevant interactions between the components are explained. Because the multiple interactions could lead to unstable feedbacks, we go through a careful process of model spin up to ensure that all components are stable and the interactions balanced. This spun-up configuration is evaluated against observed data for the Earth system components and is generally found to perform very satisfactorily. The reason for the evaluation phase is that the model is to be used for the core climate simulations carried out by the Met Office Hadley Centre for the Coupled Model Intercomparison Project (CMIP5), so it is essential that addition of the extra complexity does not detract substantially from its climate performance. Localised changes in some specific meteorological variables can be identified, but the impacts on the overall simulation of present day climate are slight. This model is proving valuable both for climate predictions, and for investigating the strengths of biogeochemical feedbacks.

Evaluation of ECMWF medium-range ensemble forecasts of precipitation for river basins

Providing probabilistic forecasts using Ensemble Prediction Systems has become increasingly popular in both the meteorological and hydrological communities. Compared to conventional deterministic forecasts, probabilistic forecasts may provide more reliable forecasts of a few hours to a number of days ahead, and hence are regarded as better tools for taking uncertainties into consideration and hedging against weather risks. It is essential to evaluate performance of raw ensemble forecasts and their potential values in forecasting extreme hydro-meteorological events. This study evaluates ECMWF’s medium-range ensemble forecasts of precipitation over the period 2008/01/01-2012/09/30 on a selected mid-latitude large scale river basin, the Huai river basin (ca. 270,000 km2) in central-east China. The evaluation unit is sub-basin in order to consider forecast performance in a hydrologically relevant way. The study finds that forecast performance varies with sub-basin properties, between flooding and non-flooding seasons, and with the forecast properties of aggregated time steps and lead times. Although the study does not evaluate any hydrological applications of the ensemble precipitation forecasts, its results have direct implications in hydrological forecasts should these ensemble precipitation forecasts be employed in hydrology.

Intercomparison and evaluation of global aerosol microphysical properties among AeroCom models of a range of complexity

Many of the next generation of global climate models will include aerosol schemes which explicitly simulate the microphysical processes that determine the particle size distribution. These models enable aerosol optical properties and cloud condensation nuclei (CCN) concentrations to be determined by fundamental aerosol processes, which should lead to a more physically based simulation of aerosol direct and indirect radiative forcings. This study examines the global variation in particle size distribution simulated by 12 global aerosol microphysics models to quantify model diversity and to identify any common biases against observations. Evaluation against size distribution measurements from a new European network of aerosol supersites shows that the mean model agrees quite well with the observations at many sites on the annual mean, but there are some seasonal biases common to many sites. In particular, at many of these European sites, the accumulation mode number concentration is biased low during winter and Aitken mode concentrations tend to be overestimated in winter and underestimated in summer. At high northern latitudes, the models strongly underpredict Aitken and accumulation particle concentrations compared to the measurements, consistent with previous studies that have highlighted the poor performance of global aerosol models in the Arctic. In the marine boundary layer, the models capture the observed meridional variation in the size distribution, which is dominated by the Aitken mode at high latitudes, with an increasing concentration of accumulation particles with decreasing latitude. Considering vertical profiles, the models reproduce the observed peak in total particle concentrations in the upper troposphere due to new particle formation, although modelled peak concentrations tend to be biased high over Europe. Overall, the multi-model-mean data set simulates the global variation of the particle size distribution with a good degree of skill, suggesting that most of the individual global aerosol microphysics models are performing well, although the large model diversity indicates that some models are in poor agreement with the observations. Further work is required to better constrain size-resolved primary and secondary particle number sources, and an improved understanding of nucleation and growth (e.g. the role of nitrate and secondary organics) will improve the fidelity of simulated particle size distributions.