Quantifying Reproducibility in Computational Biology: The Case of the Tuberculosis Drugome

How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved. Quantification is achieved by estimating the time required to reproduce each of the steps in the method described in the original paper and make them part of an explicit workflow that reproduces the original results. Reproducing the method took several months of effort, and required using new versions and new software that posed challenges to reconstructing and validating the results. The quantification leads to “reproducibility maps” that reveal that novice researchers would only be able to reproduce a few of the steps in the method, and that only expert researchers with advance knowledge of the domain would be able to reproduce the method in its entirety. The workflow itself is published as an online resource together with supporting software and data. The paper concludes with a brief discussion of the complexities of requiring reproducibility in terms of cost versus benefit, and a desiderata with our observations and guidelines for improving reproducibility. This has implications not only in reproducing the work of others from published papers, but reproducing work from one’s own laboratory.

Census, reproductive biology, and germination of Astragalus gines-lopezii (Fabaceae), a narrow and endangered endemic species of SW Spain

Astragalus gines-lopezii Talavera, Podlech, Devesa & F.M.Vázquez (Fabaceae) is a threatened endemic species with a distribution restricted to a very small area in Badajoz Province (Extremadura Region, SW Spain) and only 2 populations are known. This species was catalogued in the ?Endangered? category in the 2008 Red List and the 2010 Threatened Spanish Vascular Flora List. Despite its status as an endangered species, at present very little is known about the distribution, census, and reproductive biology of this species. In this study we have carried out an exhaustive census of A. gines-lopezii , and we have evaluated the production of flowers, fruits, and seeds and the existence or not of intra- and interpopulation variability in seed germination. Results have highlighted the high reproductive capacity of this species on the basis of a high production of flowers, fruits, and seeds. Mechanical scarification of seeds was effective for increasing germination. Thus, initial germination (22%?60%) was increased to 97%?99% when seeds were rubbed with sandpapers. A high intra- and interpopulation variability in seed germination was found in this species. A. gines-lopezii produces seeds with different degrees of physical dormancy, varying this grade among different individuals within a population.

Census, reproductive biology, and germination of Astragalus gines-lopezii (Fabaceae),a narrow and endangered endemic species of SW Spain.

Astragalus gines-lopezii Talavera, Podlech, Devesa & F.M.Vazquez (Fabaceae) is a threatened endemic species with a distribution restricted to a very small area in Badajoz Province (Extremadura Region, SW Spain) and only 2 populations are known.This species was catalogued in the "Endangered" category in the 2008 Red List and the 2010 Threatened Spanish Vascular Flora List. Despite its status as an endangered species, at present very little is known about the distribution, census, and reproductive biology of this species. In this study we have carried out anexhaustive census of A. gines-lopezii, and we have evaluated the production of flowers, fruits, and seeds and the existence or not of intra- and interpopulation variability in seed germination. Results have highlighted the high reproductive capacity of this species on the basis of a high production of flowers, fruits, and seeds. Mechanical scarification of seeds was effective for increasing germination. Thus, initial germination (22%-60%) was increased to 97%-99% when seeds were rubbed with sandpapers. A high intra- and interpopulation variability in seed germination was found in this species. A. gines-lopezii produces seeds with different degrees of physical dormancy, varying this grade among different individuals within a population.