Strengthening impact evaluation in natural resource management

The workshop on Strengthening Evaluation in Natural Resource Management Research is part of an ACIAR-funded Small Research and Development Activity (SRA) on Assessing the Impacts of Natural Resource Management and Policy Research in Development Programs, with WorldFish and Commonwealth Scientific and Industrial Research Organization (CSIRO) as partners. The SRA objectives included a review of literature to identify challenges in assessing the impact of NRMR programs and to propose a framework that addresses them. An exploratory workshop was held in February 2012 to initiate collective action within the CGIAR Research Programs (CRPs) to identify and address their impact challenges and led to the creation of the NRMR impact community of practice (COP). This follow-up workshop brought together members of the COP and partners in the SRA to discuss and reach agreement on how to progress on our collective goals of building new and appropriate approaches for NRMR IE and how to put these approaches into action through our research programs. This report is a documentation of the workshop process and outputs.

Compiling Scientific Code Using Partial Evaluation

Scientists are faced with a dilemma: either they can write abstract programs that express their understanding of a problem, but which do not execute efficiently; or they can write programs that computers can execute efficiently, but which are difficult to write and difficult to understand. We have developed a compiler that uses partial evaluation and scheduling techniques to provide a solution to this dilemma.

Partial Evaluation for Scientific Computing: The Supercomputer Toolkit Experience

We describe the key role played by partial evaluation in the Supercomputer Toolkit, a parallel computing system for scientific applications that effectively exploits the vast amount of parallelism exposed by partial evaluation. The Supercomputer Toolkit parallel processor and its associated partial evaluation-based compiler have been used extensively by scientists at M.I.T., and have made possible recent results in astrophysics showing that the motion of the planets in our solar system is chaotically unstable.

The evaluation of time series: their scientific value and contribution to policy needs. Occasional Publication of the Marine Biological Association 22

In 2000 a Review of Current Marine Observations in relation to present and future needs was undertaken by the Inter-Agency Committee for Marine Science and Technology (IACMST). The Marine Environmental Change Network (MECN) was initiated in 2002 as a direct response to the recommendations of the report. A key part of the current phase of the MECN is to ensure that information from the network is provided to policy makers and other end-users to enable them to produce more accurate assessments of ecosystem state and gain a clearer understanding of factors influencing change in marine ecosystems. The MECN holds workshops on an annual basis, bringing together partners maintaining time-series and long-term datasets as well as end-users interested in outputs from the network. It was decided that the first workshop of the MECN continuation phase should consist of an evaluation of the time series and data sets maintained by partners in the MECN with regard to their ‘fit for purpose’ for answering key science questions and informing policy development. This report is based on the outcomes of the workshop. Section one of the report contains a brief introduction to monitoring, time series and long-term datasets. The various terms are defined and the need for MECN type data to complement compliance monitoring programmes is discussed. Outlines are also given of initiatives such as the United Kingdom Marine Monitoring and Assessment Strategy (UKMMAS) and Oceans 2025. Section two contains detailed information for each of the MECN time series / long-term datasets including information on scientific outputs and current objectives. This information is mainly based on the presentations given at the workshop and therefore follows a format whereby the following headings are addressed: Origin of time series including original objectives; current objectives; policy relevance; products (advice, publications, science and society). Section three consists of comments made by the review panel concerning all the time series and the network. Needs or issues highlighted by the panel with regard to the future of long-term datasets and time-series in the UK are shown along with advice and potential solutions where offered. The recommendations are divided into 4 categories; ‘The MECN and end-user requirements’; ‘Procedures & protocols’; ‘Securing data series’ and ‘Future developments’. Ever since marine environmental protection issues really came to the fore in the 1960s, it has been recognised that there is a requirement for a suitable evidence base on environmental change in order to support policy and management for UK waters. Section four gives a brief summary of the development of marine policy in the UK along with comments on the availability and necessity of long-term marine observations for the implementation of this policy. Policy relating to three main areas is discussed; Marine Conservation (protecting biodiversity and marine ecosystems); Marine Pollution and Fisheries. The conclusion of this section is that there has always been a specific requirement for information on long-term change in marine ecosystems around the UK in order to address concerns over pollution, fishing and general conservation. It is now imperative that this need is addressed in order for the UK to be able to fulfil its policy commitments and manage marine ecosystems in the light of climate change and other factors.

Evaluation de la douleur chez les personnes cérébrolésées non-communicantes aux soins intensifs

La douleur est fréquente en milieu de soins intensifs et sa gestion est l'une des missions des infirmières. Son évaluation est une prémisse indispensable à son soulagement. Cependant lorsque le patient est incapable de signaler sa douleur, les infirmières doivent se baser sur des signes externes pour l'évaluer. Les guides de bonne pratique recommandent chez les personnes non communicantes l'usage d'un instrument validé pour la population donnée et basé sur l'observation des comportements. A l'heure actuelle, les instruments d'évaluation de la douleur disponibles ne sont que partiellement adaptés aux personnes cérébrolésées dans la mesure où ces personnes présentent des comportements qui leur sont spécifiques. C'est pourquoi, cette étude vise à identifier, décrire et valider des indicateurs, et des descripteurs, de la douleur chez les personnes cérébrolésées. Un devis d'étude mixte multiphase avec une dominante quantitative a été choisi pour cette étude. Une première phase consistait à identifier des indicateurs et des descripteurs de la douleur chez les personnes cérébrolésées non communicantes aux soins intensifs en combinant trois sources de données : une revue intégrative des écrits, une démarche consultative utilisant la technique du groupe nominal auprès de 18 cliniciens expérimentés (6 médecins et 12 infirmières) et les résultats d'une étude pilote observationnelle réalisée auprès de 10 traumatisés crâniens. Les résultats ont permis d'identifier 6 indicateurs et 47 descripteurs comportementaux, vocaux et physiologiques susceptibles d'être inclus dans un instrument d'évaluation de la douleur destiné aux personnes cérébrolésées non- communicantes aux soins intensifs. Une deuxième phase séquentielle vérifiait les propriétés psychométriques des indicateurs et des descripteurs préalablement identifiés. La validation de contenu a été testée auprès de 10 experts cliniques et 4 experts scientifiques à l'aide d'un questionnaire structuré qui cherchait à évaluer la pertinence et la clarté/compréhensibilité de chaque descripteur. Cette démarche a permis de sélectionner 33 des 47 descripteurs et valider 6 indicateurs. Dans un deuxième temps, les propriétés psychométriques de ces indicateurs et descripteurs ont été étudiés au repos, lors de stimulation non nociceptive et lors d'une stimulation nociceptive (la latéralisation du patient) auprès de 116 personnes cérébrolésées aux soins intensifs hospitalisées dans deux centres hospitaliers universitaires. Les résultats montrent d'importantes variations dans les descripteurs observés lors de stimulation nociceptive probablement dues à l'hétérogénéité des patients au niveau de leur état de conscience. Dix descripteurs ont été éliminés, car leur fréquence lors de la stimulation nociceptive était inférieure à 5% ou leur fiabilité insuffisante. Les descripteurs physiologiques ont tous été supprimés en raison de leur faible variabilité et d'une fiabilité inter juge problématique. Les résultats montrent que la validité concomitante, c'est-à-dire la corrélation entre l'auto- évaluation du patient et les mesures réalisées avec les descripteurs, est satisfaisante lors de stimulation nociceptive {rs=0,527, p=0,003, n=30). Par contre la validité convergente, qui vérifiait l'association entre l'évaluation de la douleur par l'infirmière en charge du patient et les mesures réalisés avec les descripteurs, ainsi que la validité divergente, qui vérifiait si les indicateurs discriminent entre la stimulation nociceptive et le repos, mettent en évidence des résultats variables en fonction de l'état de conscience des patients. Ces résultats soulignent la nécessité d'étudier les descripteurs de la douleur chez des patients cérébrolésés en fonction du niveau de conscience et de considérer l'hétérogénéité de cette population dans la conception d'un instrument d'évaluation de la douleur pour les personnes cérébrolésées non communicantes aux soins intensifs. - Pain is frequent in the intensive care unit (ICU) and its management is a major issue for nurses. The assessment of pain is a prerequisite for appropriate pain management. However, pain assessment is difficult when patients are unable to communicate about their experience and nurses have to base their evaluation on external signs. Clinical practice guidelines highlight the need to use behavioral scales that have been validated for nonverbal patients. Current behavioral pain tools for ICU patients unable to communicate may not be appropriate for nonverbal brain-injured ICU patients, as they demonstrate specific responses to pain. This study aimed to identify, describe and validate pain indicators and descriptors in brain-injured ICU patients. A mixed multiphase method design with a quantitative dominant was chosen for this study. The first phase aimed to identify indicators and descriptors of pain for nonverbal brain- injured ICU patients using data from three sources: an integrative literature review, a consultation using the nominal group technique with 18 experienced clinicians (12 nurses and 6 physicians) and the results of an observational pilot study with 10 traumatic brain injured patients. The results of this first phase identified 6 indicators and 47 behavioral, vocal and physiological descriptors of pain that could be included in a pain assessment tool for this population. The sequential phase two tested the psychometric properties of the list of previously identified indicators and descriptors. Content validity was tested with 10 clinical and 4 scientific experts for pertinence and comprehensibility using a structured questionnaire. This process resulted in 33 descriptors to be selected out of 47 previously identified, and six validated indicators. Then, the psychometric properties of the descriptors and indicators were tested at rest, during non nociceptive stimulation and nociceptive stimulation (turning) in a sample of 116 brain-injured ICLI patients who were hospitalized in two university centers. Results showed important variations in the descriptors observed during the nociceptive stimulation, probably due to the heterogeneity of patients' level of consciousness. Ten descriptors were excluded, as they were observed less than 5% of the time or their reliability was insufficient. All physiologic descriptors were deleted as they showed little variability and inter observer reliability was lacking. Concomitant validity, testing the association between patients' self report of pain and measures performed using the descriptors, was acceptable during nociceptive stimulation (rs=0,527, p=0,003, n=30). However, convergent validity ( testing for an association between the nurses' pain assessment and measures done with descriptors) and divergent validity (testing for the ability of the indicators to discriminate between rest and a nociceptive stimulation) varied according to the level of consciousness These results highlight the need to study pain descriptors in brain-injured patients with different level of consciousness and to take into account the heterogeneity of this population forthe conception of a pain assessment tool for nonverbal brain-injured ICU patients.

Strategies towards evaluation beyond scientific impact

Various research fields, like organic agricultural research, are dedicated to solving real-world problems and contributing to sustainable development. Therefore, systems research and the application of interdisciplinary and transdisciplinary approaches are increasingly endorsed. However, research performance depends not only on self-conception, but also on framework conditions of the scientific system, which are not always of benefit to such research fields. Recently, science and its framework conditions have been under increasing scrutiny as regards their ability to serve societal benefit. This provides opportunities for (organic) agricultural research to engage in the development of a research system that will serve its needs. This article focuses on possible strategies for facilitating a balanced research evaluation that recognises scientific quality as well as societal relevance and applicability. These strategies are (a) to strengthen the general support for evaluation beyond scientific impact, and (b) to provide accessible data for such evaluations. Synergies of interest are found between open access movements and research communities focusing on global challenges and sustainability. As both are committed to increasing the societal benefit of science, they may support evaluation criteria such as knowledge production and dissemination tailored to societal needs, and the use of open access. Additional synergies exist between all those who scrutinise current research evaluation systems for their ability to serve scientific quality, which is also a precondition for societal benefit. Here, digital communication technologies provide opportunities to increase effectiveness, transparency, fairness and plurality in the dissemination of scientific results, quality assurance and reputation. Furthermore, funders may support transdisciplinary approaches and open access and improve data availability for evaluation beyond scientific impact. If they begin to use current research information systems that include societal impact data while reducing the requirements for narrative reports, documentation burdens on researchers may be relieved, with the funders themselves acting as data providers for researchers, institutions and tailored dissemination beyond academia.

A quantitative and qualitative evaluation of reports of clinical trials published in six Brazilian dental journals indexed in the Scientific Electronic Library Online (SciELO)

Introduction: Open access publishing is becoming increasingly popular within the biomedical sciences. SciELO, the Scientific Electronic Library Online, is a digital library covering a selected collection of Brazilian scientific journals many of which provide open access to full-text articles. This library includes a number of dental journals some of which may include reports of clinical trials in English, Portuguese and/or Spanish. Thus, SciELO could play an important role as a source of evidence for dental healthcare interventions especially if it yields a sizeable number of high quality reports. Objective: The aim of this study was to identify reports of clinical trials by handsearching of dental journals that are accessible through SciELO, and to assess the overall quality of these reports. Material and methods: Electronic versions of six Brazilian dental Journals indexed in SciELO were handsearched at www.scielo.br in September 2008. Reports of clinical trials were identified and classified as controlled clinical trials (CCTs - prospective, experimental studies comparing 2 or more healthcare interventions in human beings) or randomized controlled trials (RCTs - a random allocation method is clearly reported), according to Cochrane eligibility criteria. Criteria to assess methodological quality included: method of randomization, concealment of treatment allocation, blinded outcome assessment, handling of withdrawals and losses and whether an intention-to-treat analysis had been carried out. Results: The search retrieved 33 CCTs and 43 RCTs. A majority of the reports provided no description of either the method of randomization (75.3%) or concealment of the allocation sequence (84.2%). Participants and outcome assessors were reported as blinded in only 31.2% of the reports. Withdrawals and losses were only clearly described in 6.5% of the reports and none mentioned an intention-to-treat analysis or any similar procedure. Conclusions: The results of this study indicate that a substantial number of reports of trials and systematic reviews are available in the dental journals listed in SciELO, and that these could provide valuable evidence for clinical decision making. However, it is clear that the quality of a number of these reports is of some concern and that improvement in the conduct and reporting of these trials could be achieved if authors adhered to internationally accepted guidelines, e. g. the CONSORT statement.

Evaluation of race distance, track surface and season of the year on exercise-induced pulmonary haemorrhage in flat racing Thoroughbreds in Brazil

Reasons for performing study: To our knowledge no studies evaluating risk factors to EIPH have been undertaken with such a population of horses in Brazil, which is one of the few countries in the racing community to accept the use of frusemide (FUR) as a prerace medication for bleeders. We also compared national results with those already in the literature. Objectives: To verify the influence of variables such as distance of the race, racetrack surface, going and season of the year in the recurrence and severity of EIPH and its effect on the finishing position of flat racing Thoroughbreds in Brazil. Methods: 2118 post race respiratory endoscopies were recorded, in a total of 1003 individual horses. All horses in the study were certified bleeders (deemed positive by registered veterinarians following one or more endoscopies) and were administered FUR prior to each race. Data was analysed using both ordinary logistic regression and multiple logistic regression. Results: The results suggest that race distance, season of the year and racetrack surface affect the recurrence of EIPH. Also, despite the use of FUR, the majority (62%) of horses continued to display some degree of haemorrhage. Conclusions: There was a tendency for horses that remained positive despite the use of FUR to finish unplaced in a race. Horses which had a significant reduction in bleeding and became HPIE negative after the use of FUR, tended to finish in top positions. It was not possible to determine a pattern of severity of EIPH in horses with more than one endoscopy. Further studies are required to improve our understanding of this syndrome and its correlations to performance in the racing Thoroughbred, together with the role of FUR and its possible enhancing performance effects and efficacy in consistently diminishing EIPH. Potential relevance: This study enabled a better understanding of the use of FUR in Brazilian Thoroughbred racehorses and showed there was a tendency towards improvement in performance after administrion of FUR in bleeders. The fact that a large number of horses remained positive for EIPH despite the use of FUR should encourage the scientific community to search for better ways of reducing, or preventing, this syndrome. © 2006 EVJ Ltd.

Scientific article in vitro evaluation of the influence of air abrasion on detection of occlusal caries lesions in primary teeth

The purpose of this study was to assess the influence of cleaning pits and fissures with an aluminum oxide air abrasion system on the detection of occlusal caries in primary teeth using laser fluorescence (LF) and visual examination. Methods: The sample comprised 65 pit and fissure sites on extracted primary teeth suspected to be carious. The sites were submitted to 2 visual examinations (examiner JAR) and 2 LF readings (examiner TMV). Next, the occlusal surfaces were air-abraded and re-examined thereafter using both methods. The teeth were sectioned, and the histological analysis of the sites with a stereoscopic magnifying lens at X32 magnifi cation was used as the gold standard. Results: Cohen's kappa statistic for LF and visual examination were, respectively, 0.282/0.884 before and 0.896/0.905 after air abrasion. LF showed a sensitivity of 0.28 increasing to 0.49 and a specifi city of 0.50 increasing to 0.92. Visual examination showed sensitivity of 0.78 and specifi city of 0.73. Both increased after air abrasion. Conclusion: The findings suggest that cleaning pits and fissures with aluminum oxide air abrasion increased the accuracy of LF and visual examination for detection of occlusal caries in primary teeth.

Methods for evaluation of masticatory efficiency in conventional complete denture wearers: a systematized review

The objective of this study was to present a systematized review of different methods used to evaluate the masticatory efficiency in conventional complete denture wearers. A survey was conducted in the databases PubMed, Scopus, and Cochrane, seeking scientific articles according to the previously selected terms: "Masticatory performance", "Masticatory efficiency" and "Chewing ability complete denture". Moreover, complementary studies have been carried out with library manual search/databases, which included studies related to different ways to assess masticatory efficiency, specifically as it related to conventional complete denture wearers. Forty three papers were selected to be used in the present review. Despite the wide variety of methodologies in the literature, the sieves method is currently considered the gold standard method to evaluation of conventional complete denture wearers masticatory efficiency, since it is the simplest, does not depend on specific devices (beyond the set of sieves), allows for a rational assessment, and it has been widely reproduced in various types of oral rehabilitation. More, the almond, as natural test food, and the optocal (made from the molding material Optosil), as artificial test food, are the most constantly employed test foods to evaluate masticatory efficiency.