Preparing for CRIS: Challenges and Opportunities for Systems Integration at Finnish Universities

Poster at Open Repositories 2014, Helsinki, Finland, June 9-13, 2014

Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review.

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS: In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Proposed best practice for statisticians in the reporting and publication of pharmaceutical industry-sponsored clinical trials

In this paper we set out what we consider to be a set of best practices for statisticians in the reporting of pharmaceutical industry-sponsored clinical trials. We make eight recommendations covering: author responsibilities and recognition; publication timing; conflicts of interest; freedom to act; full author access to data; trial registration and independent review. These recommendations are made in the context of the prominent role played by statisticians in the design, conduct, analysis and reporting of pharmaceutical sponsored trials and the perception of the reporting of these trials in the wider community.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Ethical dilemmas in scientific publication: pitfalls and solutions for editors

Editors of scientific journals need to be conversant with the mechanisms by which scientific misconduct is amplified by publication practices. This paper provides definitions, ways to document the extent of the problem, and examples of editorial attempts to counter fraud. Fabrication, falsification, duplication, ghost authorship, gift authorship, lack of ethics approval, non-disclosure, 'salami' publication, conflicts of interest, auto-citation, duplicate submission, duplicate publications, and plagiarism are common problems. Editorial misconduct includes failure to observe due process, undue delay in reaching decisions and communicating these to authors, inappropriate review procedures, and confounding a journal's content with its advertising or promotional potential. Editors also can be admonished by their peers for failure to investigate suspected misconduct, failure to retract when indicated, and failure to abide voluntarily by the six main sources of relevant international guidelines on research, its reporting and editorial practice. Editors are in a good position to promulgate reasonable standards of practice, and can start by using consensus guidelines on publication ethics to state explicitly how their journals function. Reviewers, editors, authors and readers all then have a better chance to understand, and abide by, the rules of publishing.

Prevalence, characteristics, and publication of discontinued randomized trials.

IMPORTANCE: The discontinuation of randomized clinical trials (RCTs) raises ethical concerns and often wastes scarce research resources. The epidemiology of discontinued RCTs, however, remains unclear. OBJECTIVES: To determine the prevalence, characteristics, and publication history of discontinued RCTs and to investigate factors associated with RCT discontinuation due to poor recruitment and with nonpublication. DESIGN AND SETTING: Retrospective cohort of RCTs based on archived protocols approved by 6 research ethics committees in Switzerland, Germany, and Canada between 2000 and 2003. We recorded trial characteristics and planned recruitment from included protocols. Last follow-up of RCTs was April 27, 2013. MAIN OUTCOMES AND MEASURES: Completion status, reported reasons for discontinuation, and publication status of RCTs as determined by correspondence with the research ethics committees, literature searches, and investigator surveys. RESULTS: After a median follow-up of 11.6 years (range, 8.8-12.6 years), 253 of 1017 included RCTs were discontinued (24.9% [95% CI, 22.3%-27.6%]). Only 96 of 253 discontinuations (37.9% [95% CI, 32.0%-44.3%]) were reported to ethics committees. The most frequent reason for discontinuation was poor recruitment (101/1017; 9.9% [95% CI, 8.2%-12.0%]). In multivariable analysis, industry sponsorship vs investigator sponsorship (8.4% vs 26.5%; odds ratio [OR], 0.25 [95% CI, 0.15-0.43]; P < .001) and a larger planned sample size in increments of 100 (-0.7%; OR, 0.96 [95% CI, 0.92-1.00]; P = .04) were associated with lower rates of discontinuation due to poor recruitment. Discontinued trials were more likely to remain unpublished than completed trials (55.1% vs 33.6%; OR, 3.19 [95% CI, 2.29-4.43]; P < .001). CONCLUSIONS AND RELEVANCE: In this sample of trials based on RCT protocols from 6 research ethics committees, discontinuation was common, with poor recruitment being the most frequently reported reason. Greater efforts are needed to ensure the reporting of trial discontinuation to research ethics committees and the publication of results of discontinued trials.

Editorial research and the publication process in biomedicine and health: Report from the Esteve Foundation Discussion Group, December 2012.

Despite the fact that there are more than twenty thousand biomedical journals in the world, research into the work of editors and publication process in biomedical and health care journals is rare. In December 2012, the Esteve Foundation, a non-profit scientific institution that fosters progress in pharmacotherapy by means of scientific communication and discussion organized a discussion group of 7 editors and/or experts in peer review biomedical publishing. They presented findings of past editorial research, discussed the lack of competitive funding schemes and specialized journals for dissemination of editorial research, and reported on the great diversity of misconduct and conflict of interest policies, as well as adherence to reporting guidelines. Furthermore, they reported on the reluctance of editors to investigate allegations of misconduct or increase the level of data sharing in health research. In the end, they concluded that if editors are to remain gatekeepers of scientific knowledge they should reaffirm their focus on the integrity of the scientific record and completeness of the data they publish. Additionally, more research should be undertaken to understand why many journals are not adhering to editorial standards, and what obstacles editors face when engaging in editorial research.

Publication bias in animal research: a systematic review protocol.

BACKGROUND: Systematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular. METHODS/DESIGN: The objectives of this systematic review are as follows: âeuro¢To investigate the epidemiology of published systematic reviews of animal studies until present. âeuro¢To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias. âeuro¢To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies. Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior. In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions. Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews. DISCUSSION: Results are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.

Detecting, quantifying and adjusting for publication bias in meta-analyses: protocol of a systematic review on methods.

BACKGROUND: Health professionals and policymakers aspire to make healthcare decisions based on the entire relevant research evidence. This, however, can rarely be achieved because a considerable amount of research findings are not published, especially in case of 'negative' results - a phenomenon widely recognized as publication bias. Different methods of detecting, quantifying and adjusting for publication bias in meta-analyses have been described in the literature, such as graphical approaches and formal statistical tests to detect publication bias, and statistical approaches to modify effect sizes to adjust a pooled estimate when the presence of publication bias is suspected. An up-to-date systematic review of the existing methods is lacking. METHODS/DESIGN: The objectives of this systematic review are as follows:âeuro¢ To systematically review methodological articles which focus on non-publication of studies and to describe methods of detecting and/or quantifying and/or adjusting for publication bias in meta-analyses.âeuro¢ To appraise strengths and weaknesses of methods, the resources they require, and the conditions under which the method could be used, based on findings of included studies.We will systematically search Web of Science, Medline, and the Cochrane Library for methodological articles that describe at least one method of detecting and/or quantifying and/or adjusting for publication bias in meta-analyses. A dedicated data extraction form is developed and pilot-tested. Working in teams of two, we will independently extract relevant information from each eligible article. As this will be a qualitative systematic review, data reporting will involve a descriptive summary. DISCUSSION: Results are expected to be publicly available in mid 2013. This systematic review together with the results of other systematic reviews of the OPEN project (To Overcome Failure to Publish Negative Findings) will serve as a basis for the development of future policies and guidelines regarding the assessment and handling of publication bias in meta-analyses.

Myynnin raportoinnin käytettävyyden parantaminen toiminnanohjausjärjestelmämuutoksen jälkeen, Case: Stora Enso Publication Papers Oy Ltd

Tutkielman tavoitteena oli parantaa kohdeyrityksen myynnin raportoinnin käytettävyyttä toiminnanohjausjärjestelmämuutoksen jälkeen. Järjestelmämuutoksen jälkeen myyntitiedot siirtyivät puutteellisesti yrityksen tuloslaskentajärjestelmään. Tutkimus on empiriaan nojautuva tapaustutkimus. Siinätarkastellaan myynnin raportoinnin merkitystä yrityksen taloushallinnon raportoinnin osana sekä keinoja raportoinnin käytettävyyden varmistamiseksi. Lisäksi työssä käsitellään tietojärjestelmien osuutta taloushallinnon raportoinnissa ja tarkastellaan kohdeyrityksen tietojärjestelmiä ja niiden yhteistoimintaa. Empiirisen tutkimuksen tuloksena myyntitietojen puutteellisen siirtymisen syyksi havaittiin virheellinen poimintamääritys järjestelmien välillä.

Statistical Reviewers Improve Reporting in Biomedical Articles: a Randomized Trial.

Background. Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. Methodology and Principal Findings. Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with"no statistical expert" and"no checklist" as controls. The two interventions were crossed in a 262 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6- 24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: 20.3+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3-6.7), showing a significant improvement in quality. Conclusions and Significance. This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.

Planning and reporting of quality-of-life outcomes in cancer trials.

BACKGROUND: Information about the impact of cancer treatments on patients' quality of life (QoL) is of paramount importance to patients and treating oncologists. Cancer trials that do not specify QoL as an outcome or fail to report collected QoL data, omit crucial information for decision making. To estimate the magnitude of these problems, we investigated how frequently QoL outcomes were specified in protocols of cancer trials and subsequently reported. DESIGN: Retrospective cohort study of RCT protocols approved by six research ethics committees in Switzerland, Germany, and Canada between 2000 and 2003. We compared protocols to corresponding publications, which were identified through literature searches and investigator surveys. RESULTS: Of the 173 cancer trials, 90 (52%) specified QoL outcomes in their protocol, 2 (1%) as primary and 88 (51%) as secondary outcome. Of the 173 trials, 35 (20%) reported QoL outcomes in a corresponding publication (4 modified from the protocol), 18 (10%) were published but failed to report QoL outcomes in the primary or a secondary publication, and 37 (21%) were not published at all. Of the 83 (48%) trials that did not specify QoL outcomes in their protocol, none subsequently reported QoL outcomes. Failure to report pre-specified QoL outcomes was not associated with industry sponsorship (versus non-industry), sample size, and multicentre (versus single centre) status but possibly with trial discontinuation. CONCLUSIONS: About half of cancer trials specified QoL outcomes in their protocols. However, only 20% reported any QoL data in associated publications. Highly relevant information for decision making is often unavailable to patients, oncologists, and health policymakers.

Methodology used in studies reporting chronic kidney disease prevalence : a systematic literature review.

BACKGROUND: Many publications report the prevalence of chronic kidney disease (CKD) in the general population. Comparisons across studies are hampered as CKD prevalence estimations are influenced by study population characteristics and laboratory methods. METHODS: For this systematic review, two researchers independently searched PubMed, MEDLINE and EMBASE to identify all original research articles that were published between 1 January 2003 and 1 November 2014 reporting the prevalence of CKD in the European adult general population. Data on study methodology and reporting of CKD prevalence results were independently extracted by two researchers. RESULTS: We identified 82 eligible publications and included 48 publications of individual studies for the data extraction. There was considerable variation in population sample selection. The majority of studies did not report the sampling frame used, and the response ranged from 10 to 87%. With regard to the assessment of kidney function, 67% used a Jaffe assay, whereas 13% used the enzymatic assay for creatinine determination. Isotope dilution mass spectrometry calibration was used in 29%. The CKD-EPI (52%) and MDRD (75%) equations were most often used to estimate glomerular filtration rate (GFR). CKD was defined as estimated GFR (eGFR) <60 mL/min/1.73 m(2) in 92% of studies. Urinary markers of CKD were assessed in 60% of the studies. CKD prevalence was reported by sex and age strata in 54 and 50% of the studies, respectively. In publications with a primary objective of reporting CKD prevalence, 39% reported a 95% confidence interval. CONCLUSIONS: The findings from this systematic review showed considerable variation in methods for sampling the general population and assessment of kidney function across studies reporting CKD prevalence. These results are utilized to provide recommendations to help optimize both the design and the reporting of future CKD prevalence studies, which will enhance comparability of study results.

Publication Bias in Five Dental Implant Journals: An Observation from 2005 to 2009

Purpose: This study evaluated possible publication bias and its related factors in implant-related research over time. Materials and Methods: Articles published in Clinical Implant Dentistry and Related Research, Clinical Oral Implants Research, Implant Dentistry, Journal of Oral Implantology, and The International Journal of Oral & Maxillofacial Implants between 2005 and 2009 were reviewed. Nonoriginal articles were excluded. For each article included, study outcome, extramural funding source, type of study, and geographic origin were recorded. Descriptive and analytic statistics (alpha = .05), including the chi-square test and logistic regression analysis, were performed where appropriate. Results: From a total of 2,085 articles, 1,503 met the inclusion criteria. of the articles analyzed, 1,226 (81.6%), 160 (10.6%), and 117 (7.8%) articles reported positive, negative, and neutral outcomes, respectively. In vitro studies, studies from Asia, and funded animal studies were more likely to report positive outcomes compared to others (P = .02, P < .0001, and P = .009, respectively). Industry-funded studies represented the lowest frequency of positive outcomes versus studies funded by other sources. Conclusions: There were a high number of implant-related studies reporting positive outcomes in the five selected journals. Some selected factors were associated with positive outcome bias. In general, funding was not associated with a positive outcome, except for animal studies. Industry-supported research did not show any association with the publication of positive outcomes. INT J ORAL MAXILLOFAC IMPLANTS 2011;26:1024-1032

Information access law and disclosing of information by joint-stock companies in Brazil: open data and Extensible Business Reporting Language (XBRL) standard

The strategic management of information plays a fundamental role in the organizational management process since the decision-making process depend on the need for survival in a highly competitive market. Companies are constantly concerned about information transparency and good practices of corporate governance (CG) which, in turn, directs relations between the controlling power of the company and investors. In this context, this article presents the relationship between the disclosing of information of joint-stock companies by means of using XBRL, the open data model adopted by the Brazilian government, a model that boosted the publication of Information Access Law (Lei de Acesso à Informação), nº 12,527 of 18 November 2011. Information access should be permeated by a mediation policy in order to subsidize the knowledge construction and decision-making of investors. The XBRL is the main model for the publishing of financial information. The use of XBRL by means of new semantic standard created for Linked Data, strengthens the information dissemination, as well as creates analysis mechanisms and cross-referencing of data with different open databases available on the Internet, providing added value to the data/information accessed by civil society.