Including or excluding conflicts of interest among expert peer reviewers had little impact on funding success, a case study from Australia

Competition for research funding is intense and the opinions of an expert peer reviewer can mean the difference between success and failure in securing funding. The allocation of expert peer reviewers is therefore vitally important and funding agencies strive to avoid using reviewers who have real or perceived conflicts of interest. This article examines the impact of including or excluding peer reviewers based on their conflicts of interest, and the final ranking of funding proposals. Two 7-person review panels assessed a sample of National Health and Medical Research Council (NHMRC) of Australia proposals in Basic Science or Public Health. Using a pre-post comparison, the proposals were first scored after the exclusion of reviewers with a high or medium conflict, and re-scored after the return of reviewers with medium conflicts. The main outcome measures are the agreements in ranks and funding success before and after excluding the medium conflicts. Including medium conflicts of interest had little impact on the ranks or funding success. The Bland–Altman 95% limits of agreement were ± 3.3 ranks and ± 3.4 ranks in the two panels which both assessed 36 proposals. Overall there were three proposals (4%) that had a reversed funding outcome after including medium conflicts. Relaxing the conflict of interest rules would increase the number of expert reviewers included in the panel discussions which could increase the quality of peer review and make it easier to find reviewers.

Overview and response to reviewers of The Singing Neanderthals

Why are humans musical? Why do people in all cultures sing or play instruments? Why do we appear to have specialized neurological apparatus for hearing and interpreting music as distinct from other sounds? And how does our musicality relate to language and to our evolutionary history? Anthropologists and archaeologists have paid little attention to the origin of music and musicality — far less than for either language or ‘art’. While art has been seen as an index of cognitive complexity and language as an essential tool of communication, music has suffered from our perception that it is an epiphenomenal ‘leisure activity’, and archaeologically inaccessible to boot. Nothing could be further from the truth, according to Steven Mithen; music is integral to human social life, he argues, and we can investigate its ancestry with the same rich range of analyses — neurological, physiological, ethnographic, linguistic, ethological and even archaeological — which have been deployed to study language. In The Singing Neanderthals Steven Mithen poses these questions and proposes a bold hypothesis to answer them. Mithen argues that musicality is a fundamental part of being human, that this capacity is of great antiquity, and that a holistic protolanguage of musical emotive expression predates language and was an essential precursor to it. This is an argument with implications which extend far beyond the mere origins of music itself into the very motives of human origins. Any argument of such range is bound to attract discussion and critique; we here present commentaries by archaeologists Clive Gamble and Iain Morley and linguists Alison Wray and Maggie Tallerman, along with Mithen's response to them. Whether right or wrong, Mithen has raised fascinating and important issues. And it adds a great deal of charm to the time-honoured, perhaps shopworn image of the Neanderthals shambling ineffectively through the pages of Pleistocene prehistory to imagine them humming, crooning or belting out a cappella harmonies as they went.

Ethnocentricity in academic marketing journals: a study of authors, reviewers, editorial boards and editors

Purpose – This paper aims to examine and compare a set of key characteristics of ethnocentricity that influence the policy of academic marketing journals, and hence the provenance, authorship and nature of articles in academic marketing journals.

Design/methodology/approach – The “fundamental” characteristics of three major marketing journals, published in the USA, the UK and New Zealand, were examined for the six-year period from the start of 2000 to the end of 2006. Data were collected from editorials and web homepages. Analysis was conducted of 811 articles, 1,676 authors, three editorial teams and three sets of reviewers

Findings – There is a challenging academic ethnocentricity in the management and implied policy of the three journals. The extent varies, but the inescapable conclusion is that the world-wide research community in marketing is not properly represented by leading journals.

Research limitations/implications – The sample was intentionally small, and unrepresentative of any category except “leading quality”. The findings are intended to add momentum to a debate and point ways forward, not to provide generalisable answers.

Practical implications – The findings suggest that: the editorial boards and reviewing teams should be made more representative geographically; editorships should be organized around the concept of a team of geographically differentiated editors; editorial and review teams should be ethnographically representative of individuals who do research and wish to publish it, particularly beyond the English-speaking world. In general, the world-wide research community in marketing would benefit from less ethnocentricity in academic journals, and these leading examples should strive to reduce it.

Originality/value – The impact of ethnocentricity is underestimated in this context. The issue needs to be discussed, because of paradigmatic influences that it can have on a journal and the profile of its authors, and hence on journal ranking and perceptions of journal quality.

Reviewing the reviewers : has the education review unit contributed to improved student learning outcomes?

This article looks at the contribution the South Australian Education Review Unit (ERU). It provides a perspective from a secondary school that has undergone and ERU review. It describes ERU procedures and methodology, explains what happened with the Le Fevre High School review, and 'reviews the reviewers' and answer the question posed in the article title.

Poor Reliability between Cochrane Reviewers and Blinded External Reviewers When Applying the Cochrane Risk of Bias Tool in Physical Therapy Trials

OBJECTIVES To test the inter-rater reliability of the RoB tool applied to Physical Therapy (PT) trials by comparing ratings from Cochrane review authors with those of blinded external reviewers. METHODS Randomized controlled trials (RCTs) in PT were identified by searching the Cochrane Database of Systematic Reviews for meta-analysis of PT interventions. RoB assessments were conducted independently by 2 reviewers blinded to the RoB ratings reported in the Cochrane reviews. Data on RoB assessments from Cochrane reviews and other characteristics of reviews and trials were extracted. Consensus assessments between the two reviewers were then compared with the RoB ratings from the Cochrane reviews. Agreement between Cochrane and blinded external reviewers was assessed using weighted kappa (κ). RESULTS In total, 109 trials included in 17 Cochrane reviews were assessed. Inter-rater reliability on the overall RoB assessment between Cochrane review authors and blinded external reviewers was poor (κ  =  0.02, 95%CI: -0.06, 0.06]). Inter-rater reliability on individual domains of the RoB tool was poor (median κ  = 0.19), ranging from κ  =  -0.04 ("Other bias") to κ  =  0.62 ("Sequence generation"). There was also no agreement (κ  =  -0.29, 95%CI: -0.81, 0.35]) in the overall RoB assessment at the meta-analysis level. CONCLUSIONS Risk of bias assessments of RCTs using the RoB tool are not consistent across different research groups. Poor agreement was not only demonstrated at the trial level but also at the meta-analysis level. Results have implications for decision making since different recommendations can be reached depending on the group analyzing the evidence. Improved guidelines to consistently apply the RoB tool and revisions to the tool for different health areas are needed.

Blinded by PRISMA: are systematic reviewers focusing on PRISMA and ignoring other guidelines?

BACKGROUND PRISMA guidelines have been developed to improve the reporting of systematic reviews (SRs). Other reporting guidelines and techniques to assess methodological quality of SRs have been developed. We aimed to assess the frequency of the use of reporting and other guidelines in SRs to assess whether PRISMA is being used inappropriately as a substitute for other relevant guidelines. METHODS Web of Knowledge was searched to identify articles citing the PRISMA guidelines over a 12-month period. The use of reporting guidelines (including PRISMA and MOOSE) and tools for assessing methodological quality (including QUADAS) was assessed. Factors associated with appropriate use of guidelines including review type, field of publication and involvement of a methodologist were investigated. RESULTS Over the 12-month period, 701 SRs were identified. MOOSE guidelines were cited in just 17% of epidemiologic reviews; QUADAS or QUADAS-2 was referred to in just 40% of diagnostic SRs. In the multivariable analysis, medical field of publication and methodologist involvement (OR = 1.97, 95% CI: 1.37, 2.83) were significant predictors of appropriate use of guidelines. Inclusion of a meta-analysis resulted in 73% higher odds of appropriate usage of systematic review guidelines (OR = 1.73, 95% CI: 1.22, 2.35). CONCLUSIONS Usage of SR reporting guidelines and tools for assessment of methodological quality other than PRISMA may be under-utilized with negative implications both for the reporting and methodological quality of systematic reviews.

Nonlesions, misdiagnoses, missed diagnoses, and other interpretive challenges in fish histopathology studies: a guide for investigators, authors, reviewers, and readers.

Differentiating salient histopathologic changes from normal anatomic features or tissue artifacts can be decidedly challenging, especially for the novice fish pathologist. As a consequence, findings of questionable accuracy may be reported inadvertently, and the potential negative impacts of publishing inaccurate histopathologic interpretations are not always fully appreciated. The objectives of this article are to illustrate a number of specific morphologic findings in commonly examined fish tissues (e.g., gills, liver, kidney, and gonads) that are frequently either misdiagnosed or underdiagnosed, and to address related issues involving the interpretation of histopathologic data. To enhance the utility of this article as a guide, photomicrographs of normal and abnormal specimens are presented. General recommendations for generating and publishing results from histopathology studies are additionally provided. It is hoped that the furnished information will be a useful resource for manuscript generation, by helping authors, reviewers, and readers to critically assess fish histopathologic data.