Including or excluding conflicts of interest among expert peer reviewers had little impact on funding success, a case study from Australia

Competition for research funding is intense and the opinions of an expert peer reviewer can mean the difference between success and failure in securing funding. The allocation of expert peer reviewers is therefore vitally important and funding agencies strive to avoid using reviewers who have real or perceived conflicts of interest. This article examines the impact of including or excluding peer reviewers based on their conflicts of interest, and the final ranking of funding proposals. Two 7-person review panels assessed a sample of National Health and Medical Research Council (NHMRC) of Australia proposals in Basic Science or Public Health. Using a pre-post comparison, the proposals were first scored after the exclusion of reviewers with a high or medium conflict, and re-scored after the return of reviewers with medium conflicts. The main outcome measures are the agreements in ranks and funding success before and after excluding the medium conflicts. Including medium conflicts of interest had little impact on the ranks or funding success. The Bland–Altman 95% limits of agreement were ± 3.3 ranks and ± 3.4 ranks in the two panels which both assessed 36 proposals. Overall there were three proposals (4%) that had a reversed funding outcome after including medium conflicts. Relaxing the conflict of interest rules would increase the number of expert reviewers included in the panel discussions which could increase the quality of peer review and make it easier to find reviewers.