Pre-validation methods for developing a patient reported outcome instrument

BACKGROUND: Measures that reflect patients' assessment of their health are of increasing importance as outcome measures in randomised controlled trials. The methodological approach used in the pre-validation development of new instruments (item generation, item reduction and question formatting) should be robust and transparent. The totality of the content of existing PRO instruments for a specific condition provides a valuable resource (pool of items) that can be utilised to develop new instruments. Such 'top down' approaches are common, but the explicit pre-validation methods are often poorly reported. This paper presents a systematic and generalisable 5-step pre-validation PRO instrument methodology.

METHODS: The method is illustrated using the example of the Aberdeen Glaucoma Questionnaire (AGQ). The five steps are: 1) Generation of a pool of items; 2) Item de-duplication (three phases); 3) Item reduction (two phases); 4) Assessment of the remaining items' content coverage against a pre-existing theoretical framework appropriate to the objectives of the instrument and the target population (e.g. ICF); and 5) qualitative exploration of the target populations' views of the new instrument and the items it contains.

RESULTS: The AGQ 'item pool' contained 725 items. Three de-duplication phases resulted in reduction of 91, 225 and 48 items respectively. The item reduction phases discarded 70 items and 208 items respectively. The draft AGQ contained 83 items with good content coverage. The qualitative exploration ('think aloud' study) resulted in removal of a further 15 items and refinement to the wording of others. The resultant draft AGQ contained 68 items.

CONCLUSIONS: This study presents a novel methodology for developing a PRO instrument, based on three sources: literature reporting what is important to patient; theoretically coherent framework; and patients' experience of completing the instrument. By systematically accounting for all items dropped after the item generation phase, our method ensures that the AGQ is developed in a transparent, replicable manner and is fit for validation. We recommend this method to enhance the likelihood that new PRO instruments will be appropriate to the research context in which they are used, acceptable to research participants and likely to generate valid data.

Patient-reported outcomes in randomised controlled trials on age-related macular degeneration

BACKGROUND/AIMS: The purpose of this systematic review was to identify the frequency and type of patient-reported outcome measures (PROMs) used in recent randomised controlled trials (RCTs) for age-related macular degeneration (AMD).

METHODS: The authors conducted a systematic search between January 2010 and November 2013 in MEDLINE, EMBASE, Scopus, Cochrane Library (Central) and the clinical trials registries (http://www.controlled-trials.com and http://www.ClinicalTrials.gov) according to defined inclusion criteria (RCTs on AMD in English). Two independent reviewers evaluated studies for inclusion. One reviewer extracted data of included studies, and a second masked reviewer assessed 10% to confirm accuracy in data collection. Reference lists of included papers and appendices of relevant Cochrane systematic reviews were scanned to identify other relevant RCTs. Information collected on extracted outcomes was analysed using descriptive statistics.

RESULTS: Literature and registry search yielded 3816 abstracts of journal articles and 493 records from trial registries. A total of 177 RCTs were deemed to have met inclusion criteria. Of the 858 outcomes reported, 38 outcomes were identified as PROMs (4.4%). Of the 177 RCTs examined, PROMs were used in 25 trials (14.1%). The National Eye Institute Visual Function Questionnaire-25 was the most frequently used PROM instrument (64% of RCTs with PROMs included).

CONCLUSIONS: This review highlights that a small proportion of AMD RCTs included PROMs as outcome measures and that there was a variety in the instruments used.

Systematic evaluation of patient-reported outcome (PRO) protocol content and reporting in UK cancer clinical trials: the EPiC study protocol

Introduction Emerging evidence suggests that patient-reported outcome (PRO)-specific information may be omitted in trial protocols and that PRO results are poorly reported, limiting the use of PRO data to inform cancer care. This study aims to evaluate the standards of PRO-specific content in UK cancer trial protocols and their arising publications and to highlight examples of best-practice PRO protocol content and reporting where they occur. The objective of this study is to determine if these early findings are generalisable to UK cancer trials, and if so, how best we can bring about future improvements in clinical trials methodology to enhance the way PROs are assessed, managed and reported. Hypothesis: Trials in which the primary end point is based on a PRO will have more complete PRO protocol and publication components than trials in which PROs are secondary end points.

Methods and analysis Completed National Institute for Health Research (NIHR) Portfolio Cancer clinical trials (all cancer specialities/age-groups) will be included if they contain a primary/secondary PRO end point. The NIHR portfolio includes cancer trials, supported by a range of funders, adjudged as high-quality clinical research studies. The sample will be drawn from studies completed between 31 December 2000 and 1 March 2014 (n=1141) to allow sufficient time for completion of the final trial report and publication. Two reviewers will then review the protocols and arising publications of included trials to: (1) determine the completeness of their PRO-specific protocol content; (2) determine the proportion and completeness of PRO reporting in UK Cancer trials and (3) model factors associated with PRO protocol and reporting completeness and with PRO reporting proportion.

Ethics and dissemination The study was approved by the ethics committee at University of Birmingham (ERN_15-0311). Trial findings will be disseminated via presentations at local, national and international conferences, peer-reviewed journals and social media including the CPROR twitter account and UOB departmental website (http://www.birmingham.ac.uk/cpro0r).

Using postal questionnaires to evaluate physical activity and diet behaviour change: case study exploring implications of valid responder characteristics in interpreting intervention outcomes.

Background: Patient reported outcome measures (PROMs) are used to evaluate lifestyle interventions but littleis known about differences between patients returning valid and invalid responses, or of potential for bias inevaluations. We aimed to examine the characteristics of patients who returned valid responses to lifestylequestionnaires compared to those whose responses were invalid for evaluating lifestyle change. 
Methods: We conducted a secondary data analysis from the SPHERE Study, a trial of an intervention to improveoutcomes for patients with coronary heart disease in primary care. Postal questionnaires were used to assessphysical activity (Godin) and diet (DINE) among study participants at baseline and 18 month follow-up. Three binaryresponse variables were generated for analysis: (1) valid Godin score; (2) valid DINE Fibre score; and (3) validDINE Total Fat score. Multivariate analysis comprised generalised estimating equation regression to examine theassociation of patients’ characteristics with their return of valid responses at both timepoints. 
Results: Overall, 92.1% of participants (832/903) returned questionnaires at both baseline and 18 months. Relativelyfewer valid Godin scores were returned by those who left school aged <15 years (36.5%) than aged 18 and over(50.5%), manual workers (39.5%) than non-manual (49.5%) and those with an elevated cholesterol (>5 mmol)(34.7%) than those with a lower cholesterol (44.4%) but multivariate analysis identified that only school leaving age(p = 0.047) was of statistical significance.Relatively fewer valid DINE scores were returned by manual than non-manual workers (fibre: 80.8% v 86.8%;fat: 71.2% v 80.0%), smokers (fibre: 72.6% v 84.7%; fat: 67.5% v 76.9%), patients with diabetes (fibre: 75.9% v 82.9%;fat: 66.9% v 75.8%) and those with cholesterol >5 mmol (fat: 68.2% v 76.2%) but multivariate analysis showedstatistical significance only for smoking (fibre: p = 0.013; fat: p = 0.045), diabetes (fibre: p = 0.039; fat: p = 0.047), andcholesterol (fat: p = 0.039). 
Conclusions: Our findings illustrate the importance of detailed reporting of research methods, with clearinformation about response rates, respondents and valid outcome data. Outcome measures which are relevant to astudy population should be chosen carefully. The impact of methods of outcome measurement and valid responserates in evaluating healthcare requires further study.

Sector analysis provides additional spatial information on the permanent prostate brachytherapy learning curve

PURPOSE: To evaluate the permanent prostate brachytherapy (PPB) learning curve using postimplant multisector dosimetric analysis and to assess the correlation between sector -specific dosimetry and patient-reported outcome measures (PROMs).

METHODS AND METHODS: First 200 patients treated with (125)I PPB monotherapy (145 Gy) at a single institution were assessed. Postimplant dosimetry (PID) using CT was evaluated for whole prostate (global) and 12 sectors, assessing minimum dose to 90% of prostate (D90) and dose to 0.1 cm(3) of rectum (D0.1cc). Global and sector PID results were evaluated to investigate changes in D90 with case number. Urinary and bowel PROMs were assessed using the International Prostate Symptom Score and the Expanded Prostate Cancer Index Composite questionnaire. The correlation between global and individual sector PID and urinary/bowel PROMs was also evaluated.

RESULTS: Linear regression confirmed a significant improvement in global D90 with case number (r(2) = 0.20; p = 0.001) at a rate of 0.11 Gy/case. Postimplant D90 of base sectors increased at a rate of 0.11-0.15 Gy/case (p = 0.0001) and matched global improvement. The regression lines of midgland and apex sectors were significantly different from global D90 (p = 0.01). Posterior midgland sectors showed a significant reduction in D90 with case number at a rate of 0.13-0.19 Gy/case (p = 0.01). Dose to posterior midgland sectors correlated with rectal D0.1cc dose but not bowel PROMs. Dose to posterior midgland sectors correlated with urinary International Prostate Symptom Score change, which was not apparent when global D90 alone was considered.

CONCLUSIONS: Sector analysis provided increased spatial information regarding the PPB learning curve. Furthermore, sector analysis correlated with urinary PROMs and rectal dose.

Conventional versus hypofractionated high-dose intensity-modulated radiotherapy for prostate cancer: 5-year outcomes of the randomised, non-inferiority, phase 3 CHHiP trial

BACKGROUND: Prostate cancer might have high radiation-fraction sensitivity that would give a therapeutic advantage to hypofractionated treatment. We present a pre-planned analysis of the efficacy and side-effects of a randomised trial comparing conventional and hypofractionated radiotherapy after 5 years follow-up.

METHODS: CHHiP is a randomised, phase 3, non-inferiority trial that recruited men with localised prostate cancer (pT1b-T3aN0M0). Patients were randomly assigned (1:1:1) to conventional (74 Gy delivered in 37 fractions over 7·4 weeks) or one of two hypofractionated schedules (60 Gy in 20 fractions over 4 weeks or 57 Gy in 19 fractions over 3·8 weeks) all delivered with intensity-modulated techniques. Most patients were given radiotherapy with 3-6 months of neoadjuvant and concurrent androgen suppression. Randomisation was by computer-generated random permuted blocks, stratified by National Comprehensive Cancer Network (NCCN) risk group and radiotherapy treatment centre, and treatment allocation was not masked. The primary endpoint was time to biochemical or clinical failure; the critical hazard ratio (HR) for non-inferiority was 1·208. Analysis was by intention to treat. Long-term follow-up continues. The CHHiP trial is registered as an International Standard Randomised Controlled Trial, number ISRCTN97182923.

FINDINGS: Between Oct 18, 2002, and June 17, 2011, 3216 men were enrolled from 71 centres and randomly assigned (74 Gy group, 1065 patients; 60 Gy group, 1074 patients; 57 Gy group, 1077 patients). Median follow-up was 62·4 months (IQR 53·9-77·0). The proportion of patients who were biochemical or clinical failure free at 5 years was 88·3% (95% CI 86·0-90·2) in the 74 Gy group, 90·6% (88·5-92·3) in the 60 Gy group, and 85·9% (83·4-88·0) in the 57 Gy group. 60 Gy was non-inferior to 74 Gy (HR 0·84 [90% CI 0·68-1·03], pNI=0·0018) but non-inferiority could not be claimed for 57 Gy compared with 74 Gy (HR 1·20 [0·99-1·46], pNI=0·48). Long-term side-effects were similar in the hypofractionated groups compared with the conventional group. There were no significant differences in either the proportion or cumulative incidence of side-effects 5 years after treatment using three clinician-reported as well as patient-reported outcome measures. The estimated cumulative 5 year incidence of Radiation Therapy Oncology Group (RTOG) grade 2 or worse bowel and bladder adverse events was 13·7% (111 events) and 9·1% (66 events) in the 74 Gy group, 11·9% (105 events) and 11·7% (88 events) in the 60 Gy group, 11·3% (95 events) and 6·6% (57 events) in the 57 Gy group, respectively. No treatment-related deaths were reported.

INTERPRETATION: Hypofractionated radiotherapy using 60 Gy in 20 fractions is non-inferior to conventional fractionation using 74 Gy in 37 fractions and is recommended as a new standard of care for external-beam radiotherapy of localised prostate cancer.

FUNDING: Cancer Research UK, Department of Health, and the National Institute for Health Research Cancer Research Network.

Overview of clinical outcome measures used in glaucoma randomised controlled trials (RCTs)

Background: In clinical trials the selection of appropriate outcomes is crucial to the assessment of whether one intervention is better than another. Glaucoma is a chronic eye disease and the leading cause of irreversible blindness in the world. A variety of outcomes has been used and reported in glaucoma RCTs.

Objectives: The purpose of this review is to identify different clinical outcome measures used in glaucoma RCTs between January 2006 and March 2012.

Methods: A systematic review was conducted using standard methodology. We searched for RCTs in glaucoma published in English with no restrictions on the population type or size, or applied interventions. All clinical outcomes were included. Patient-reported, pharmacokinetic and economic outcomes were excluded.

Results: The search strategy identified 4288 potentially relevant abstracts. There were 315 publications retrieved, of which 233 RCTs were included. A total of 967 clinical measures were reported. There were large variations in the definitions used to describe different outcomes and their measures. Intraocular pressure (IOP) was the most commonly reported outcome (used in 201 RCTs, 86%) with a total of 422 measures (44%). Amongst the IOPrelated measures, the most commonly used was mean IOP (n=143, 15% of all measures). Safety outcomes were commonly reported, in 145 RCTs (62%) whereas visual field outcomes were utilized in 38 RCTs (16%).

Conclusions: There is a large variability in clinical outcomes used for glaucoma RCTs and in the way each outcome is reported. This lack of standardisation may impair the ability to evaluate the evidence of glaucoma interventions.

Establishing a population-based patient-reported outcomes study (PROMs) using national cancer registries across two jurisdictions: The Prostate Cancer Treatment, your experience (PiCTure) study

Objective: To establish an international patient-reported outcomes (PROMs) study among prostate cancer survivors, up to 18 years postdiagnosis, in two countries with different healthcare systems and ethical frameworks. Design: A cross-sectional, postal survey of prostate cancer survivors sampled and recruited via two population-based cancer registries. Healthcare professionals (HCPs) evaluated patients for eligibility to participate. Questionnaires contained validated instruments to assess health-related quality of life and psychological well-being, including QLQ-C30, QLQPR-25, EQ-5D-5L, 21-question Depression, Anxiety and Stress Scale (DASS-21) and the Decisional Regret Scale. Setting: Republic of Ireland (RoI) and Northern Ireland (NI). Primary outcome measures: Registration completeness, predictors of eligibility and response, data missingness, unweighted and weighted PROMs. Results: Prostate cancer registration was 80% (95% CI 75% to 84%) and 91% (95% CI 89% to 93%) complete 2 years postdiagnosis in NI and RoI, respectively. Of 12 322 survivors sampled from registries, 53% (n=6559) were classified as eligible following HCP screening. In the multivariate analysis, significant predictors of eligibility were: being ≤59 years of age at diagnosis (p<0.001), short-term survivor (<5 years postdiagnosis; p<0.001) and from RoI (p<0.001). 3348 completed the questionnaire, yielding a 54% adjusted response rate. 13% of men or their families called the study freephone with queries for assistance with questionnaire completion or to talk about their experience. Significant predictors of response in multivariate analysis were: being ≤59 years at diagnosis (p<0.001) and from RoI (p=0.016). Mean number of missing questions in validated instruments ranged from 0.12 (SD 0.71; EQ-5D-5L) to 3.72 (SD 6.30; QLQ-PR25). Weighted and unweighted mean EQ-5D-5L, QLQ-C30 and QLQ-PR25 scores were similar, as were the weighted and unweighted prevalences of depression, anxiety and distress. Conclusions: It was feasible to perform PROMs studies across jurisdictions, using cancer registries as sampling frames; we amassed one of the largest, international, population-based data set of prostate cancer survivors. We highlight improvements which could inform future PROMs studies, including utilising general practitioners to assess eligibility and providing a freephone service.

Choosing appropriate patient-reported outcomes instrument for glaucoma research: a systematic review of vision instruments

PURPOSE: To identify vision Patient-Reported Outcomes instruments relevant to glaucoma and assess their content validity.

METHODS: MEDLINE, MEDLINE in Process, EMBASE and SCOPUS (to January 2009) were systematically searched. Observational studies or randomised controlled trials, published in English, reporting use of vision instruments in glaucoma studies involving adults were included. In addition, reference lists were scanned to identify additional studies describing development and/or validation to ascertain the final version of the instruments. Instruments' content was then mapped onto a theoretical framework, the World Health Organization International Classification of Functioning, Disability and Health. Two reviewers independently evaluated studies for inclusion and quality assessed instrument content.

RESULTS: Thirty-three instruments were identified. Instruments were categorised into thirteen vision status, two vision disability, one vision satisfaction, five glaucoma status, one glaucoma medication related to health status, five glaucoma medication side effects and six glaucoma medication satisfaction measures according to each instruments' content. The National Eye Institute Visual Function Questionnaire-25, Impact of Vision Impairment and Treatment Satisfaction Survey-Intraocular Pressure had the highest number of positive ratings in the content validity assessment.

CONCLUSION: This study provides a descriptive catalogue of vision-specific PRO instruments, to inform the choice of an appropriate measure of patient-reported outcomes in a glaucoma context.