Establishing the validity of English GP Patient Survey items evaluating out-of-hours care

Background A 2014 national audit used the English General Practice Patient Survey (GPPS) to compare service users’ experience of out-of-hours general practitioner (GP) services, yet there is no published evidence on the validity of these GPPS items. Objectives Establish the construct and concurrent validity of GPPS items evaluating service users’ experience of GP out-of-hours care. Methods Cross-sectional postal survey of service users (n=1396) of six English out-of-hours providers. Participants reported on four GPPS items evaluating out-of-hours care (three items modified following cognitive interviews with service users), and 14 evaluative items from the Out-of-hours Patient Questionnaire (OPQ). Construct validity was assessed through correlations between any reliable (Cochran's α>0.7) scales, as suggested by a principal component analysis of the modified GPPS items, with the ‘entry access’ (four items) and ‘consultation satisfaction’ (10 items) OPQ subscales. Concurrent validity was determined by investigating whether each modified GPPS item was associated with thematically related items from the OPQ using linear regressions. Results The modified GPPS item-set formed a single scale (α=0.77), which summarised the two-component structure of the OPQ moderately well; explaining 39.7% of variation in the ‘entry access’ scores (r=0.63) and 44.0% of variation in the ‘consultation satisfaction’ scores (r=0.66), demonstrating acceptable construct validity. Concurrent validity was verified as each modified GPPS item was highly associated with a distinct set of related items from the OPQ. Conclusions Minor modifications are required for the English GPPS items evaluating out-of-hours care to improve comprehension by service users. A modified question set was demonstrated to comprise a valid measure of service users’ overall satisfaction with out-of-hours care received. This demonstrates the potential for the use of as few as four items in benchmarking providers and assisting services in identifying, implementing and assessing quality improvement initiatives.

A methodological review of the Short Form Health Survey 36 (SF-36) and its derivatives among breast cancer survivors

Purpose: A systematic review of the validity, reliability and sensitivity of the Short Form (SF) health survey measures among breast cancer survivors.
Methods: We searched a number of databases for peer-reviewed papers. The methodological quality of the papers was assessed using the COnsenus-based Standards for the selection of health Measurement INstruments (COSMIN).
Results: The review identified seven papers that assessed the psychometric properties of the SF-36 (n = 5), partial SF-36 (n = 1) and SF-12 (n = 1) among breast cancer survivors. Internal consistency scores for the SF measures ranged from acceptable to good across a range of language and ethnic sub-groups. The SF-36 demonstrated good convergent validity with respective subscales of the Functional Assessment of Cancer Treatment—General scale and two lymphedema-specific measures. Divergent validity between the SF-36 and Lymph-ICF was modest. The SF-36 demonstrated good factor structure in the total breast cancer survivor study samples. However, the factor structure appeared to differ between specific language and ethnic sub-groups. The SF-36 discriminated between survivors who reported or did not report symptoms on the Breast Cancer Prevention Trial Symptom Checklist and SF-36 physical sub-scales, but not mental sub-scales, discriminated between survivors with or without lymphedema. Methodological quality scores varied between and within papers.
Conclusion: Short Form measures appear to provide a reliable and valid indication of general health status among breast cancer survivors though the limited data suggests that particular caution is required when interpreting scores provided by non-English language groups. Further research is required to test the sensitivity or responsiveness of the measure.

The use of patient experience survey data by out-of-hours primary care services: a qualitative interview study

Background English National Quality Requirements mandate out-of-hours primary care services to routinely audit patient experience, but do not state how it should be done.

Objectives We explored how providers collect patient feedback data and use it to inform service provision. We also explored staff views on the utility of out-of-hours questions from the English General Practice Patient Survey (GPPS).

Methods A qualitative study was conducted with 31 staff (comprising service managers, general practitioners and administrators) from 11 out-of-hours primary care providers in England, UK. Staff responsible for patient experience audits within their service were sampled and data collected via face-to-face semistructured interviews.

Results Although most providers regularly audited their patients’ experiences by using patient surveys, many participants expressed a strong preference for additional qualitative feedback. Staff provided examples of small changes to service delivery resulting from patient feedback, but service-wide changes were not instigated. Perceptions that patients lacked sufficient understanding of the urgent care system in which out-of-hours primary care services operate were common and a barrier to using feedback to enable change. Participants recognised the value of using patient experience feedback to benchmark services, but perceived weaknesses in the out-of-hours items from the GPPS led them to question the validity of using these data for benchmarking in its current form.

Conclusions The lack of clarity around how out-of-hours providers should audit patient experience hinders the utility of the National Quality Requirements. Although surveys were common, patient feedback data had only a limited role in service change. Data derived from the GPPS may be used to benchmark service providers, but refinement of the out-of-hours items is needed.