Establishing the validity of English GP Patient Survey items evaluating out-of-hours care

Background A 2014 national audit used the English General Practice Patient Survey (GPPS) to compare service users’ experience of out-of-hours general practitioner (GP) services, yet there is no published evidence on the validity of these GPPS items. Objectives Establish the construct and concurrent validity of GPPS items evaluating service users’ experience of GP out-of-hours care. Methods Cross-sectional postal survey of service users (n=1396) of six English out-of-hours providers. Participants reported on four GPPS items evaluating out-of-hours care (three items modified following cognitive interviews with service users), and 14 evaluative items from the Out-of-hours Patient Questionnaire (OPQ). Construct validity was assessed through correlations between any reliable (Cochran's α>0.7) scales, as suggested by a principal component analysis of the modified GPPS items, with the ‘entry access’ (four items) and ‘consultation satisfaction’ (10 items) OPQ subscales. Concurrent validity was determined by investigating whether each modified GPPS item was associated with thematically related items from the OPQ using linear regressions. Results The modified GPPS item-set formed a single scale (α=0.77), which summarised the two-component structure of the OPQ moderately well; explaining 39.7% of variation in the ‘entry access’ scores (r=0.63) and 44.0% of variation in the ‘consultation satisfaction’ scores (r=0.66), demonstrating acceptable construct validity. Concurrent validity was verified as each modified GPPS item was highly associated with a distinct set of related items from the OPQ. Conclusions Minor modifications are required for the English GPPS items evaluating out-of-hours care to improve comprehension by service users. A modified question set was demonstrated to comprise a valid measure of service users’ overall satisfaction with out-of-hours care received. This demonstrates the potential for the use of as few as four items in benchmarking providers and assisting services in identifying, implementing and assessing quality improvement initiatives.

The Reliability and Validity of a Child and Adolescent Participation in Decision Making Questionnaire

Background
There is a growing impetus across the research, policy and practice communities for children and young people to participate in decisions that affect their lives. Furthermore, there is a dearth of general instruments that measure children and young people’s views on their participation in decision making. This paper presents the reliability and validity of the Child and Adolescent Participation in Decision Making Questionnaire (CAP-DMQ) and specifically looks at a population of looked-after children where a lack of participation in decision making is an acute issue.
Methods
The participants were 151 looked after children and adolescents between 10-23 years of age who completed the 10 item CAP-DMQ. Of the participants 113 were in receipt of an advocacy service that had an aim of increasing participation in decision-making with the remaining participants not having received this service.
Results
The results showed that the CAP-DMQ had good reliability (Cronbach’s alpha = .94) and showed promising uni-dimensional construct validity through an exploratory factor analysis. The items in the CAP-DMQ also demonstrated good content validity by overlapping with prominent models of child and adolescent participation (Lundy 2007) and decision making (Halpern 2014). A regression analysis showed that age and gender were not significant predictors of CAP-DMQ scores but receipt of advocacy was a significant predictor of scores (effect size d=.88), thus showing appropriate discriminant criterion validity.
Conclusion
Overall, the CAP-DMQ showed good reliability and validity. Therefore, the measure has excellent promise for theoretical investigation in the area of child and adolescent participation in decision making and equally shows empirical promise for use as a measure in evaluating services which have increasing the participation of children and adolescents in decision making as an intended outcome.

The adaptations of a quality of life questionnaire for routine use in clinical practice: the Chronic Respiratory Disease Questionnaire in Cystic Fibrosis.

The assessment of quality of life (QOL) is necessary to monitor the course of disease and to assess the effect of new and existing interventions in clinical practice. This will only be achieved if QOL can be measured accurately and routinely. The aim of this study was to demonstrate the methodology involved in the adaptation and shortening of the Chronic Respiratory Disease Questionnaire (CRDQ) in a population of adults with cystic fibrosis (CF). A single interviewer administered the CRDQ to a sample of 45 adult patients (32 males) with CF prior to assessment of spirometric measures of lung function. Those patients whose lung function was stable at the time of study, and who could attend for a retest within 14 days, were asked to complete the questionnaire at a subsequent visit (n=10). The average interval between visits was 7 days (range 5-14 days). Correlations between spirometry and CRDQ dimensions ranged from -0.003 to 0.426. The fatigue, emotion and mastery dimensions showed high internal consistency, and adequate construct validity. In the small number of patients suitable for retest, the results indicated that the dimensions exhibited adequate test retest reliability. In contrast low internal consistency was demonstrated for the dyspnoea dimension. The fatigue, emotion and mastery dimensions could be reduced, in terms of their number of items without a substantial loss in explanatory power. This study suggests that QOL measurement can be made convenient, and so more easily accessible for routine clinical assessment.

A measure of satisfaction with food-related life

A measure of satisfaction with food-related life is developed and tested in three studies in eight European countries. Five items are retained from an original pool of seven; these items exhibit good reliability as measured by Cronbach's alpha, good temporal stability, convergent validity with two related measures, and construct validity as indicated by relationships with other indicators of quality of life, including the Satisfaction With Life and the SF-8 scales. It is concluded that this scale will be useful in studies trying to identify factors contributing to satisfaction with food-related life.

Index of sources of stress in nursing students: a confirmatory factor analysis.

Aim. This paper is a report of a study to test the proposed factor structure of the Index of Sources of Stress in Nursing Students. Background. Research across many countries has identified a number of sources of distress in nursing students but little attempt has been made to understand and measure sources of eustress or those stressors likely to enhance performance and well-being. The Index of Sources of Stress in Nursing Students was developed to do this. Exploratory factor analysis suggested a three-factor structure, the factors being labelled: learning and teaching; placement-related and course organization. It is important, however, to subject the instrument to confirmatory factor analysis as a further test of construct validity. Method. A convenience sample of final year nursing students (n = 176) was surveyed in one university in Northern Ireland in 2007. The Index of Sources of Stress in Nursing Students, which measures sources of stress likely to contribute to distress and eustress, was completed electronically. The LISREL programme was used to carry out the confirmatory factor analysis and test the factor structure suggested in the exploratory analysis. Findings. The proposed factor structure for the items measuring ‘Uplifts’ proved to be a good fit to the data and the proposed factor structure for the items measuring ‘Hassles’ showed adequate fit. Conclusion. In nursing programmes adopting the academic model and combining university-based learning with placement experience, this instrument can be used to help identify the sources of stress or course demands that students rate as distressing and those that help them to achieve. The validity of the ISSN could be further evaluated in other education settings.

Surveying nursing students on their sources of stress: A validation study

This study tested the psychometric properties of a questionnaire that measured sources of distress and eustress, or good stress, in nursing students. The Transactional model of stress construes stress in these different ways and is frequently used to understand sources of stress, coping and stress responses. Limited research has attempted to measure sources of distress and eustress or sources that can potentially enhance performance and well-being. A volunteer sample of final year nursing students (n = 120) was surveyed in the United Kingdom in 2007. The questionnaire measured sources of stress and measures of psychological well-being were taken to test construct validity. This was tested through an exploratory factor analysis. This reduced the questionnaire from 49 to 29 items and suggested three factors: learning and teaching, placement related and course organization; second, it was analysed by testing the assumptions of the Transactional model, the model on which the questionnaire was based. In line with the assumptions of the model, measures of distress related to adverse well-being, and measures of eustress related to healthier well-being responses. The test–retest reliability estimate was 0.8. While certain programme issues were associated with distress, placement-related experiences were the most important source of eustress.

The structure of the Beck Hopelessness Scale: A confirmatory factor analysis in UK students

The aim of this paper was to confirm the factor structure of the 20-item Beck Hopelessness Scale in a non-clinical population. Previous research has highlighted a lack of clarity in its construct validity with regards to this population.

Based on previous factor analytic findings from both clinical and non-clinical studies, 13 separate confirmatory factor models were specified and estimated using LISREL 8.72 to test the one, two and three-factor models.

Psychology and medical students at Queen's University, Belfast (n = 581) completed both the BHS and the Beck Depression Inventory (BDI).

All models showed reasonable fit, but only one, a four-item single-factor model demonstrated a nonsignificant chi-squared statistic. These four items can be used to derive a Short-Form BHS (SBHS) in which increasing scores (0-4) corresponded with increasing scores in the BDI. The four items were also drawn from all three of Beck's proposed triad, and included both positively and negatively scored items.

This study in a UK undergraduate non-clinical population suggests that the BHS best measures a one-factor model of hopelessness. It appears that a shorter four-item scale can also measure this one-factor model. (C) 2011 Elsevier Ltd. All rights reserved.

Bedside application of the Neonatal Facial Coding System in pain assessment of premature neonates

Assessment of infant pain is a pressing concern, especially within the context of neonatal intensive care where infants may be exposed to prolonged and repeated pain during lengthy hospitalization. In the present study the feasibility of carrying out the complete Neonatal Facial Coding System (NFCS) in real time at bedside, specifically reliability, construct and concurrent validity, was evaluated in a tertiary level Neonatal Intensive Care Unit (NICU). Heel lance was used as a model of procedural pain, and observed with n = 40 infants at 32 weeks gestational age. Infant sleep/wake state, NFCS facial activity and specific hand movements were coded during baseline, unwrap, swab, heel lance, squeezing and recovery events. Heart rate was recorded continuously and digitally sampled using a custom designed computer system. Repeated measures analysis of variance (ANOVA) showed statistically significant differences across events for facial activity (P <0.0001) and heart rate (P <0.0001). Planned comparisons showed facial activity unchanged during baseline, swab and unwrap, then increased significantly during heel lance (P <0.0001), increased further during squeezing (P <0.003), then decreased during recovery (P <0.0001). Systematic shifts in sleep/wake state were apparent. Rise in facial activity was consistent with increased heart rate, except that facial activity more closely paralleled initiation of the invasive event. Thus facial display was more specific to tissue damage compared with heart rate. Inter-observer reliability was high. Construct validity of the NFCS at bedside was demonstrated as invasive procedures were distinguished from tactile. While bedside coding of behavior does not permit raters to be blind to events, mechanical recording of heart rate allowed for an independent source of concurrent validation for bedside application of the NFCS scale.

A comparison of two measures of facial activity during pain in the newborn child

Facial activity is strikingly visible in infants reacting to noxious events. Two measures that reduce this activity to composite events, the Neonatal Facial Coding System (NFCS) and the Facial Action Coding System (FACS), were used to examine facial expressions of 56 neonates responding to routine heel lancing for blood sampling purposes. The NFCS focuses upon a limited subset of all possible facial actions that had been identified previously as responsive to painful events, whereas the FACS is a comprehensive system that is inclusive of all facial actions. Descriptions of the facial expressions obtained from the two measurement systems were very similar, supporting the convergent validity of the shorter, more readily applied system. As well, the cluster of facial activity associated with pain in this sample, using either measure, was similar to the cluster of facial activity associated with pain in adults and other newborns, both full-term and preterm, providing construct validity for the position that the face encodes painful distress in infants and adults.

Assessing the consequences of traumatic brain injury

Research concerned with assessing the rehabilitation outcome for the survivor of traumatic brain injury has suffered from both conceptual and procedural difficulties. The purpose of this paper is twofold: to assess the psychometric features in instruments used in assessing outcome; and to describe a test development framework based on the principles of construct validity. A construct validation approach is viewed as a means of avoiding common measurement difficulties, as well as integrating perspectives important to this population. Emphasis has been given to the cognitive/social dimensions of recovery, as it is this area which has the greatest impact for rehabilitation success.

Organizational assessment in paediatric primary care:Development and initial validation of the primary care organizational questionnaire

Primary care in the United States is undergoing many changes. Reliable and valid instruments are needed to assess the effects of these changes. The Primary Care Organizational Questionnaire (PCOQ), a 56-item 5-point Likert scale survey that evaluates interactions among members of the clinic/practice and job-related attributes, was administered to clinicians and staff in 36 primary care practices serving paediatric populations in Connecticut. A priori scales were reliable (Cronbach alpha =0.7). Analysis of variance (ANOVA) showed greater heterogeneity across clinics than within clinics for 13 of 15 a priori scales, which were then included in a principal component factor analysis with varimax rotation. Eigenvalue analysis showed nine significant factors, largely similar to the a priori scales, indicating concurrent construct validity. Further research will ascertain the utility of the PCOQ in predicting the effectiveness of primary care practices in implementing disease management programmes. © 2006 Royal Society of Medicine Press.

Characterizing changes in the excitability of corticospinal projections to proximal muscles of the upper limb

Background: There has been an explosion of interest in methods of exogenous brain stimulation that induce changes in the excitability of human cerebral cortex. The expectation is that these methods may promote recovery of function following brain injury. To assess their effects on motor output, it is typical to assess the state of corticospinal projections from primary motor cortex to muscles of the hand, via electromyographic responses to transcranial magnetic stimulation. If a range of stimulation intensities is employed, the recruitment curves (RCs) obtained can, at least for intrinsic hand muscles, be fitted by a sigmoid function.

Objective/hypothesis: To establish whether sigmoid fits provide a reliable basis upon which to characterize the input–output properties of the corticospinal pathway for muscles proximal to the hand, and to assess as an alternative the area under the (recruitment) curve (AURC).

Methods: A comparison of the reliability of these measures, using RCs obtained for muscles that are frequently the targets of rehabilitation.

Results: The AURC is an extremely reliable measure of the state of corticospinal projections to hand and forearm muscles, which has both face and concurrent validity. Construct validity is demonstrated by detection of widely distributed (across muscles) changes in corticospinal excitability induced by paired associative stimulation (PAS).

Conclusion(s): The parameters derived from sigmoid fits are unlikely to provide an adequate means to assess the effectiveness of therapeutic regimes. The AURC can be employed to characterize corticospinal projections to a range of muscles, and gauge the efficacy of longitudinal interventions in clinical rehabilitation.

Support for the Multidimensional Adolescent Stress Questionnaire in a Sample of Adolescents in the United Kingdom

Adolescence is a time of physical, social and emotional development, and this development can be accompanied by feelings of stress. The Adolescent Stress Questionnaire is a 56-item scale measuring stress in 10 domains. Developed in Australia, the scale has been translated, and its reliability and validity have been tested in a number of countries across Europe, where the 10-factor, 56-item version of the scale has received little support. The present study tested the factor structure, construct validity and reliability in a sample (n=610) of adolescents in the United Kingdom. Support was found for the 10-factor, 56-item version of the scale, and correlations with self-concept measures, sex scores on stress factors and Cronbach's α-values, suggesting that the scale may be a viable assessment tool for adolescent stress. Results for alcohol-specific analyses support the domain-specific nature of the scale. Future work may seek to investigate the stability of age-specific stress domains (e.g. the stress of Emerging Adult Responsibility) in samples that include younger adolescents. 

Development of the Consumer Refrigerator Safety Questionnaire (CRSQ): A Measure of Consumer Perceptions and Practices

Food preparation and storage behaviors in the home deviating from the ‘best practice’ food safety recommendations may result in food borne illnesses. Currently, there are limited tools available to fully evaluate the consumer knowledge, perceptions and behavior in the area of refrigerator safety. The current study aimed to develop a valid and reliable tool in the form of a questionnaire (CFSQCRSQ) for assessing systematically all these aspects. Items relating to refrigerator safety knowledge (n=17), perceptions (n=46), reported behavior (n=30) were developed and pilot tested by an expert reference group and various consumer groups to assess face and content validity (n=20), item difficulty and item consistency (n=55) and construct validity (n=23). The findings showed that the CFSQCRSQ has acceptable face and content validity with acceptable levels of item difficulty. Item consistency was observed for 12 out of 15 refrigerator safety knowledge. Further, all five of the subscales of consumer perceptions of refrigerator safety practices relating to risk of developing foodborne disease food poisoning showed acceptable internal consistency (Cronbach’s α value > 0.8). Construct validity of the CFSQCRSQ was shown to be very good (p=0.022). The CFSQCRSQ exhibited acceptable test-retest reliability at 14 days with majority of knowledge items (93.3%) and reported behavior items (96.4%) having correlation coefficients of greater than 0.70. Overall, the CFSQCRSQ was deemed valid and reliable in assessing refrigerator safety knowledge and behavior and therefore has the potential for future use in identifying groups of individuals at increased risk of deviating from recommended refrigerator safety practices as well as the assessment of refrigerator safety knowledge, behavior for use before and after an intervention.

Reliability and factorial validity of Adolescent Time Inventory-Time Attitude (ATI-TA) Scores in Scottish and Northern Irish adolescents

The contemporary literature investigating the construct broadly known as time perspective is replete with methodological and conceptual concerns. These concerns focus on the reliability and factorial validity of measurement tools, and the sample-specific modification of scales. These issues continue to hamper the development of this potentially useful psychological construct. An emerging body of evidence has supported the six-factor structure of scores on the Adolescent Time Inventory-Time Attitudes Scale, as well as their reliability. The present study utilized data from the first wave of a longitudinal study in the United Kingdom to examine the reliability, validity, and cross-cultural invariance of the scale. Results showed that the hypothesized six-factor model provided the best fit for the data; all alpha and omega estimates were >. .70; scores on ATI-TA factors related meaningfully to self-efficacy scores; and the factor structure was invariant across both research sites. Results are discussed in the context of the extant temporal literature.