Quantitative data collection and study validity

The quality of data collection methods selected and the integrity of the data collected are integral tot eh success of a study. This chapter focuses on data collection and study validity. After reading the chapter, readers should be able to define types of data collection methods in quantitative research; list advantages and disadvantages of each method; discuss factors related to internal and external validity; critically evaluate data collection methods and discuss the need to operationalise variables of interest for data collection.

Validation of a parent-proxy quality of life questionnaire for paediatric chronic cough (PC-QOL)

Background Quality of life (QOL) measures are an important patient-relevant outcome measure for clinical studies. Currently there is no fully validated cough-specific QOL measure for paediatrics. The objective of this study was to validate a cough-specific QOL questionnaire for paediatric use. Method 43 children (28 males, 15 females; median age 29 months, IQR 20–41 months) newly referred for chronic cough participated. One parent of each child completed the 27-item Parent Cough-Specific QOL questionnaire (PC-QOL), and the generic child (Pediatric QOL Inventory 4.0 (PedsQL)) and parent QOL questionnaires (SF-12) and two cough-related measures (visual analogue score and verbal category descriptive score) on two occasions separated by 2–3 weeks. Cough counts were also objectively measured on both occasions. Results Internal consistency for both the domains and total PC-QOL at both test times was excellent (Cronbach alpha range 0.70–0.97). Evidence for repeatability and criterion validity was established, with significant correlations over time and significant relationships with the cough measures. The PC-QOL was sensitive to change across the test times and these changes were significantly related to changes in cough measures (PC-QOL with: verbal category descriptive score, rs=−0.37, p=0.016; visual analogue score, rs=−0.47, p=0.003). Significant correlations of the difference scores for the social domain of the PC-QOL and the domain and total scores of the PedsQL were also noted (rs=0.46, p=0.034). Conclusion The PC-QOL is a reliable and valid outcome measure that assesses QOL related to childhood cough at a given time point and measures changes in cough-specific QOL over time.

Validity of family child care providers' proxy reports on children's physical activity

Background Interventions to promote physical activity (PA) in children attending family child care homes (FCCHs) require valid, yet practical, measurement tools. The aim of this study was to assess the validity of two proxy report instruments designed to measure PA in children attending FCCHs. Methods A sample of 37 FCCH providers completed the Burdette parent proxy report, modified for the family child care setting for 107 children 3.4±1.2 years of age. A second sample of 42 FCCH providers completed the Harro parent and teacher proxy report, modified for the family child care setting, for 131 children 3.8±1.3 years of age. Both proxy reports were assessed for validity using accelerometry as a criterion measure. Results Significant positive correlations were observed between provider-reported PA scores from the modified Burdette proxy report and objectively measured total PA (r=0.30; p<0.01) and moderate-to-vigorous PA (MVPA; r=0.34; p<0.01). Across levels of provider-reported PA, both total PA and MVPA increased significantly in a linear dose-response fashion. The modified Harro proxy report was not associated with objectively measured PA. Conclusion Proxy PA reports completed by family child care providers may be a valid assessment option in studies where more burdensome objective measures are not feasible.

Comparison of accelerometer cut points for predicting activity intensity in youth

The absence of comparative validity studies has prevented researchers from reaching consensus regarding the application of intensity-related accelerometer cut points for children and adolescents. PURPOSE This study aimed to evaluate the classification accuracy of five sets of independently developed ActiGraph cut points using energy expenditure, measured by indirect calorimetry, as a criterion reference standard. METHODS A total of 206 participants between the ages of 5 and 15 yr completed 12 standardized activity trials. Trials consisted of sedentary activities (lying down, writing, computer game), lifestyle activities (sweeping, laundry, throw and catch, aerobics, basketball), and ambulatory activities (comfortable walk, brisk walk, brisk treadmill walk, running). During each trial, participants wore an ActiGraph GT1M, and VO 2 was measured breath-by-breath using the Oxycon Mobile portable metabolic system. Physical activity intensity was estimated using five independently developed cut points: Freedson/Trost (FT), Puyau (PU), Treuth (TR), Mattocks (MT), and Evenson (EV). Classification accuracy was evaluated via weighted κ statistics and area under the receiver operating characteristic curve (ROC-AUC). RESULTS Across all four intensity levels, the EV (κ = 0.68) and FT (κ = 0.66) cut points exhibited significantly better agreement than TR (κ = 0.62), MT (κ = 0.54), and PU (κ = 0.36). The EV and FT cut points exhibited significantly better classification accuracy for moderate-to vigorous-intensity physical activity (ROC-AUC = 0.90) than TR, PU, or MT cut points (ROC-AUC = 0.77-0.85). Only the EV cut points provided acceptable classification accuracy for all four levels of physical activity intensity and performed well among children of all ages. The widely applied sedentary cut point of 100 counts per minute exhibited excellent classification accuracy (ROC-AUC = 0.90). CONCLUSIONS On the basis of these findings, we recommend that researchers use the EV ActiGraph cut points to estimate time spent in sedentary, light-, moderate-, and vigorous-intensity activity in children and adolescents. Copyright © 2011 by the American College of Sports Medicine.

Validity of the computer science and applications (CSA) activity monitor in children

Purpose The purpose of this study was to evaluate the validity of the CSA activity monitor as a measure of children's physical activity using energy expenditure (EE) as a criterion measure. Methods Thirty subjects aged 10 to 14 performed three 5-min treadmill bouts at 3, 4, and 6 mph, respectively. While on the treadmill, subjects wore CSA (WAM 7164) activity monitors on the right and left hips. (V) over dot O-2 was monitored continuously by an automated system. EE was determined by multiplying the average (V) over dot O-2 by the caloric equivalent of the mean respiratory exchange ratio. Results Repeated measures ANOVA indicated that both CSA monitors were sensitive to changes in treadmill speed. Mean activity counts from each CSA unit were not significantly different and the intraclass reliability coefficient for the two CSA units across all speeds was 0.87. Activity counts from both CSA units were strongly correlated with EE (r = 0.86 and 0.87, P < 0.001). An EE prediction equation was developed from 20 randomly selected subjects and cross-validated on the remaining 10. The equation predicted mean EE within 0.01 kcal.min(-1). The correlation between actual and predicted values was 0.93 (P < 0.01) and the SEE was 0.93 kcal.min(-1). Conclusion These data indicate that the CSA monitor is a valid and reliable tool for quantifying treadmill walking and running in children.

Calibration of the biotrainer pro activity monitor in children

The unique physical and movement characteristics of children necessitate the development of accelerometer equations and cut points that are population specific. The purpose of this study is to develop an ecologically valid cut point for the Biotrainer Pro monitor that reflects a threshold for moderate-intensity physical activity in elementary school children. A sample of 30 children (ages 8-12) wore a Biotrainer monitor while completing a series of 7 movement tasks (calibration phase) and while participating in an organized group activity (cross-validation phase). Videotapes from each session were processed using a computerized direct-observation technique to provide a criterion measure of physical activity. Analyses involved the use of mixed-model regression and receiver operator characteristic (ROC) curves. The results indicated that a cut point of 4 counts/min provides the optimal balance between the related needs for sensitivity (accurately detecting activity) and specificity (limiting misclassification of activity as inactivity). Results with the cross-validation data demonstrated that this value yielded the best overall kappa (.58) and a high classification agreement (84%) for activity determination. The specificity of 93% demonstrates that the proposed cut point can accurately detect activity; however, the lower sensitivity value of 61% suggests that some minutes of activity might be incorrectly classified as inactivity. The cut point of 4 counts/min provides an ecologically valid cut point to capture physical activity in children using the Biotrainer Pro activity monitor.

Validity of accelerometry in ambulatory children and adolescents with cerebral palsy

To evaluate the validity of the ActiGraph accelerometer for the measurement of physical activity intensity in children and adolescents with cerebral palsy (CP) using oxygen uptake (VO 2) as the criterion measure. Thirty children and adolescents with CP (mean age 12.6 ± 2.0 years) wore an ActiGraph 7164 and a Cosmed K4b 2 portable indirect calorimeter during four activities; quiet sitting, comfortable paced walking, brisk paced walking and fast paced walking. VO 2 was converted to METs and activity energy expenditure and classiWed as sedentary, light or moderate-to-vigorous intensity according to the conventions for children. Mean ActiGraph counts min -1 were classiWed as sedentary, light or moderate-to-vigorous (MVPA) intensity using four diVerent sets of cut-points. VO 2 and counts min¡1 increased signiWcantly with increases in walking speed (P < 0.001). Receiver operating characteristic (ROC) curve analysis indicated that, of the four sets of cut-points evaluated, the Evenson et al. (J Sports Sci 26(14):1557-1565, 2008) cut-points had the highest classiWcation accuracy for sedentary (92%) and MVPA (91%), as well as the second highest classiWcation accuracy for light intensity physical activity (67%). A ROC curve analysis of data from our participants yielded a CP-speciWc cut-point for MVPA that was lower than the Evenson cut-point (2,012 vs. 2,296 counts min¡1), however, the diVerence in classiWcation accuracy was not statistically signiWcant 94% (95% CI = 88.2-97.7%) vs. 91% (95% CI = 83.5-96.5%). In conclusion, among children and adolescents with CP, the ActiGraph is able to diVerentiate between diVerent intensities of walking. The use of the Evenson cut-points will permit the estimation of time spent in MVPA and allows comparisons to be made between activity measured in typically developing adolescents and adolescents with CP. © 2011 Springer-Verlag.

Cross-cultural adaptation, reliability and validity of the Spanish version of the upper limb functional index

Background The Upper Limb Functional Index (ULFI) is an internationally widely used outcome measure with robust, valid psychometric properties. The purpose of study is to develop and validate a ULFI Spanish-version (ULFI-Sp). Methods A two stage observational study was conducted. The ULFI was cross-culturally adapted to Spanish through double forward and backward translations, the psychometric properties were then validated. Participants (n = 126) with various upper limb conditions of >12 weeks duration completed the ULFI-Sp, QuickDASH and the Euroqol Health Questionnaire 5 Dimensions (EQ-5D-3 L). The full sample determined internal consistency, concurrent criterion validity, construct validity and factor structure; a subgroup (n = 35) determined reliability at seven days. Results The ULFI-Sp demonstrated high internal consistency (α = 0.94) and reliability (r = 0.93). Factor structure was one-dimensional and supported construct validity. Criterion validity with the EQ-5D-3 L was fair and inversely correlated (r = −0.59). The QuickDASH data was unavailable for analysis due to excessive missing responses. Conclusions The ULFI-Sp is a valid upper limb outcome measure with similar psychometric properties to the English language version.

An acute cough-specific quality of life questionnaire for children: Development and validation

Background Patient-relevant outcome measures are essential for high-quality clinical research, and quality-of-life (QoL) tools are the current standard. Currently, there is no validated children's acute cough-specific QoL questionnaire. Objective The objective of this study was to develop and validate the Parent-proxy Children's Acute Cough-specific QoL Questionnaire (PAC-QoL). Methods Using focus groups, a 48-item PAC-QoL questionnaire was developed and later reduced to 16 items by using the clinical impact method. Parents of children with a current acute cough (<2 weeks) at enrollment completed 2 validated cough score measures, the preliminary 48-item PAC-QoL, and 3 other questionnaires (the State Trait Anxiety Inventory [STAI], the Short-Form 8-item 24-hour recall Health Survey [SF-8], and the Depression, Anxiety, and Stress 21-item Scale [DASS21]). All measures were repeated on days 3 and 14. Results The median age of the 155 children enrolled was 2.3 years (interquartile range, 1.3-4.6). Median cough duration at enrollment was 3 days (interquartile range, 2-5). The reduced 16-item scale had high internal consistency (Cronbach α = 0.95). Evidence for repeatability and criterion validity was shown by significant correlations between the domains and total PAC-QoL scores and the SF-8 (r = −0.36 and −0.51), STAI (r = −0.27 and −0.39), and DASS21 (r = −0.32 and −0.41) scales on days 0 and 3, respectively. The final PAC-QoL questionnaire was sensitive to change over time, with changes significantly relating to changes in cough score measures (P < .001). Conclusion The 16-item PAC-QoL is a reliable and valid outcome measure that assesses QoL related to childhood acute cough at a given time point and reflects changes in acute cough-specific QoL over time.

Prognostic validity of 3-Minute Nutrition Screening (3-MinNS) in predicting length of hospital stay, readmission, cost of hospitalisation and mortality : a cohort study

Background: It is important to identify patients who are at risk of malnutrition upon hospital admission as malnutrition results in poor outcomes such as longer length of hospital stay, readmission, hospitalisation cost and mortality. The aim of this study was to determine the prognostic validity of 3-Minute Nutrition Screening (3-MinNS) in predicting hospital outcomes in patients admitted to an acute tertiary hospital through a list of diagnosis-related groups (DRG). Methods: In this study, 818 adult patients were screened for risk of malnutrition using 3-MinNS within 24 hours of admission. Mortality data was collected from the National Registry with other hospitalisation outcomes retrieved from electronic hospital records. The results were adjusted for age, gender and ethnicity, and matched for DRG. Results: Patients identified to be at risk of malnutrition (37%) using 3-MinNS had significant positive association with longer length of hospital stay (6.6 ± 7.1 days vs. 4.5 ± 5.5 days, p<0.001), higher hospitalisation cost (S$4540 ± 7190 vs. S$3630 ± 4961, p<0.001) and increased mortality rate at 1 year (27.8% vs. 3.9%), 2 years (33.8% vs. 7.2%) and 3 years (39.1% vs. 10.5%); p<0.001 for all. Conclusions: The 3-MinNS is able to predict clinical outcomes and can be used to screen newly admitted patients for nutrition risk so that appropriate nutrition assessment and early nutritional intervention can be initiated.

Inventory of pedagogy and practice: Confirmatory factor analysis of multiple facets of teaching

Student perceptions of teaching have often been used in tertiary education for evaluation purposes. However, there is a paucity of research on the validity, reliability, and applicability of instruments that cover a wide range of student perceptions of pedagogies and practices in high school settings for descriptive purposes. The study attempts to validate an inventory of pedagogy and practice (IPP) that provides researchers and practitioners with a psychometrically sound instrument that covers the most salient factors related to teaching. Using a sample of students (N = 1515) from 39 schools in Singapore, 14 factors about teaching in English lessons from the students’ perspective were tested with confirmatory factor analysis (classroom task goal, structure and clarity, curiosity and interest, positive class climate, feedback, questioning, quality homework, review of students’ work, conventional teaching, exam preparation, behaviour management, maximizing learning time, student-centred pedagogy, and subject domain teaching). Two external criterion factors were used to further test the IPP factor structure. The inventory will enable teachers to understand more about their teaching and researchers to examine how teaching may be related to learning outcomes.

Cross-cultural adaptation, reliability and validity of the Turkish version of the spine functional index

Background The Spine Functional Index (SFI) is a patient reported outcome measure with sound clinimetric properties and clinical viability for the determination of whole-spine impairment. To date, no validated Turkish version is available. The purpose of this study is to cross-culturally adapted the SFI for Turkish-speaking patients (SFI-Tk) and determine the psychometric properties of reliability, validity and factor structure in a Turkish population with spine musculoskeletal disorders. Methods The SFI English version was culturally adapted and translated into Turkish using a double forward and backward method according to established guidelines. Patients (n = 285, cervical = l29, lumbar = 151, cervical and lumbar region = 5, 73% female, age 45 ± 1) with spine musculoskeletal disorders completed the SFI-Tk at baseline and after a seven day period for test-retest reliability. For criterion validity the Turkish version of the Functional Rating Index (FRI) was used plus the Neck Disability Index (NDI) for cervical patients and the Oswestry Disability Index (ODI) for back patients. Additional psychometric properties were determined for internal consistency (Chronbach’s α), criterion validity and factor structure. Results There was a high degree of internal consistency (α = 0.85, item range 0.80-0.88) and test-retest reliability (r = 0.93, item range = 0.75-0.95). The factor analysis demonstrated a one-factor solution explaining 24.2% of total variance. Criterion validity with the ODI was high (r = 0.71, p < 0.001) while the FRI and NDI were fair (r = 0.52 and r = 0.58, respectively). The SFI-Tk showed no missing responses with the ‘half-mark’ option used in 11.75% of total responses by 77.9% of participants. Measurement error from SEM and MDC90 were respectively 2.96% and 7.12%. Conclusions The SFI-Tk demonstrated a one-factor solution and is a reliable and valid instrument. The SFI-Tk consists of simple and easily understood wording and may be used to assess spine region musculoskeletal disorders in Turkish speaking patients.

Extending the validity of the Feeding Practices and Structure Questionnaire

Background Feeding practices are commonly examined as potentially modifiable determinants of children’s eating behaviours and weight status. Although a variety of questionnaires exist to assess different feeding aspects, many lack thorough reliability and validity testing. The Feeding Practices and Structure Questionnaire (FPSQ) is a tool designed to measure early feeding practices related to non-responsive feeding and structure of the meal environment. Face validity, factorial validity, internal reliability and cross-sectional correlations with children’s eating behaviours have been established in mothers with 2-year-old children. The aim of the present study was to further extend the validity of the FPSQ by examining factorial, construct and predictive validity, and stability. Methods Participants were from the NOURISH randomised controlled trial which evaluated an intervention with first-time mothers designed to promote protective feeding practices. Maternal feeding practices (FP) and child eating behaviours were assessed when children were aged 2 years and 3.7 years (n=388). Confirmatory Factor analysis, group differences, predictive relationships, and stability were tested. Results The original 9-factor structure was confirmed when children were aged 3.7±0.3 years. Cronbach’s alpha was above the recommended 0.70 cut-off for all factors except Structured Meal Timing, Over Restriction and Distrust in Appetite which were 0.58, 0.67 and 0.66 respectively. Allocated group differences reflected behaviour consistent with intervention content and all feeding practices were stable across both time points (range of r= 0.45-0.70). There was some evidence for the predictive validity of factors with 2 FP showing expected relationships, 2 FP showing expected and unexpected relationships and 5 FP showing no relationship. Conclusions Reliability and validity was demonstrated for most subscales of the FPSQ. Future validation is warranted with culturally diverse samples and with fathers and other caregivers. The use of additional outcomes to further explore predictive validity is recommended as well as testing construct validity and test-retest reliability of the questionnaire.

Validity of BMI as a measure of obesity in Australian white Caucasian and Australian Sri Lankan children

Background: Body mass index (BMI) is used to diagnose obesity. However, its ability to predict the percentage fat mass (%FM) reliably is doubtful. Therefore validity of BMI as a diagnostic tool of obesity is questioned. Aim: This study is focused on determining the ability of BMI-based cut-off values in diagnosing obesity among Australian children of white Caucasian and Sri Lankan origin. Subjects and methods: Height and weight was measured and BMI (W/H2) calculated. Total body water was determined by deuterium dilution technique and fat free mass and hence fat mass derived using age- and gender-specific constants. A %FM of 30% for girls and 20% for boys was considered as the criterion cut-off level for obesity. BMI-based obesity cut-offs described by the International Obesity Task Force (IOTF), CDC/NCHS centile charts and BMI-Z were validated against the criterion method. Results: There were 96 white Caucasian and 42 Sri Lankan children. Of the white Caucasians, 19 (36%) girls and 29 (66%) boys, and of the Sri Lankans 7 (46%) girls and 16 (63%) boys, were obese based on %FM. The FM and BMI were closely associated in both Caucasians (r = 0.81, P<0.001) and Sri Lankans (r = 0.92, P<0.001). Percentage FM and BMI also had a lower but significant association. Obesity cut-off values recommended by IOTF failed to detect a single case of obesity in either group. However, NCHS and BMI-Z cut-offs detected cases of obesity with low sensitivity. Conclusions: BMI is a poor indicator of percentage fat and the commonly used cut-off values were not sensitive enough to detect cases of childhood obesity in this study. In order to improve the diagnosis of obesity, either BMI cut-off values should be revised to increase the sensitivity or the possibility of using other indirect methods of estimating the %FM should be explored.

Infant product-related injuries: comparing specialised injury surveillance and routine emergency department data

- Objective To explore the potential for using a basic text search of routine emergency department data to identify product-related injury in infants and to compare the patterns from routine ED data and specialised injury surveillance data. - Methods Data was sourced from the Emergency Department Information System (EDIS) and the Queensland Injury Surveillance Unit (QISU) for all injured infants between 2009 and 2011. A basic text search was developed to identify the top five infant products in QISU. Sensitivity, specificity, and positive predictive value were calculated and a refined search was used with EDIS. Results were manually reviewed to assess validity. Descriptive analysis was conducted to examine patterns between datasets. - Results The basic text search for all products showed high sensitivity and specificity, and most searches showed high positive predictive value. EDIS patterns were similar to QISU patterns with strikingly similar month-of-age injury peaks, admission proportions and types of injuries. - Conclusions This study demonstrated a capacity to identify a sample of valid cases of product-related injuries for specified products using simple text searching of routine ED data. - Implications As the capacity for large datasets grows and the capability to reliably mine text improves, opportunities for expanded sources of injury surveillance data increase. This will ultimately assist stakeholders such as consumer product safety regulators and child safety advocates to appropriately target prevention initiatives.