The test-retest reliability of the child-initiated pretend play assessment

OBJECTIVE: The purpose of this study is to establish the test–retest reliability of the Child-Initiated Pretend Play Assessment (ChIPPA) (Stagnitti, 2002a; Stagnitti, Unsworth, & Rodger, 2000).

METHOD: The first author rated 38 preschool children ages 4 and 5 years (4 with developmental delay and 34 typically developing) on the ChIPPA. The ChIPPA employs conventional play materials and unstructured play materials to assess three qualities of a child's play ability: elaborateness of play action, ability to substitute objects during play, and the child's need to imitate the modelled actions of the examiner. The ChIPPA was administered twice, at a 2-week interval, to each participant.

RESULTS: Test–retest intraclass correlation coefficients (ICCs) (Type 2,1) calculated for each of the three elaborate play measures ranged from .73 to .84. A test–retest ICC of .56 was obtained for object substitution with unstructured play materials. The test–retest ICC obtained for the combined score for unstructured and conventional play materials was .57. Percentage agreement figures ranging from 63.2% to 84.2% were obtained on test–retest of the object substitution with conventional toys and imitated actions measures. There was no significant difference between test and retest scores for these measures based on a Wilcoxon Matched Pairs Signed-Ranks Test (Wilcoxon Sign Test).

CONCLUSION: Elaborate play scores, object substitution with conventional toys score, and imitation scores on the ChIPPA showed stability over time. Object substitution scores using unstructured materials were the least stable play measures and appeared to be related to the child's play themes. Since play is the primary occupation of children, it is essential that therapists have a reliable measure of play behavior. The test–retest reliability results from the ChIPPA provide evidence that this assessment produces a stable measure of play behavior that can then guide therapists when planning intervention strategies for children.

Development and reliability of a self-report questionnaire to examine children's perceptions of the physical activity environment at home and in the neighbourhood

Background: Environmental factors are increasingly being implicated as key influences on children's physical activity. Few studies have comprehensively examined children's perceptions of their environment, and there is a paucity of literature on acceptable and reliable scales for measuring these. This study aimed to develop and test the acceptability and reliability of a scale which examined a broad range of environmental perceptions among children.
Methods: Based on constructs from ecological models, a survey incorporating items on children's perceptions of the physical and social environment at home and in the neighbourhood was developed. This was administered on two occasions, nine days apart, to a sample of 39 children aged 11 years (54% boys), attending a metropolitan Australian elementary school. The acceptability of the survey was determined by the proportion of missing responses to each item. The test-retest reliability of individual items, scores and scales were determined using Kappa statistics and percent agreement for categorical variables, and intraclass correlation coefficients (ICC) for continuous variables.
Results: There were few missing responses to each question, with only 4% of all responses missing. Although some Kappa values were low, all categorical variables showed acceptable reliability when examined for percent agreement between test and retest (range 68%–100% agreement). Continuous variables all showed moderate to good ICC values (range 0.72–0.92).
Conclusion: Findings suggest this questionnaire is reliable and acceptable to children for assessing environmental perceptions relevant to physical activity among 11-year-old children.

Automated attention flags in chronic disease care planning

Objectives: To assess the value of computerised decision support in the management of chronic respiratory disease by comparing agreement between three respiratory specialists, general practitioners (care coordinators), and decision support software.
Methods: Care guidelines for two chronic obstructive pulmonary disease projects of the SA HealthPlus Coordinated Care Trial were formulated. Decision support software, Care Plan On-Line (CPOL), was created to represent the intent of these guidelines via automated attention flags to appear in patients' electronic medical records. For a random sample of 20 patients with care plans, decisions about the use of nine additional services (eg,.smoking cessation, pneumococcal vaccination) were compared between the respiratory specialists, the patients' GPs and the CPOL attention flags.
Results: Agreement among the specialists was at the lower end of moderate (intraclass correlation coefficient [ICC], 0.48; 95% CI, 0.39-0.56), with a 20% rate of contradictory decisions. Agreement with recommendations of specialists was moderate to poor for GPs (le, 0.49; 95% CI, 0.33-0.66) and moderate to good for CPOL (K, 0.72; 95% CI, 0.55-0.90). CPOL agreement with GPs was moderate to poor (le, 0.41; 95% CI, 0.24-0.58). GPs were less likely than specialists or CPOL to decide in favour of an additional service (P< 0.001). CPOL was 87% accurate as an indicator of specialist decisions. It gave a 16% false-positive rate according to specialist decisions, and flagged 61% of decisions where GPs said No and specialists said Yes.
Conclusions: Automated decision support may provide GPs with improved access to the intent of guidelines; however, further investigation is required.

Development of a reliable measure of walking within and outside the local neighbourhood: RESIDE's neighborhood physical activity questionnarire.

Background: The RESIDential Environment project (RESIDE) is a longitudinal study evaluating the impact of a new residential design code on walking. Objective: To develop a reliable measure of walking – undertaken within and outside the neighborhood – and overall physical activity. Methods: A test–retest reliability study was undertaken (n = 82, mean age 39 years). The instrument was based on the International Physical Activity Questionnaire (IPAQ-short version) and Active Australia Survey. It measured usual frequency and duration of (1) recreational- and transport-related walking within and outside the neighborhood and (2) other vigorous and moderate physical activities. Results: Reliability of recall of whether participants had walked within (k = 0.84) and outside (0.73) the neighborhood was acceptable. Similarly, recall of frequency and duration of transport and recreational-related walking within the neighborhood was excellent (ICC ≥ 0.82), as was recall of transport-related walking trips outside the neighborhood (ICC ≥ 0.84). Reliability for duration of recreational walking outside the neighborhood was fair to good (ICC = 0.55). The reliability of indices of total physical activity based on MET min/week (ICC = 0.82) and MET min/week dichotomized to ‘sufficient’ physical activity for health (kappa = 0.67) were both acceptable. Conclusions:  The Neighborhood Physical Activity Questionnaire (NPAQ) is sufficiently reliable for studies examining environmental correlates of walking within the neighborhood.

A simplified stroke rehabilitation assessment of movement instrument

Background and Purpose. An efficient, reliable, and valid instrument for assessing motor function in patients with stroke is needed by both clinicians and researchers. To improve administration efficiency, we applied the multidimensional Rasch model to the 30-item, 3-subscale Stroke Rehabilitation Assessment of Movement (STREAM) instrument to produce a concise, reliable, and valid instrument (simplified STREAM [S-STREAM]) for measuring motor function in patients with stroke. Subjects and Methods. The STREAM (consisting of 3 subscales: upper-limb movements, lower-limb movements, and mobility) was administered to 351 subjects with first stroke occurrence and a median time after stroke of 19.5 months. The unidimensionality of each subscale of the STREAM first was verified with unidimensional Rasch analysis. Each subscale of the STREAM then was simplified by deleting redundant items on the basis of expert opinion and the results of the Rasch analysis. The Rasch reliability of the S-STREAM and the concurrent validity of the S-STREAM with the STREAM were examined with multidimensional Rasch analysis and the intraclass correlation coefficient (ICC), respectively. Results. After deleting the items that did not fit the Rasch model, we found that the 8-item upper-limb movement subscale, the 9-item lower-limb movement subscale, and the 10-item mobility subscale assessed single, unidimensional upper-limb movements, lower-limb movements, and mobility, respectively. We selected 5 items from each subscale to construct the S-STREAM and found that the reliability of each subscale of the resulting simplified instrument was high (Rasch reliability coefficients of [greater than or equal to] .91). The agreement between the subscale scores (Rasch estimates) of the S-STREAM and those of the STREAM was excellent (ICC of [greater than or equal to] .99, with a lower limit for the 95% confidence interval of [greater than or equal to] .985), indicating good concurrent validity of the S-STREAM with the STREAM. Discussion and Conclusion. The S-STREAM demonstrates high Rasch reliability, unidimensionality, and concurrent validity with the STREAM in patients with stroke. Furthermore, the S-STREAM is efficient to administer, as it consists of only half the number of items in the original STREAM. Additional studies to examine other psychometric properties (eg, predictive validity and responsiveness) of the S-STREAM or its psychometric properties in various recovery stages after stroke are needed to further establish its utility in both clinical and research settings.

Psychometric properties of the modified Emory functional ambulation profile in stroke patients

Objective: To examine the test – retest reliability, convergent and predictive validity and responsiveness of the modified Emory Functional Ambulation Profile (mEFAP) in assessing gait function in stroke patients undergoing rehabilitation. Subjects: Forty subacute stroke inpatients were enrolled in the validity and responsiveness study. Twenty chronic patients participated in the reliability study. Setting: Rehabilitation department of a medical centre. Design: In the validity and responsiveness study, patients were tested using the mEFAP, 10-m walking speed test, Barthel Index and Rivermead Mobility Index at admission and at discharge. For the reliability study, the patients completed the mEFAP twice one week apart. Results: The patients' performances on the mEFAP were moderately to highly correlated with results of the 10-m walking speed test and Rivermead Mobility Index (absolute correlation coefficients ≥ 0.67), indicating good convergent validity. Patients' performance on the mEFAP at admission was moderately correlated with the Barthel Index and Rivermead Mobility Index scores at discharge (Spearman's rank correlation coefficients = -0.52 and -0.78, respectively), indicating good predictive validity. The standardized response mean of the mEFAP was 1.1 (P < 0.0001), suggesting good responsiveness. The intraclass correlation coefficient ICC(3,1) for the mEFAP was 0.997, indicating excellent test – retest reliability. Conclusions: Our results provide strong evidence that the mEFAP has good reliability, validity and responsiveness for assessing stroke patients undergoing rehabilitation. The mEFAP is a useful scale for measuring walking function and recovery in stroke patients.

Evaluation of the Chinese version of the disability of the arm, shoulder and hand (DASH-HKPWH): cross-cultural adaptation process, internal consistency and reliability study

Abstract
The Disability of the Arm, Shoulder and Hand (DASH) was translated into Chinese by a physiotherapy team of the Prince of Wales Hospital, Hong Kong (DASH-HKPWH).

Objectives
This study evaluated the cross-cultural adaptation process, face validity, internal consistency and reliability of the DASH-HKPWH.

Method
Language officers and medical professionals from different fields were invited to translate and evaluate the face validity of the DASH-HKPWH. 88 patients were recruited to complete two DASH questionnaires on two occasions 1-2 weeks apart.

Results
Some adjustments were made to the translations based on the cultural and linguistic practice in Hong Kong. The face validity was satisfactory with a mean endorsement score of 3.2. The difference between the mean of DASH scores was not significant (t = −0.35, p = 0.73). The ICC (1,1) and Cronbach's alpha for the 30-item Disability/Symptom of the DASH-HKPWH was 0.77 and 0.94, respectively.

Conclusion
The translation was valid and reliable and acceptably equivalent to the original version. The questionnaire is suitable for measuring changes experienced by patients with any upper extremity disorders.

Cross-national agreement on disability weights:the Eurpoean disability weights project

Background: Disability weights represent the relative severity of disease stages to be incorporated in summary measures of population health. The level of agreement on disability weights in Western European countries was investigated with different valuation methods.

Methods: Disability weights for fifteen disease stages were elicited empirically in panels of health care professionals or non-health care professionals with an academic background following a strictly standardised procedure. Three valuation methods were used: a visual analogue scale (VAS); the time trade-off technique (TTO); and the person trade-off technique (PTO). Agreement among England, France, the Netherlands, Spain, and Sweden on the three disability weight sets was analysed by means of an intraclass correlation coefficient (ICC) in the framework of generalisability theory. Agreement among the two types of panels was similarly assessed.

Results: A total of 232 participants were included. Similar rankings of disease stages across countries were found with all valuation methods. The ICC of country agreement on disability weights ranged from 0.56 [95% CI, 0.52–0.62] with PTO to 0.72 [0.70–0.74] with VAS and 0.72 [0.69–0.75] with TTO. The ICC of agreement between health care professionals and non-health care professionals ranged from 0.64 [0.58–0.68] with PTO to 0.73 [0.71–0.75] with VAS and 0.74 [0.72–0.77] with TTO.

Conclusions: Overall, the study supports a reasonably high level of agreement on disability weights in Western European countries with VAS and TTO methods, which focus on individual preferences, but a lower level of agreement with the PTO method, which focuses more on societal values in resource allocation.

The validity and reliability of an instrument to assess children's outdoor play in various locations

The aim of this study was to develop and test the reliability and validity of survey items that examine the frequency with which primary school-aged children play in particular outdoor locations. Parents reported the number of days their child spent playing in specified outdoor locations (i.e., yard at home, own street/court/footpath, and park/playground) out-of-school hours on weekdays and weekend days during a typical week. To test the reliability of these items, the survey was administered on two occasions, 2 weeks apart, to a sample of 53 parents of children attending primary schools located in metropolitan Melbourne. The validity study involved the completion of a log book by 46 parents of primary school children over a 1-week period. Two weeks later, the same sample of parents completed the survey items. The test–retest reliability of individual items was determined using intra-class correlation coefficients (ICC). The kappa statistic and percent agreement between responses were used to assess validity by comparing the information provided in the log book with that provided in the survey. Results from the two studies suggest that the survey was generally a reliable and valid instrument for assessing the frequency with which children play in particular locations especially at home or in the street. Evidence of the reliability and validity of items assessing where children play is novel and important considering the need to promote children's physical activity in a variety of settings.

Reliability of surface EMG measurements of the quadriceps during maximal isometric contractions following water immersion

Background: The influence of water immersion on neuromuscular function is of importance to a number of disciplines; however, the reliability of surface electromyography (SEMG) following water immersion is not known. This study examined the reliability of SEMG amplitude during maximal voluntary isometric contractions (MVICs) of the vastus lateralis following water immersion.

Methods: Using a Biodex isokinetic dynamometer and in a randomized order, 12 healthy male subjects performed four MVICs at 60° knee flexion on both the dominant and nondominant kicking legs, and the SEMG was recorded. Each subject's dominant and nondominant kicking leg was then randomly assigned to have SEMG electrodes removed or covered during 15 min of water immersion (20°C–25°C). Following water immersion, subjects performed a further four MVICs.

Results: Intraclass correlation coefficient (ICC) and the relative standard error of measurement (%SEM) of SEMG amplitude showed moderate to high trial-to-trial reliability when electrodes were covered (0.93% and 2.79%) and removed (0.95% and 2.10%, respectively).

Conclusions: The results of the this study indicate that SEMG amplitude of the vastus lateralis may be accurately determined during maximal voluntary contractions following water immersion if electrodes are either removed or covered with water-resistive tape during the immersion.

Examination of EMG normalisation methods for the study of the posterior and posterolateral neck muscles in healthy controls

The purpose of this study was to examine the reliability of normalisation methods used in the study of the posterior and posterolateral neck muscles in a group of healthy controls. Six asymptomatic male subjects performed a total of 12 maximum voluntary isometric contractions (MVIC) and 60%-submaximal isometric contractions (60%-MVIC) against the torque arm of an isokinetic dynamometer whilst surface and intramuscular electromyography (EMG) was recorded unilaterally from representative posterior and posterolateral locations. Reliability was calculated using intra-class correlation coefficient (ICC), relative standard error of measurement (%SEM) and relative coefficient of variation (%CV). Maximal torque output was found to be highly reliable in the directions of extension and right lateral bending when the first of three MVIC contractions was excluded. When averaged across contraction direction, high reliability was found for both surface (MVIC: ICC = 0.986, %SEM = 7.5, %CV = 9.2; 60%-MVIC: ICC = 0.975, %SEM = 10, %CV = 13.7) and intramuscular (MVIC: ICC = 0.910, %SEM = 20, %CV = 19.1; 60%-MVIC: ICC = 0.952, %SEM = 16.5, %CV = 13.5) electrodes. Intramuscular electrodes displayed the least reliability in right lateral bending. The use of visual feedback markedly increased the reliability of 60%-MVIC contractions.

Reliability of normalisation methods for EMG analysis of neck muscles

Acceptable reliability of normalisation contractions in electromyography (EMG) is paramount for testing conducted over a number of days or if normal laboratory strength testing equipment is unavailable. This study examined the reliability of maximal voluntary isometric contractions (MVIC) and sub-maximal (60%) isometric contractions for use in neck muscle EMG studies. Surface EMG was recorded bilaterally from eight sites around the neck at C4/5 level from five healthy male subjects. Subjects performed MVIC and sub-maximal normalisation contractions using an isokinetic dynamometer (ID) and a portable cable dynamometer with attached strain gauge (PCD) in addition to a MVIC against a manual resistance (MR). Subjects were tested in flexion, extension, left and right lateral bending and were retested by the same tester within a two-week period. Intra class correlation co-efficients (ICC) were calculated for each testing method and contraction direction and a mean ICC was calculated across all contraction directions. All normalisation methods produced excellent within-day reliability (mean ICC >0.80) but only the MVICs using the ID and PCD had acceptable reliability when assessed between-days. This study confirmed the validity of using MVICs elicited using the ID and PCD as reliable reference contractions for the normalisation of neck EMG.

Day-to-day and seasonal variability of physical activity during school recess

Objective : Children's physical activity levels are difficult to establish on a day-to-day or season-to-season basis. Most studies have attempted to measure reliability in habitual settings. This study investigated the variability in children's physical activity during recess.

Methods : Fifteen boys and 19 girls (aged 6 to 11 years) from 2 schools in North West England wore heart rate monitors for 5 consecutive days in summer and winter terms to assess day-to-day and seasonal variability during school recess. Data were collected in 2004. Repeated measures ANOVA's and intraclass correlations (ICC) analysed the day-to-day and seasonal variability in children's moderate-to-vigorous (MVPA) and vigorous physical activity (VPA) data.

Results : There were no significant differences in children's MVPA and VPA across days and seasons. ICCs for MVPA across 2 days ranged from 0.75 to 0.85 in summer, and from 0.53 to 0.81 in winter. Three-day MVPA ICCs were 0.83 in summer and 0.71 in winter.

Conclusions : The results revealed no significant variation in children's recess physical activity levels across days and seasons. Whilst children were free to choose their recess activities in school, the results suggested that children were relatively consistent in their choices, limiting physical activity variability.

The reliability of the 1RM strength test for untrained middle-aged individuals

The one-repetition maximum (1RM) test is considered the gold standard for assessing muscle strength in non-laboratory situations. Since most previous 1RM reliability studies have been conducted with experienced young participants, it is unclear if acceptable test–retest reliability exists for untrained middle-aged individuals. This study examined the reliability of the 1RM strength test of untrained middle-aged individuals. Fifty-three untrained males (n = 25) and females (n = 28) aged 51.2 ± 0.9 years participated in the study. Participants undertook the first 1RM test (T1) 4–8 days after a familiarisation session with the same exercises. 1RM was assessed for seven different exercises. Four to eight days after T1, participants underwent another identical 1RM test (T2). Ten weeks later, 27 participants underwent a third test (T3). Intraclass correlation coefficients (ICC), typical error as a coefficient of variation (TEcv), retest correlation, repeated measures ANOVA, Bland–Altman plots, and estimation of 95% confidence limits were used to assess reliability. A high ICC (ICC > 0.99) and high correlation (r > 0.9) were found for all exercises. TEcv ranged from 2.2 to 10.1%. No significant change was found for six of the seven exercises between T1 and T2. Leg press was slightly higher at T2 compared to T1 (1.6 ± 0.6%, p = 0.02). No significant change was found between T2 and T3 for any exercise. 1RM is a reliable method of evaluating the maximal strength in untrained middle-aged individuals. It appears that 1RM-testing protocols that include one familiarisation session and one testing session are sufficient for assessing maximal strength in this population.

Reliability of isokinetic strength and aerobic power testing for patients with chronic heart failure

PURPOSE: The objective of this study was to assess the reliability of testing skeletal muscle strength and peak aerobic power in a clinical population of patients with chronic heart failure (CHF).

METHODS: Thirty-three patients with CHF (New York Heart Association (NYHA) Functional Class 2.3 ± 0.5; left ventricular ejection fraction 27% ± 7%; age 65 ± 9 years; 28:5 male-female ratio) underwent two identical series of tests (T1 and T2), 1 week apart, for strength and endurance of the muscle groups responsible for knee extension/flexion and elbow extension/flexion. The patients also underwent two graded exercise tests on a bicycle ergometer to measure peak oxygen consumption (VO_2peak). Three months later, 18 of the patients underwent a third test (T3) for each of the measures. Means were compared using MANOVA with repeated measures for strength and endurance, and ANOVA with repeated measures for VO_2peak.

RESULTS: Combining data for all four movement patterns, the expression of strength increased from T1 to T2 by 12% ± 25% (P < .001; intraclass correlation coefficient [ICC] = 0.89). Correspondingly, endurance increased by 13% ± 23% (P = .004; ICC = 0.87). Peak oxygen consumption was not significantly different (16.2 ± 0.8 and 16.1 ± 0.8 mL·kg^-1·min^-1 for T1 and T2, respectively;P = .686; ICC = 0.91). There were no significant differences between T2 and T3 for strength (2% ± 17%;P = .736; ICC = 0.92) or muscle endurance (-1% ± 15%;P = .812; ICC = 0.96), but VO_2peak decreased from 16.7 ± 1.2 to 14.9 ± 0.9 mL·kg^-1·min^-1 (-10% ± 18%;P = .021; ICC = 0.89).

CONCLUSIONS: These data suggest that in a population of patients with CHF, a familiarization trial for skeletal muscle strength testing is necessary. Although familiarization is not required for assessing oxygen consumption as a single measurement, VO_2peak declined markedly in the 3-month period for which these patients were followed. Internal consistency within patients was high for the second and third strength trials and the first and second tests of VO_2peak.