Test-retest reliability of physical activity measures used in Australian population surveys

PURPOSE Accurate monitoring of prevalence and trends in population levels of physical activity is fundamental to the planning of health promotion and disease-prevention strategies. Test-retest reliability (repeatability) was assessed for four self-report measures of physical activity commonly used in population surveys: the Active Australia survey (AA, N=356), the short form of the International Physical Activity Questionnaire (IPAQ-S, N=104), the physical activity items in the Behavioral Risk Factor Surveillance System (BRFSS, N=127) and the physical activity items in the Australian National Health Survey (NHS, N=122). METHODS Percent agreement and Kappa statistics were used to assess the reliability of classification of activity status (where ‘active’= 150 minutes of activity per week) and sedentariness (where ‘sedentary’ = reporting no physical activity). Intraclass correlations (ICCs) were used to assess agreement on minutes of activity reported for each item of each survey and on total minutes reported in each survey. RESULTS Percent agreement scores for both activity status and sedentariness were very good on all four instruments. Overall the percent agreement between repeated surveys was between 73% (NHS) and 87% (IPAQ) for the criterion measure of achieving 150 minutes per week, and between 77% (NHS) and 89% (IPAQ) for the criterion of being sedentary. Corresponding Kappa statistics ranged from 0.46 (NHS) to 0.61 (AA) for activity status and from 0.20 (BRFSS) to 0.52 (AA) for sedentariness. For the individual items ICCs were highest for walking (0.45 to 0.56) and vigorous activity (0.22 to 0.64) and lowest for the moderate questions (0.16 to 0.44). CONCLUSION All four measures provide acceptable levels of test-retest reliability for assessing both activity status and sedentariness, and moderate reliability for assessing total minutes of activity. Supported by the Australian Commonwealth Department of Health and Ageing.

Reliability in the Parameterization of the Functional Reach Test in Elderly Stroke Patients: A Pilot Study

BACKGROUND: Postural instability is one of the major complications found in stroke survivors. Parameterising the functional reach test (FRT) could be useful in clinical practice and basic research. OBJECTIVES: To analyse the reliability, sensitivity, and specificity in the FRT parameterisation using inertial sensors for recording kinematic variables in patients who have suffered a stroke. DESIGN: Cross-sectional study. While performing FRT, two inertial sensors were placed on the patient's back (lumbar and trunk). PARTICIPANTS: Five subjects over 65 who suffer from a stroke. MEASUREMENTS: FRT measures, lumbosacral/thoracic maximum angular displacement, maximum time of lumbosacral/thoracic angular displacement, time return initial position, and total time. Speed and acceleration of the movements were calculated indirectly. RESULTS: FRT measure is  12.75±2.06 cm. Intrasubject reliability values range from 0.829 (time to return initial position (lumbar sensor)) to 0.891 (lumbosacral maximum angular displacement). Intersubject reliability values range from 0.821 (time to return initial position (lumbar sensor)) to 0.883 (lumbosacral maximum angular displacement). FRT's reliability was 0.987 (0.983-0.992) and 0.983 (0.979-0.989) intersubject and intrasubject, respectively. CONCLUSION: The main conclusion could be that the inertial sensors are a tool with excellent reliability and validity in the parameterization of the FRT in people who have had a stroke.

Reliability and criterion-related validity with a smartphone used in timed-up-and-go test

Background The capacity to diagnosys, quantify and evaluate movement beyond the general confines of a clinical environment under effectiveness conditions may alleviate rampant strain on limited, expensive and highly specialized medical resources. An iPhone 4® mounted a three dimensional accelerometer subsystem with highly robust software applications. The present study aimed to evaluate the reliability and concurrent criterion-related validity of the accelerations with an iPhone 4® in an Extended Timed Get Up and Go test. Extended Timed Get Up and Go is a clinical test with that the patient get up from the chair and walking ten meters, turn and coming back to the chair. Methods A repeated measure, cross-sectional, analytical study. Test-retest reliability of the kinematic measurements of the iPhone 4® compared with a standard validated laboratory device. We calculated the Coefficient of Multiple Correlation between the two sensors acceleration signal of each subject, in each sub-stage, in each of the three Extended Timed Get Up and Go test trials. To investigate statistical agreement between the two sensors we used the Bland-Altman method. Results With respect to the analysis of the correlation data in the present work, the Coefficient of Multiple Correlation of the five subjects in their triplicated trials were as follows: in sub-phase Sit to Stand the ranged between r = 0.991 to 0.842; in Gait Go, r = 0.967 to 0.852; in Turn, 0.979 to 0.798; in Gait Come, 0.964 to 0.887; and in Turn to Stand to Sit, 0.992 to 0.877. All the correlations between the sensors were significant (p < 0.001). The Bland-Altman plots obtained showed a solid tendency to stay at close to zero, especially on the y and x-axes, during the five phases of the Extended Timed Get Up and Go test. Conclusions The inertial sensor mounted in the iPhone 4® is sufficiently reliable and accurate to evaluate and identify the kinematic patterns in an Extended Timed Get and Go test. While analysis and interpretation of 3D kinematics data continue to be dauntingly complex, the iPhone 4® makes the task of acquiring the data relatively inexpensive and easy to use.

Parameterization and reliability of single-leg balance test assessed with inertial sensors in stroke survivors: a cross-sectional study

Background and purpose There are no published studies on the parameterisation and reliability of the single-leg stance (SLS) test with inertial sensors in stroke patients. Purpose: to analyse the reliability (intra-observer/inter-observer) and sensitivity of inertial sensors used for the SLS test in stroke patients. Secondary objective: to compare the records of the two inertial sensors (trunk and lumbar) to detect any significant differences in the kinematic data obtained in the SLS test. Methods Design: cross-sectional study. While performing the SLS test, two inertial sensors were placed at lumbar (L5-S1) and trunk regions (T7–T8). Setting: Laboratory of Biomechanics (Health Science Faculty - University of Málaga). Participants: Four chronic stroke survivors (over 65 yrs old). Measurement: displacement and velocity, Rotation (X-axis), Flexion/Extension (Y-axis), Inclination (Z-axis); Resultant displacement and velocity (V): RV=(Vx2+Vy2+Vz2)−−−−−−−−−−−−−−−−−√ Along with SLS kinematic variables, descriptive analyses, differences between sensors locations and intra-observer and inter-observer reliability were also calculated. Results Differences between the sensors were significant only for left inclination velocity (p = 0.036) and extension displacement in the non-affected leg with eyes open (p = 0.038). Intra-observer reliability of the trunk sensor ranged from 0.889-0.921 for the displacement and 0.849-0.892 for velocity. Intra-observer reliability of the lumbar sensor was between 0.896-0.949 for the displacement and 0.873-0.894 for velocity. Inter-observer reliability of the trunk sensor was between 0.878-0.917 for the displacement and 0.847-0.884 for velocity. Inter-observer reliability of the lumbar sensor ranged from 0.870-0.940 for the displacement and 0.863-0.884 for velocity. Conclusion There were no significant differences between the kinematic records made by an inertial sensor during the development of the SLS testing between two inertial sensors placed in the lumbar and thoracic regions. In addition, inertial sensors. Have the potential to be reliable, valid and sensitive instruments for kinematic measurements during SLS testing but further research is needed.

Test-retest reliability of graph theory measures of structural brain connectivity

The human connectome has recently become a popular research topic in neuroscience, and many new algorithms have been applied to analyze brain networks. In particular, network topology measures from graph theory have been adapted to analyze network efficiency and 'small-world' properties. While there has been a surge in the number of papers examining connectivity through graph theory, questions remain about its test-retest reliability (TRT). In particular, the reproducibility of structural connectivity measures has not been assessed. We examined the TRT of global connectivity measures generated from graph theory analyses of 17 young adults who underwent two high-angular resolution diffusion (HARDI) scans approximately 3 months apart. Of the measures assessed, modularity had the highest TRT, and it was stable across a range of sparsities (a thresholding parameter used to define which network edges are retained). These reliability measures underline the need to develop network descriptors that are robust to acquisition parameters.

Validation and reliability of a Spanish version of Simple Shoulder Test (SST-Sp)

Background: The Simple Shoulder Test (SST-Sp) is a widely used outcome measure. Objective: The purpose of this study was to develop and validate a Spanish-version SST (SST-Sp). Methods: A two-stage observational study was conducted. The SST was initially cross-culturally adapted to Spanish through double forward and backward translation and then validated for its psychometric characteristics. Participants (n = 66) with several shoulder disorders completed the SST-Sp, DASH, VAS and SF-12. The full sample was employed to determine factor structure, internal consistency and concurrent criterion validity. Reliability was determined in the first 24–48 h in a subsample of 21 patients. Results: The SST-Sp showed three factors that explained the 56.1 % of variance, and the internal consistency for each factor was α = 0.738, 0.723 and 0.667, and reliability was ICC = 0.687–0.944. The factor structure was three-dimensional and supported construct validity. Criterion validity determined from the relationship between the SST-Sp and DASH was strong (r = −0.73; p < 0.001) and fair for VAS (r = −0.537; p < 0.001). Relationships between SST-Sp and SF-12 were weak for both physical (r = −0.47; p < 0.001) and mental (r = −0.43; p < 0.001) dimensions. Conclusions: The SST-Sp supports the findings of the original English version as being a valid shoulder outcome measure with similar psychometric properties to the original English version.

The reliability and validity of the Severity of Dependence Scale for detecting cannabis dependence in psychosis.

Aims: To determine the reliability and validity of the Severity of Dependence Scale (SDS) for detecting cannabis dependence in a large sample of in-patients with a schizophrenia spectrum disorder. Design: Cross-sectional study. Participants: Participants were 153 in-patients with a schizophrenia spectrum disorder in Brisbane, Australia. Measurements: Participants were administered the SDS for cannabis dependence in the past 12 months. The presence of Diagnostic and Statistical Manual Version-IV (DSM-IV) cannabis dependence in the previous 12 months was assessed using the Comprehensive International Diagnostic Interview (CIDI). Findings: The SDS had high levels of internal consistency and strong construct and concurrent validity. Individuals with a score of ≥2 on the SDS were nearly 30 times more likely to have DSM-IV cannabis dependence. The SDS was the strongest predictor of DSM-IV cannabis dependence after controlling for other predictor variables. Conclusions: The SDS is a brief, valid and reliable screen for cannabis dependence among people with psychosis

Assessment of lifetime cumulative sun exposure using a self-administered questionnaire : reliability of two approaches

Few studies have evaluated the reliability of lifetime sun exposure estimated from inquiring about the number of hours people spent outdoors in a given period on a typical weekday or weekend day (the time-based approach). Some investigations have suggested that women have a particularly difficult task in estimating time outdoors in adulthood due to their family and occupational roles. We hypothesized that people might gain additional memory cues and estimate lifetime hours spent outdoors more reliably if asked about time spent outdoors according to specific activities (an activity-based approach). Using self-administered, mailed questionnaires, test-retest responses to time-based and to activity-based approaches were evaluated in 124 volunteer radiologic technologist participants from the United States: 64 females and 60 males 48 to 80 years of age. Intraclass correlation coefficients (ICC) were used to evaluate the test-retest reliability of average number of hours spent outdoors in the summer estimated for each approach. We tested the differences between the two ICCs, corresponding to each approach, using a t test with the variance of the difference estimated by the jackknife method. During childhood and adolescence, the two approaches gave similar ICCs for average numbers of hours spent outdoors in the summer. By contrast, compared with the time-based approach, the activity-based approach showed significantly higher ICCs during adult ages (0.69 versus 0.43, P = 0.003) and over the lifetime (0.69 versus 0.52, P = 0.05); the higher ICCs for the activity-based questionnaire were primarily derived from the results for females. Research is needed to further improve the activity-based questionnaire approach for long-term sun exposure assessment. (Cancer Epidemiol Biomarkers Prev 2009;18(2):464–71)

The Validity of the Brief Michigan Alcohol Screening Test (bMAST) as a Problem Drinking Severity Measure

Objective: The Brief Michigan Alcoholism Screening Test (bMAST) is a 10-item test derived from the 25-item Michigan Alcoholism Screening Test (MAST). It is widely used in the assessment of alcohol dependence. In the absence of previous validation studies, the principal aim of this study was to assess the validity and reliability of the bMAST as a measure of the severity of problem drinking. Method: There were 6,594 patients (4,854 men, 1,740 women) who had been referred for alcohol-use disorders to a hospital alcohol and drug service who voluntarily participated in this study. Results: An exploratory factor analysis defined a two-factor solution, consisting of Perception of Current Drinking and Drinking Consequences factors. Structural equation modeling confirmed that the fit of a nine-item, two-factor model was superior to the original one-factor model. Concurrent validity was assessed through simultaneous administration of the Alcohol Use Disorders Identification Test (AUDIT) and associations with alcohol consumption and clinically assessed features of alcohol dependence. The two-factor bMAST model showed moderate correlations with the AUDIT. The two-factor bMAST and AUDIT were similarly associated with quantity of alcohol consumption and clinically assessed dependence severity features. No differences were observed between the existing weighted scoring system and the proposed simple scoring system. Conclusions: In this study, both the existing bMAST total score and the two-factor model identified were as effective as the AUDIT in assessing problem drinking severity. There are additional advantages of employing the two-factor bMAST in the assessment and treatment planning of patients seeking treatment for alcohol-use disorders. (J. Stud. Alcohol Drugs 68: 771-779,2007)

A new method for improving reliability and line loss in distribution networks

In this paper, both Distributed Generators (DG) and capacitors are allocated and sized optimally for improving line loss and reliability. The objective function is composed of the investment cost of DGs and capacitors along with loss and reliability which are converted to the genuine dollar. The bus voltage and line current are considered as constraints which should be satisfied during the optimization procedure. Hybrid Particle Swarm Optimization as a heuristic based technique is used as the optimization method. The IEEE 69-bus test system is modified and employed to evaluate the proposed algorithm. The results illustrate that the lowest cost planning is found by optimizing both DGs and capacitors in distribution networks.

Parents’ self-efficacy, outcome expectations, and self-reported task performance when managing atopic dermatitis in children: Instrument reliability and validity

BACKGROUND: Support and education for parents faced with managing a child with atopic dermatitis is crucial to the success of current treatments. Interventions aiming to improve parent management of this condition are promising. Unfortunately, evaluation is hampered by lack of precise research tools to measure change. OBJECTIVES: To develop a suite of valid and reliable research instruments to appraise parents' self-efficacy for performing atopic dermatitis management tasks; outcome expectations of performing management tasks; and self-reported task performance in a community sample of parents of children with atopic dermatitis. METHODS: The Parents' Eczema Management Scale (PEMS) and the Parents' Outcome Expectations of Eczema Management Scale (POEEMS) were developed from an existing self-efficacy scale, the Parental Self-Efficacy with Eczema Care Index (PASECI). Each scale was presented in a single self-administered questionnaire, to measure self-efficacy, outcome expectations, and self-reported task performance related to managing child atopic dermatitis. Each was tested with a community sample of parents of children with atopic dermatitis, and psychometric evaluation of the scales' reliability and validity was conducted. SETTING AND PARTICIPANTS: A community-based convenience sample of 120 parents of children with atopic dermatitis completed the self-administered questionnaire. Participants were recruited through schools across Australia. RESULTS: Satisfactory internal consistency and test-retest reliability was demonstrated for all three scales. Construct validity was satisfactory, with positive relationships between self-efficacy for managing atopic dermatitis and general perceived self-efficacy; self-efficacy for managing atopic dermatitis and self-reported task performance; and self-efficacy for managing atopic dermatitis and outcome expectations. Factor analyses revealed two-factor structures for PEMS and PASECI alike, with both scales containing factors related to performing routine management tasks, and managing the child's symptoms and behaviour. Factor analysis was also applied to POEEMS resulting in a three-factor structure. Factors relating to independent management of atopic dermatitis by the parent, involving healthcare professionals in management, and involving the child in the management of atopic dermatitis were found. Parents' self-efficacy and outcome expectations had a significant influence on self-reported task performance. CONCLUSIONS: Findings suggest that PEMS and POEEMS are valid and reliable instruments worthy of further psychometric evaluation. Likewise, validity and reliability of PASECI was confirmed.

Sensitivity of the lane change test as a measure of in-vehicle system demand

The Lane Change Test (LCT) is one of the growing number of methods developed to quantify driving performance degradation brought about by the use of in-vehicle devices. Beyond its validity and reliability, for such a test to be of practical use, it must also be sensitive to the varied demands of individual tasks. The current study evaluated the ability of several recent LCT lateral control and event detection parameters to discriminate between visual-manual and cognitive surrogate In-Vehicle Information System tasks with different levels of demand. Twenty-seven participants (mean age 24.4 years) completed a PC version of the LCT while performing visual search and math problem solving tasks. A number of the lateral control metrics were found to be sensitive to task differences, but the event detection metrics were less able to discriminate between tasks. The mean deviation and lane excursion measures were able to distinguish between the visual and cognitive tasks, but were less sensitive to the different levels of task demand. The other LCT metrics examined were less sensitive to task differences. A major factor influencing the sensitivity of at least some of the LCT metrics could be the type of lane change instructions given to participants. The provision of clear and explicit lane change instructions and further refinement of its metrics will be essential for increasing the utility of the LCT as an evaluation tool.

A measurement instrument for process modeling research : development, test and procedural model

Process modeling is an emergent area of Information Systems research that is characterized through an abundance of conceptual work with little empirical research. To fill this gap, this paper reports on the development and validation of an instrument to measure user acceptance of process modeling grammars. We advance an extended model for a multi-stage measurement instrument development procedure, which incorporates feedback from both expert and user panels. We identify two main contributions: First, we provide a validated measurement instrument for the study of user acceptance of process modeling grammars, which can be used to assist in further empirical studies that investigate phenomena associated with the business process modeling domain. Second, in doing so, we describe in detail a procedural model for developing measurement instruments that ensures high levels of reliability and validity, which may assist fellow scholars in executing their empirical research.

Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.