Reliability of the Knee Examination in Osteoarthritis: Effect of Standardization

Objective. To assess the reliability of physical examination of the osteoarthritic (OA) knee by rheumatologists, and to evaluate the benefits of standardization. Methods. Forty-two physical signs and techniques were evaluated using a 6 X 6 Latin square design. Patients with mild to severe knee OA, based on physical and radiographic signs, were examined in random order prior to and following standardization of techniques. For those signs with dichotomous scales, agreement among the rheumatologists was calculated as the prevalence-adjusted bias-adjusted kappa (PABAK), while for the signs with continuous and ordinal scales, a reliability coefficient (R-c) was calculated using analysis of variance. A PABAK of >0.60 and an Re of >0.80 were considered to indicate adequate reliability. Results. Adequate poststandardization reliability was achieved for 30 of 42 physical signs/techniques (71%). The most highly reliable signs identified by physical examination of the OA knee included alignment by goniometer (R-c = 0.99), bony swelling (R-c = 0.97), general passive crepitus (R-c = 0.96), gait by inspection (PABAK = 0.78), effusion bulge sign (R-c = 0.97), quadriceps atrophy (R. = 0.97), medial tibiofemoral tenderness (R-c = 0.94), lateral tibiofemoral tenderness (R-c = 0.85), patellofemoral tenderness by grind test (R-c = 0.94), and flexion contracture (R-c = 0.95). The standardization process resulted in substantial improvements in reliability for evaluation of a number of physical signs, although for some signs, minimal or no effect of standardization was noted. After standardization, warmth (PABAK = 0.14), medial instability at 30degrees flexion (PABAK = 0.02), and lateral instability at 30degrees flexion (PABAK = 0.34) were the only 3 signs that were highly unreliable. Conclusion. With the exception of physical examinations for instability, a comprehensive knee examination can be performed with adequate reliability. Standardization further improves the reliability for some physical signs and techniques. The application of these findings to future OA studies will contribute to improved outcome assessments in OA.

Test-retest reliability of four physical activity measures used in population surveys

Accurate monitoring of prevalence and trends in population levels of physical activity (PA) is a fundamental public health need. Test-retest reliability (repeatability) was assessed in population samples for four self-report PA measures: the Active Australia survey (AA, N=356), the short International Physical Activity Questionnaire (IPAQ, N=104), the physical activity items in the Behavioral Risk Factor Surveillance System (BRFSS, N=127) and in the Australian National Health Survey (NHS, N=122). Percent agreement and Kappa statistics were used to assess reliability of classification of activity status as 'active', 'insufficiently active' or 'sedentary'. Intraclass correlations (ICCs) were used to assess agreement on minutes of activity reported for each item of each survey and for total minutes. Percent agreement scores for activity status were very good on all four instruments, ranging from 60% for the NHS to 79% for the IPAQ. Corresponding Kappa statistics ranged from 0.40 (NHS) to 0.52 (AA). For individual items, ICCs were highest for walking (0.45 to 0.78) and vigorous activity (0.22 to 0.64) and lowest for the moderate questions (0.16 to 0.44). All four measures provide acceptable levels of test-retest reliability for assessing both activity status and sedentariness, and moderate reliability for assessing total minutes of activity.

Random error in willingness to pay measurement: A multiple indicators, latent variable approach to the reliability of contingent values

The reliability of measurement refers to unsystematic error in observed responses. Investigations of the prevalence of random error in stated estimates of willingness to pay (WTP) are important to an understanding of why tests of validity in CV can fail. However, published reliability studies have tended to adopt empirical methods that have practical and conceptual limitations when applied to WTP responses. This contention is supported in a review of contingent valuation reliability studies that demonstrate important limitations of existing approaches to WTP reliability. It is argued that empirical assessments of the reliability of contingent values may be better dealt with by using multiple indicators to measure the latent WTP distribution. This latent variable approach is demonstrated with data obtained from a WTP study for stormwater pollution abatement. Attitude variables were employed as a way of assessing the reliability of open-ended WTP (with benchmarked payment cards) for stormwater pollution abatement. The results indicated that participants' decisions to pay were reliably measured, but not the magnitude of the WTP bids. This finding highlights the need to better discern what is actually being measured in VVTP studies, (C) 2003 Elsevier B.V. All rights reserved.

Papillary urothelial neoplasm of low malignant potential: reliability of diagnosis and outcome

OBJECTIVE To determine the ability of pathologists to reproducibly diagnose a newly defined lesion, i.e. the papillary urothelial neoplasm of low malignant potential (PUNLMP) using the published criteria, defined by the 1998 World Health Organisation/International Society of Urological Pathology (WHO/ISUP) classification system; in addition, debate remains about the clinical behaviour of these lesions, thus the rates of recurrence and progression of PUNLMP lesions were assessed and compared with low-grade papillary urothelial carcinomas (LG-PUC) and high-grade (HG-PUC) over a 10-year follow-up. PATIENTS AND METHODS Forty-nine cases of superficial bladder cancer (G1-3 pTa) representing an initial diagnosis of transitional cell carcinoma made in 1990 were identified and re-graded using the 1998 WHO/ISUP classification by two pathologists. Inter-observer agreement was assessed using Cohen weighted kappa statistics. After reclassification the clinical follow-up was reviewed retrospectively, and episodes of recurrence and progression recorded. RESULTS The inter-observer agreement was moderate, regardless of whether one (kappa 0.45) or two (kappa 0.60) pathologists were used to grade these lesions. Re-classification identified 12 PUNLMP, 28 LG-PUC and nine HG-PUC. PUNLMP lesions recurred in 25% (3/12) of cases; no progression was documented. Recurrence rates were 75% (21/28) and 67% (6/9) for LG- and HG-PUC, respectively, and progression rates were 4% (1/28) and 22% (2/9). CONCLUSION The 1998 WHO/ISUP classification of urothelial neoplasms can be reproducibly applied by pathologists, with a moderate level of agreement. There is evidence that PUNLMP lesions have a more indolent clinical behaviour than urothelial carcinomas. However, the risk of recurrence and progression remains, and clinical monitoring of these patients is important.

The concurrent validity and test-retest reliability of a visuospatial working memory task

To determine whether the visuospatial n-back working memory task is a reliable and valid measure of cognitive processes believed to underlie intelligence, this study compared the reaction times and accuracy of perforniance of 70 participants, with performance on the Multidimensional Aptitude Battery (MAB). Testing was conducted over two sessions separated by 1 week. Participants completed the MAB during the second test session. Moderate testretest reliability for percentage accuracy scores was found across the four levels of the n-back task, whilst reaction times were highly reliable. Furthermore, participants' performance on the MAB was negatively correlated with accuracy of performance at the easier levels of the n-back task and positively correlated with accuracy of performance at the harder task levels. These findings confirm previous research examining the cognitive basis of intelligence, and suggest that intelligence is the product of faster speed of information processing, as well as superior working memory capacity. (C) 2004 Elsevier Inc. All rights reserved.

Testing the reliability of a new ultrasonic cardiac output monitor, the USCOM, by using aortic flowprobes in anesthetized dogs

We have used an animal model to test the reliability of a new portable continuous-wave Doppler ultrasonic cardiac output monitor, the USCOM. In six anesthetized dogs, cardiac output was measured with a high-precision transit time ultrasonic flowprobe placed on the ascending aorta. The dogs' cardiac output was increased with a dopamine infusion (0-15 mug (.) kg(-1) (.) min(-1)). Simultaneous flowprobe and USCOM cardiac output measurements were made. Up to 64 pairs of readings were collected from each dog. Data were compared by using the Bland and Altman plot method and Lin's concordance correlation coefficient. A total of 319 sets of paired readings were collected. The mean (+/-SD) cardiac output was 2.62 +/- 1.04 L/min, and readings ranged from 0.79 to 5.73 L/min. The mean bias between the 2 sets of readings was -0.01 L/min, with limits of agreement (95% confidence intervals) of -0.34 to 0.31 L/min. This represents a 13% error. In five of six dogs, there was a high degree of concordance, or agreement, between the 2 methods, with coefficients >0.9. The USCOM provided reliable measurements of cardiac output over a wide range of values. Clinical trials are needed to validate the device in humans.

Reliability of a measure of prediagnosis physical activity for cancer survivors

Purpose: This study was conducted to examine the test-retest reliability of a measure of prediagnosis physical activity participation administered to colorecial cancer survivors recruited from a population-based state cancer registry. Methods: A total of 112 participants completed two telephone interviews. I month apart, reporting usual weekly physical activity in the year before their cancer diagnosis. Intraclass correlation coefficients (ICC) and standard en-or of measurement (SEM) were used to describe the test-retest reliability of the measure across the sample: the Bland-Altman approach was used to describe reliability at the individual level. The test-retest reliability for categorized total physical activity (active, insufficiently active, sedentary) was assessed using the kappa statistic. Results: When the complete sample was considered, the ICC ranged from 0.40 (95% Cl: 0.24, 0.55) for vigorous gardening to 0.77 (95% Cl: 0.68, 0.84) for moderate physical activity. The SEM, however, were large. indicating high measurement error. The Bland-Altman plots indicated that the reproducibility of data decreases as the aniount of physical activity reported each week increases The kappa coefficient for the categorized data was 0.62 (95% Cl: 0.48, 0.76). Conclusion: Overall. the results indicated low levels of repeatability for this measure of historical physical activity. Categorizing participants as active, insufficiently active, or sedentary provides a higher level of test-retest reliability.

The reliability of ten-meter sprint time using different starting techniques

Acceleration is an important factor for success in team-sport athletes. The purpose of this investigation was to compare the reliability of 10-m sprint times when using different starting techniques. Junior male rugby players (n=15) were tested for speed over 10 m on 2 different testing sessions. Three trials of 3 different starting techniques (standing, foot, and thumb starts) were assessed. Despite large differences in the time taken to perform 10-m sprints from different starts, there was minimal difference in the typical error (similar to 0.02 seconds, or

Validation and reliability of the Modified Elderly Mobility Scale

Objective: To establish concurrent validity, interrater and test-retest reliability of the Modified Elderly Mobility Scale (MEMS). Methods: Ninety elderly patients were scored on the MEMS. To establish concurrent validity, 75 patients MEMS scores were compared to Functional Independence Measure (FIM) scores using Spearman's correlation. Videotaped patient performances were used to establish interrater and test-retest reliability using percentage absolute agreement and intraclass correlation coefficients (ICCs). Results: The total MEMS score demonstrated a significant association with the motor (r = 0.725) and total FIM scores (r = 0.718). Absolute agreement for interrater reliability was greater than 93% for all test items, with 97 and 98% for the two new measures, respectively. Test-retest reliability demonstrated similar high levels of absolute agreement and had ICCs ranging from 0.870 to 1.0. Conclusions: The MEMS is a quick, valid and reliable test of motor function of elderly patients with a spread of functional levels.