Intratherapist reliability in the rating of scapula posture in multiple planes of reference

Background: Evaluation of scapular posture is a fundamental component in the clinical evaluation of the upper quadrant. This study examined the intrarater reliability of scapular posture ratings. Methods: A test-retest reliability investigation was undertaken with one week between assessment sessions. At each session physical therapists conducted visual assessments of scapula posture (relative to the thorax) in five different scapula postural planes (plane of scapula, sagittal plane, transverse plane, horizontal plane, and vertical plane). These five plane ratings were performed for four different scapular posture perturbating conditions (rest, isometric shoulder; flexion, abduction, and external rotation). Results. A total of 100 complete scapular posture ratings (50 left, 50 right) were undertaken at each assessment. The observed agreement between the test and retest postural plane ratings ranged from 59% to 87%; 16 of the 20 plane-condition combinations exceeded 75% observed agreement. Kappa (and prevalence adjusted bias adjusted kappa) values were inconsistent across the postural planes and perturbating conditions. Conclusions: This investigation generally revealed fair to moderate intrarater reliability in the rating of scapular posture by visual inspection. However, enough disagreement between assessments was present to warrant caution when interpreting perceived changes in scapula position between longitudinal assessments using visual inspection alone.

Pitfalls in the use of kappa when interpreting agreement between multiple raters in reliability studies

OBJECTIVE To compare different reliability coefficients (exact agreement, and variations of the kappa (generalised, Cohen's and Prevalence Adjusted and Biased Adjusted (PABAK))) for four physiotherapists conducting visual assessments of scapulae. DESIGN Inter-therapist reliability study. SETTING Research laboratory. PARTICIPANTS 30 individuals with no history of neck or shoulder pain were recruited with no obvious significant postural abnormalities. MAIN OUTCOME MEASURES Ratings of scapular posture were recorded in multiple biomechanical planes under four test conditions (at rest, and while under three isometric conditions) by four physiotherapists. RESULTS The magnitude of discrepancy between the two therapist pairs was 0.04 to 0.76 for Cohen's kappa, and 0.00 to 0.86 for PABAK. In comparison, the generalised kappa provided a score between the two paired kappa coefficients. The difference between mean generalised kappa coefficients and mean Cohen's kappa (0.02) and between mean generalised kappa and PABAK (0.02) were negligible, but the magnitude of difference between the generalised kappa and paired kappa within each plane and condition was substantial; 0.02 to 0.57 for Cohen's kappa and 0.02 to 0.63 for PABAK, respectively. CONCLUSIONS Calculating coefficients for therapist pairs alone may result in inconsistent findings. In contrast, the generalised kappa provided a coefficient close to the mean of the paired kappa coefficients. These findings support an assertion that generalised kappa may lead to a better representation of reliability between three or more raters and that reliability studies only calculating agreement between two raters should be interpreted with caution. However, generalised kappa may mask more extreme cases of agreement (or disagreement) that paired comparisons may reveal.