Magnitude and meaningfulness of change in SF-36 scores in four types of orthopedic surgery

Background: The Medical Outcomes General Health Survey (SF-36) is a widely used health status measure; however, limited evidence is available for its performance in orthopedic settings. The aim of this study was to examine the magnitude and meaningfulness of change and sensitivity of SF-36 subscales following orthopedic surgery.

Methods: Longitudinal data on outcomes of total hip replacement (THR, n = 255), total knee replacement (TKR, n = 103), arthroscopic partial meniscectomy (APM, n = 74) and anterior cruciate ligament reconstruction (ACL, n = 62) were used to estimate the effect sizes (ES, magnitude of change) and minimal detectable change (sensitivity) at the group and individual level. To provide context for interpreting the magnitude of changes in SF-36 scores, we also compared patients' scores with age and sex-matched population norms. The studies were conducted in Sweden. Follow-up was five years in THR and TKR studies, two years in ACL, and three months in APM.

Results: On average, large effect sizes (ES≥0.80) were found after orthopedic surgery in SF-36 subscales measuring physical aspects (physical functioning, role physical, and bodily pain). Small (0.20–0.49) to moderate (0.50–0.79) effect sizes were found in subscales measuring mental and social aspects (role emotional, vitality, social functioning, and mental health). General health scores remained relatively unchanged during the follow-up. Despite improvements, post-surgery mean scores of patients were still below the age and sex matched population norms on physical subscales. Patients' scores on mental and social subscales approached population norms following the surgery. At the individual level, scores of a large proportion of patients were affected by floor or ceiling effects on several subscales and the sensitivity to individual change was very low.

Conclusion: Large to moderate meaningful changes in group scores were observed in all SF-36 subscales except General Health across the intervention groups. Therefore, in orthopedic settings, the SF-36 can be used to show changes for groups in physical, mental, and social dimensions and in comparison with population norms. However, SF-36 subscales have low sensitivity to individual change and so we caution against using SF-36 to monitor the health status of individual patients undergoing orthopedic surgery.

Tests of measurement invariance failed to support the application of the "then-test"

Objective
The use of then-test (retrospective pre-test) scores has frequently been proposed as a solution to potential confounding of change scores because of response shift, as it is assumed that then-test and post-test responses are provided from the same perspective. However, this assumption has not been formally tested using robust quantitative methods. The aim of this study was to compare the psychometric performance of then-test/post-test with traditional pre-test/post-test data and assessing whether the resulting data structures support the application of the then-test for evaluations of chronic disease self-management interventions.

Study Design and Setting
Pre-test, post-test, and then-test data were collected from 314 participants of self-management courses using the Health Education Impact Questionnaire (heiQ). The derived change scores (pre-test/post-test; then-test/post-test) were examined for their psychometric performance using tests of measurement invariance.

Results
Few questionnaire items were noninvariant across pre-test/post-test, with four items identified and requiring removal to enable an unbiased comparison of factor means. In contrast, 12 items were identified and required removal in then-test/post-test data to avoid biased change score estimates.

Conclusion
Traditional pre-test/post-test data appear to be robust with little indication of response shift. In contrast, the weaker psychometric performance of then-test/post-test data suggests psychometric flaws that may be the result of implicit theory of change, social desirability, and recall bias.

Separating the contributions of hearing, lexical knowledge and speech production to speech perception scores in children with hearing impairments.

Open-set word and sentence speech-perception test scores are commonly used as a measure of hearing abilities in children and adults using cochlear implants and/or hearing aids. These tests are usually presented auditorily with a verbal response. In the case of children, scores are typically lower and more variable than for adults with hearing impairments using similar devices. It is difficult to interpret children's speech-perception scores without considering the effects of lexical knowledge and speech-production abilities on their responses. This study postulated a simple mathematical model to describe the effects of hearing, lexical knowledge, and speech production on the perception test scores for monosyllabic words by children with impaired hearing. Thirty-three primary-school children with impaired hearing, fitted with hearing aids and/or cochlear implants, were evaluated using speech-perception, reading-aloud, speech-production, and language measures. These various measures were incorporated in the mathematical model, which revealed that performance in an open-set word-perception test in the auditory-alone mode is strongly dependent on residual hearing levels, lexical knowledge, and speech-production abilities. Further applications of the model provided an estimate of the effect of each component on the overall speech-perception score for each child.

The inclusion of ‘then-test’ questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation

Objectives Program evaluations are frequently based on ‘then-test’ data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects’ responses to post-test.
Patients and methods To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
Results Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then test questions reporting significantly higher post-test scores than respondents of the other groups.
Conclusions This study provides evidence that the inclusion of then-test questions alters post-test responses,
suggesting that change scores based on then-test data be interpreted with care.

Accountability and the affective labour of teachers: a Marxist-Vygotskian perspective

This article addresses the issue of affective labour in education in the context of standards-based reforms and accountability. In particular, it focuses on neoliberal strategies of rationalization and control that produce a number of social pathologies, such as alienated teaching and learning and reified social relations between teachers and students. The article turns to affective labour as something that enables teachers to counteract these effects. This argument arises from the analysis of interviews with teachers who continue to generate and sustain the sociality of teaching and learning. Affect directs teachers’ commitment to practice that is governed by feeling, passion and the ethics of care. What gives affective labour such an important position is that it is both outside and beyond accountability and performativity measures. It is identified with the general pedagogical activity that cannot be structured by measuring devices such as students’ test scores or standards. The article concludes with the application of Vygotsky’s ideas about the role of affect in education and argues that affective labour has an expansive power of ontological freedom that cannot be controlled. of ontological freedom that cannot be controlled.

Psychometric data on the mathematics self-efficacy scale

Descriptive measurement reliability and validity data are reported on scores from the Mathematics Self-Efficacy Scale (MSES) in two contexts—class and test—for a sample of 302 Australian high school students. Summated scores on the MSES correlated r = .74, and together these items yielded one component that explained 49% of the variance. MSES scores demonstrated internal reliability for both class and test (Cronbach alphas = .86 and .90). Statistically significant correlations between MSES scores, past mathematics grades, and Marsh’s Self-Description Questionnaire III (Math) were used to indicate the convergent validity of scores for the MSES. Discriminant validity of the MSES scores was suggested by the lack of significant correlations between the MSES measures and students’ desired English grades. Statistically significant correlations between MSES scores and students’ desired mathematics grades indicated concurrent validity of the MSES measures. A statistically significant difference between class and test scores supported Bandura’s prescription for context specificity in efficacy research.

The effect of private school competition on public school performance in Georgia

This article presents estimates of the effect of private school competition on public school performance. Using data on school districts in Georgia, the authors estimate models relating tenth- and third-grade test scores for either reading or mathematics to the level of private school competition. Test scores are not measurably or significantly higher in areas with greater private school competition, a result robust through multiple estimations using three measures of private school competition and a variety of control variables. The authors address the possible endogeneity between test scores and private school competition using instrumental variables estimators, with percentage of the population that is Catholic, county population in 1980, lagged competition, and various other measures as alternative instruments.

Measuring moral judgement and the implications of cooperative education and rule-based learning

The Defining Issues Test (DIT), developed by Rest (1986), measures a person's level of moral development using hypothetical social dilemmas. Although the DIT is useful for measuring moral development in social settings, it might not adequately capture an individual's moral judgement abilities in solving work-related problems (Weber, 1990; Trevino, 1992; Welton et al., 1994). In the present study, the moral judgement levels of 97 accounting students were measured over a 1 year period using two separate test instruments, the DIT and a context-specific instrument developed by Welton et al. (1994). The test scores are significantly higher on the DIT than the Welton instrument (between the instruments and over time), suggesting that accounting students use higher levels of moral reasoning in resolving hypothetical social dilemmas and lower levels of moral reasoning in resolving context-specific dilemmas. The difference in test scores was highest during cooperative education (work placement programme), implying that the environment is a significant determinant on students' test scores.

The moral reasoning abilities of Australian and Malaysian accounting students : a comparative analysis

If national culture is a significant determinant of ethical attitudes, it is not unreasonable to expect ethical decision-making to be influenced by one's culture. However, problems arise when the notion of right differs from one culture to another. The question addressed in this paper is whether the moral reasoning abilities of Australian and Malaysian accounting students in their final year of study differ because of their cultural upbringing. This study uses primary data collected from 34 final year accounting students (12 Australian and 22 Malaysian) enrolled in an Australian degree program. The test scores collected at the beginning and end of the academic year indicate that culture and other explanatory variables do not have an affect on students' moral judgment. The findings in this study suggest that culture as an independent variable does not influence the way accounting students analyse and resolve ethical dilemmas.

Raising 'standards' in schools : considerations in the first stage of an improvement intervention

This article discusses the initial phase of a government-funded intervention to raise school achievement in a cluster of schools with below average standardised test results. Five schools - four primary schools and one secondary school - in the far outer reaches of Melbourne achieved lower standardised test scores in literacy and numeracy than the state average. Two departmental strategies were in place before my involvement the appointment of a coaching teacher in each school, and funding for a four day intensive residential professional learning workshop for these coaching teachers, to be held at the beginning of the 2009 school year. lt was at this stage that I became involved in what was planned to be an ongoing research program to record, monitor and evaluate the success of the intervention.

The effect of the work environment on accounting students' moral reasoning and development

Puxty et al. (1994) claim that professional accountants are induced to act ethically through two aspects of their socialisation, the education process, and the influence of work experience and role models who show what it means to be ethical. The education of accountants is not simply a matter of becoming technically competent, it is also a process of internalising accepted norms of professional conduct. Student accountants learn acceptable behaviour by learning the principles of good conduct in their education, and receiving advice and observing what significant others do in the workplace. The purpose of this paper is to explore the effect of the work environment on accounting students moral reasoning and development by comparing the DIT P-scores of accounting students pre and post cooperative education. Cooperative eduction is an industry placement program where students are required to work in commerce and industry for one year. Findings indicate that DIT P-scores decrease during cooperative education suggesting that accounting students, whilst in the work environment, do not reason according to their capability as measured by their pre-test scores.

Convergent validity of two visual motor integration tests

Background: Occupational therapists often assess visual motor integration (VMI) skills. It is, therefore, imperative that therapists use VMI tests with robust measurement properties.

Objective: This study examined the convergent validity of two VMI tests used to assess children, adolescents and adults.

Method: Three groups of healthy participants (n = 153) completed the Beery-Buktenica Developmental Test of Visual-Motor Integration (DTVMI) and the Full Range Test of Visual Motor Integration (FRTVMI). Seventy-three children aged 5-10 years (37 males and 36 females; mean age 7.5 years, SD = 2.20), 19 adolescents aged 11-17 years (8 males and 11 females; 13.1 years, SD = 2.16), and 61 adults (18 males and 43 females; mean age 31.82 years, SD = 11.20) completed the DTVMI and the FRTVMI. Spearman rho correlation coefficients were used to investigate whether each pair of the VMI test scores for each of the three participant age groups were associated.

Results: The Spearman rho correlation coefficients between all three versions of the DTVMI and FRTVMI were statistically significant. For the child group, the correlation coefficient was rho = 0.70 (p<0.000), while the correlation between the VMI scores obtained by the adolescent group on the two tests was rho = 0.77 (p<0.000). For the adult participant group, the correlation coefficient between the DTVMI and the FRTVMI was rho = 0.70 (p<0.000).

Conclusion: The VMI scores obtained by the three participant age groups on the DTVMI and the FRTVMI were all significantly correlated with each other. Overall, the DTVMI and the FRTVMI exhibited large levels of convergent validity with each other, indicating that the two tests appear to measure similar visual-motor integration constructs.

A weighted version of the Melbourne low-vision ADL index : a measure of disability impact

Objective. To develop a version of the Melbourne Low-Vision ADL Index that measures the personal impact of disability in activities of daily living (ADL's). Also, to determine the relationship between clinical measures of vision impairment and disability impact.

Methods. The Melbourne Low-Vision ADL Index (MLVAI) is a desk-based clinical assessment of disability in ADL's. Ability to perform each item is rated on a five-level descriptive scale from zero to four. In this study, the original version of the MLVAI was modified to measure disability impact. The simple modification involved weighting each item by the importance of that item to the person being tested. Importance was also rated on a five-level scale from zero to four. The validity and reliability of the Weighted Melbourne Low-Vision ADL Index (MLVAIW) was determined for 97 vision-impaired subjects in a cross-sectional study.

Results. Cronbach's alpha coefficient indicated an internal reliability of 0.94, and an intraclass correlation coefficient indicated an overall reliability of 0.88. The standard error of measurement was 24.7 points (out of a possible score of 400). There was a statistically significant difference in test scores between normal subjects and vision-impaired subjects. All vision measures had a high, statistically significant correlation with MLVAIW score. Near-word acuity had the strongest correlation (rs = 0.78, p < 0.001), followed by Melbourne Edge Test contrast sensitivity (rs = -0.72, p < 0.001). Visual field had the weakest correlation (rs = -0.52, p < 0.001). The best predictive model of MLVAIW score incorporated the variables age, near-word acuity, and visual field. Together, these variables accounted for 65.1% of the variance in MLVAIW score.

Conclusions. The MLVAI is highly valid and reliable when weighted by a scale that reflects the personal importance of ADL's. The MLVAIW can provide information over and above that obtained with the usual clinical vision measures and may be used to assess low-vision patients and to measure low-vision rehabilitation outcomes. It is suggested that the assessment of disability using the original MLVAI and the assessment of the impact of disability using the MLVAIW should be kept separate to facilitate the clear interpretation of the outcomes of low-vision rehabilitation.

Two tests of approach-iron golf skill and their ability to predict tournament performance

The aim of this study was to investigate the convergent and predictive validity of two skill tests that examine the ability of golfers to hit accurate approach-iron shots. Twenty-four high-level golfers (handicap = 2.6 ± 1.7) performed the Nine-Ball Skills Test (assesses the ability to shape/control ball trajectory with high accuracy) and the Approach-Iron Skill Test (assesses the ability to hit straight shots from varying distances with high accuracy). Participants then completed at least eight rounds of tournament golf over the following 90 days and reported an indicator of approach-iron accuracy (per cent error index). A moderate correlation (r = 0.50, P < 0.05) was noted between scores for both tests. Generalised estimating equations, using two covariates (lie of the ball and distance to hole), were used to determine model fit and the amount of variance explained for tournament per cent error index. Results showed that the Approach-Iron Skill Test was the slightly stronger predictor of on-course per cent error index. With both test scores considered together, a minimal amount of additional variance was explained. These findings suggest that either of the tests used individually or combined may be used to predict tournament approach iron performance in high-level golfers.

Authentically assessing graduate teaching: outside and beyond neo-liberal constructs

In this paper, we challenge the current focus on ‘best practice’, graduate teacher tests, and student test scores as the panacea for ensuring teaching quality and argue for ways of thinking about evidence of quality beginning teaching outside and beyond the current neoliberal accountability discourses circulating in Australia and other countries. We suggest that teacher educators need to reinsert themselves as key players in the debates around quality beginning teaching, rather than being viewed as a source of the problem. To enable teacher educators to assume accountability for quality beginning teachers, we propose the framework of a capstone teacher performance assessment—a structured portfolio called the Authentic Teacher Assessment (ATA)— and examine examples of these assessments through the lens of critical discourse analysis. As a measure of ‘readiness to teach’, the ATA is compared with supervising teachers’ assessments of preservice teachers. We argue that structured portfolios that include artefacts derived from preservice teachers’ practice in classrooms along with graduate teacher self assessments provide a stronger accountability measure of effec- tive beginning teaching and demonstrably address the current anxiety regarding ‘evi- dence’. We suggest that such an approach should be reliable enough to be ‘read’ by external assessors (and moderated across other teacher education institutions). Rigor- ous research on a national basis is called for in order to develop and implement a structured portfolio as rich evidence of graduates’ quality and readiness to teach.