Standard setting in specific-purpose language testing: what can a qualitative study add?

This paper explores the views of nursing and medical domain experts in considering the standards for a specific-purpose English language screening test, the Occupational English Test (OET), for professional registration for immigrant health professionals. Since individuals who score performances in the test setting are often language experts rather than domain experts, there are possible tensions between what is being measured by a language test and what is deemed important by domain experts. Another concern is a lack of qualitative research on the process of the standard setting. To date, no published qualitative work has been identified about the contributions of domain experts in the standard setting for healthcare communication. In this study, a standard-setting exercise was conducted for the speaking component of the OET, using judgements of nursing and medical clinical educators and supervisors. In all, 13 medical and 18 nursing clinical educators and supervisors rated medical and nursing candidate performances respectively. These performances were audio-recorded OET role-plays that were selected across a range of proficiency levels. Domain experts were invited to comment on the basis of their decisions and the extent of alignment between these decisions and the criteria used to assess performance on the OET. Nursing and medical domain experts showed that they attended to all of the OET criteria in making their decisions about standards. However, clinical scenario simulation also invited judgements of clinical competence from participants, even where they knew that clinical competence should be excluded from their decision-making. Another concern related to the authenticity limitations of the role-play tasks as evidence of readiness to handle communication in the workplace. Overall, findings support the value of qualitative evidence from the standard setting in providing insight into the factors informing and impeding decision-making.

Comparing alternate teaching styles to teach computing skills to girls in their English classes

Low female participation rates in computing are a current concern of the education sector. To address this problem an intervention was developed — computing skills were introduced to girls in their English classes using three different teaching styles: peer tutoring, cross-age tutoring and teacher instruction (control). The sample comprised 136 girls from Years 8 and 10 from a single-sex government school. A pre-test post-test quantitative design was used. To describe the students perspectives, qualitative data were collected from six focus groups conducted with 8–10 students — one from each of the six classes. It was predicted that cross-age tutoring would yield more positive effects than peer tutoring which, in turn, would yield more positive effects than traditional teacher instruction as assessed by achievement on class tasks and attitudes towards computing. The hypothesis was not supported by the quantitative analysis, however in the qualitative data cross-age tutoring was appraised more favourably than peer tutoring or teacher instruction. The latter was the least preferred condition due to: (1) inefficiency; (2) difficulty understanding teachers' explanations; and (3) lack of teacher knowledge. Problems with the implementation of the intervention identified in the focus groups were teacher differences, system failures, missed classes, lack of communication, and selection of computing activities. Practical suggestions were provided relevant to the introduction of cross-age tutoring and the use of computers within secondary level English classes.

The test-retest reliability of the child-initiated pretend play assessment

OBJECTIVE: The purpose of this study is to establish the test–retest reliability of the Child-Initiated Pretend Play Assessment (ChIPPA) (Stagnitti, 2002a; Stagnitti, Unsworth, & Rodger, 2000).

METHOD: The first author rated 38 preschool children ages 4 and 5 years (4 with developmental delay and 34 typically developing) on the ChIPPA. The ChIPPA employs conventional play materials and unstructured play materials to assess three qualities of a child's play ability: elaborateness of play action, ability to substitute objects during play, and the child's need to imitate the modelled actions of the examiner. The ChIPPA was administered twice, at a 2-week interval, to each participant.

RESULTS: Test–retest intraclass correlation coefficients (ICCs) (Type 2,1) calculated for each of the three elaborate play measures ranged from .73 to .84. A test–retest ICC of .56 was obtained for object substitution with unstructured play materials. The test–retest ICC obtained for the combined score for unstructured and conventional play materials was .57. Percentage agreement figures ranging from 63.2% to 84.2% were obtained on test–retest of the object substitution with conventional toys and imitated actions measures. There was no significant difference between test and retest scores for these measures based on a Wilcoxon Matched Pairs Signed-Ranks Test (Wilcoxon Sign Test).

CONCLUSION: Elaborate play scores, object substitution with conventional toys score, and imitation scores on the ChIPPA showed stability over time. Object substitution scores using unstructured materials were the least stable play measures and appeared to be related to the child's play themes. Since play is the primary occupation of children, it is essential that therapists have a reliable measure of play behavior. The test–retest reliability results from the ChIPPA provide evidence that this assessment produces a stable measure of play behavior that can then guide therapists when planning intervention strategies for children.

Validity and reliability of measures of television viewing time and other non-occupational sedentary behaviour of adults : a review

Time spent in non-occupational sedentary behaviours (particularly television viewing time) is associated with excess adiposity and an increased risk of metabolic disorders among adults; however, there are no reviews of the validity and reliability of assessing these behaviours. This paper aims to document measures used to assess adults' time spent in leisure-time sedentary behaviours and to review the evidence on their reliability and validity. Medline, CINAHL and Psych INFO databases and reference lists from published papers were searched to identify studies in which leisure-time sedentary behaviours had been measured in adults. Sixty papers reporting measurement of at least one type of leisure-time sedentary behaviour were identified. Television viewing time was the most commonly measured sedentary behaviour. The main method of data collection was by questionnaire. Nine studies examined reliability and three examined validity for the questionnaire method of data collection. Test–retest reliabilities were predominantly moderate to high, but the validity studies reported large differences in correlations of self-completion questionnaire data with the various referent measures used. To strengthen future epidemiological and health behaviour studies, the development of reliable and valid self-report instruments that cover the full range of leisure-time sedentary behaviour is a priority.

Occupational and non-occupational physical activity and the social determinants of physical activity : results from the Health Survey for England

Background: This article aims to examine the relative contribution of occupational activity to English adults’ meeting of government recommendations for physical activity (PA).

Methods: Data were extracted from a cross-sectional survey of householders in the UK via the Health Survey for England.¹ In total, 14,018 adult participants were included in the analysis. Multivariate logistic regression was used to examine the odds of achieving PA recommendations with and without including occupational activity and to examine the contribution of gender and social and demographic characteristics.

Results: When occupational PA was included, 36% of men and 25% of women were active at the recommended level. Once occupational PA was removed, these proportions were 23% and 19%, respectively. These results were socially patterned, most notably by age and gender.

Conclusions: Occupational PA provides a substantial contribution to those meeting the government target for PA.

Comprehension strategies of above average English as a foreign language (EFL) readers

The study reported in this thesis is a single-shot case study, which aims to provide a detailed description ofthe reading comprehension strategies used by fifteen student teachers ofEnglish from Indonesian- and Javanese-speaking backgrounds in the last year of their four-year Strata-One study at a university In Yogyakarta, Indonesia. These readers were above average among their peers in that their average indexes of grades in Reading and Speaking classes were 3.22 and 3,34 respectively, while the average indexes ofthe peer group were 271 and 2.63, respectively, out ofa scale of 0 to 400. In addition, while students in this university may complete their study by course work or by research, these readers were all enrolled as research students. As studying comprehension strategies involves complex issues, a multi-method approach is required, not only for breadth of coverage, but also to allow for a check on the validity of individual methods. To achieve the goal of the study, thinka1oud tasks, retellings, a reading comprehensIon test, indepth interviews and observations were employed to explore the strategies used. An analysis of the recorded data indicates that these readers used thirty strategies classified under five clusters: infomiation gathering, information processing, text interpretation, comprehension monitoring, and comprehension utilisation. In general, readers started gathering information by silent reading, interpreted the text by an inference or a paraphrase, and ended the task by making selfreflections relevant to the text. Most readers managed to identify problems when they occurred, and monitored their comprehension when they doubted their interpretation, as could be seen from their rereading the text or vocalising its pail(s). When direct interpretation was difficult, readers associated the text with prior knowledge or interrelated parts of the text, The readers in this study share characteristics of both poor and good native readers, in the sense that there was evidence ofgood strategy use butthe readers did not manage to maintain it consistently. As a result, even the successful readers were not able to maximise their potential. The implication is that in order to develop students into independent readers, strategy instruction should be part of and appropriately embedded in, the reading instruction. There is a need not merely to teach strategies as such, but rather to teach flexibility in strategy use. While there was sufficient evidence that thinkaloud tasks and their complementary methods worked to achieve the goals ofthe present study, similar studies with different cohorts are suggested for crosschecks.

Workplace sitting breaks questionnaire (SITBRQ): an assessment of concurrent validity and test-retest reliability

Breaks in prolonged sitting may have beneficial cardiometabolic and musculoskeletal health outcomes. Desk-based work settings are an important environment to promote and support breaks in sitting time. However, few studies have reported the psychometric properties of self-report measures to assess the frequency and duration of breaks from sitting. This study examined the concurrent validity and test-retest reliability of the Workplace Sitting Breaks Questionnaire (SITBRQ) designed to assess frequency and duration of breaks in sitting within desk-based work settings.

Evaluation of the Test-mate ChE (cholinesterase) field kit in acute organophosphorus poisoning

STUDY OBJECTIVE: Measurement of acetylcholinesterase (AChE) is recommended in the management of organophosphorus poisoning, which results in 200,000 deaths worldwide annually. The Test-mate ChE 400 is a portable field kit designed for detecting occupational organophosphorus exposure that measures RBC AChE and plasma cholinesterase (PChE) within 4 minutes. We evaluate Test-mate against a reference laboratory test in patients with acute organophosphorus self-poisoning. METHODS: This was a cross-sectional comparison study of 14 patients with acute organophosphorus poisoning between May 2007 and June 2008. RBC AChE and PChE were measured in 96 and 91 samples, respectively, with the Test-mate ChE field kit and compared with a reference laboratory, using the limits of agreement method (Bland and Altman), κ statistics, and Spearman's correlation coefficients. RESULTS: There was good agreement between the Test-mate ChE and the reference laboratory for RBC AChE. The mean difference (Test-mate-reference) was -0.62 U/g hemoglobin, 95% limits of agreement -10.84 to 9.59 U/g hemoglobin. Good agreement was also observed between the categories of mild, moderate, and severe RBC AChE inhibition (weighted κ 0.85; 95% confidence interval [CI] 0.83 to 0.87). Measurement of PChE also showed good agreement, with a mean difference (Test-mate-reference) of +0.06 U/mL blood, 95% limits of agreement -0.41 to 0.53 U/mL blood. Spearman's correlation coefficients were 0.87 (95% CI 0.81 to 0.91) for RBC AChE and 0.76 (95% CI 0.66 to 0.84) for PChE. Analysis for within-subject correlation of subjects did not change the limits of agreement. CONCLUSION: The Test-mate ChE field kit reliably provides rapid measurement of RBC AChE in acute organophosphorus poisoning.

Itinerant foreign harvest workers in Australia: the impact of precarious employment on occupational safety and health

Horticulture work in many high-income economies is increasingly performed by temporary migrant workers from low-wage economies. In Australia, such work is now performed predominantly by international backpackers – young well-educated workers with mostly sound English language skills. These workers are drawn to harvesting work by a government scheme which provides an incentive for completing a specified number of days work in horticulture. This article examines the health and safety experience of these workers, through focus groups, interviews and an online survey. Notwithstanding their distinctive backgrounds, the harvesting experience of these temporary migrant workers is similar to that of low-skilled migrants working in other high-income countries. Health and safety risks associated with work organisation and payment systems, and a lack of compliance with OHS legal requirements, are commonplace but potentially compounded by a sense of invincibility amongst these young travellers. Furthermore, a growing pool of undocumented workers is placing downward pressures on their employment conditions. The vulnerability associated with work and earnings uncertainty, and the harsh environment in which harvesting work occurs, remains a constant notwithstanding the background of these workers.

Measurement invariance of English and French Health Education Impact Questionnaire (heiQ) empowerment scales validated for cancer

PURPOSE: If measurement invariance (MI) is demonstrated for a scale completed by respondents from two different language groups, it means that the scale measures the same construct in the same way in both groups. We assessed MI of the French- and English-language versions of the five Health Education Impact Questionnaire (heiQ) empowerment scales validated for the cancer setting. METHODS: Data came from two cross-sectional studies of Canadian cancer survivors (704 English, 520 French). Single-group confirmatory factor analysis (CFA) was used to test whether the hypothesized factor structure of the French-language heiQ empowerment scales fit the data. Multi-group CFAs were conducted to assess different levels of MI conditions (configural, metric, scalar, strict, as well as MI of factor variances, covariances, and latent means) of the French- and English-language heiQ empowerment scales. RESULTS: The correlated five-factor model showed good fit in both language groups (goodness-of-fit indices: CFI ≥ .97; RMSEA ≤ .07). Goodness-of-fit indices and tests of differences in fit between models supported MI of the five-factor model across the two language groups (∆CFI ≤ -.010 combined with ∆RMSEA ≤ .015). CONCLUSIONS: The French- and English-language heiQ empowerment scales measure the same five dimensions of empowerment in the same way across both language groups. Thus, any observed similarities or differences between French- and English-speaking respondents completing these scales are valid and reflect similarities or differences in empowerment across language groups, not measurement artifact. Consequently, heiQ empowerment data from English- and French-speaking respondents can be directly pooled or contrasted in data analyses.