Standard setting in specific-purpose language testing: what can a qualitative study add?

This paper explores the views of nursing and medical domain experts in considering the standards for a specific-purpose English language screening test, the Occupational English Test (OET), for professional registration for immigrant health professionals. Since individuals who score performances in the test setting are often language experts rather than domain experts, there are possible tensions between what is being measured by a language test and what is deemed important by domain experts. Another concern is a lack of qualitative research on the process of the standard setting. To date, no published qualitative work has been identified about the contributions of domain experts in the standard setting for healthcare communication. In this study, a standard-setting exercise was conducted for the speaking component of the OET, using judgements of nursing and medical clinical educators and supervisors. In all, 13 medical and 18 nursing clinical educators and supervisors rated medical and nursing candidate performances respectively. These performances were audio-recorded OET role-plays that were selected across a range of proficiency levels. Domain experts were invited to comment on the basis of their decisions and the extent of alignment between these decisions and the criteria used to assess performance on the OET. Nursing and medical domain experts showed that they attended to all of the OET criteria in making their decisions about standards. However, clinical scenario simulation also invited judgements of clinical competence from participants, even where they knew that clinical competence should be excluded from their decision-making. Another concern related to the authenticity limitations of the role-play tasks as evidence of readiness to handle communication in the workplace. Overall, findings support the value of qualitative evidence from the standard setting in providing insight into the factors informing and impeding decision-making.

A domain-specific visual modeling language for testing environment emulation

Software integration testing plays an increasingly important role as the software industry has experienced a major change from isolated applications to highly distributed computing environments. Conducting integration testing is a challenging task because it is often very difficult to replicate a real enterprise environment. Emulating testing environment is one of the key solutions to this problem. However, existing specification-based emulation techniques require manual coding of their message processing engines, therefore incurring high development cost. In this paper, we present a suite of domain-specific visual modelinglanguages to describe emulated testing environments at a highabstraction level. Our solution allows domain experts to model atesting environment from abstract interface layers. These layermodels are then transformed to runtime environment for application testing. Our user study shows that our visual languages are easy to use, yet with sufficient expressive power to model complex testing applications.

Cultural adaptation and validation of the Health Literacy Questionnaire (HLQ): robust nine-dimension Danish language confirmatory factor model

Health literacy is an important construct in population health and healthcare requiring rigorous measurement. The Health Literacy Questionnaire (HLQ), with nine scales, measures a broad perception of health literacy. This study aimed to adapt the HLQ to the Danish setting, and to examine the factor structure, homogeneity, reliability and discriminant validity. The HLQ was adapted using forward-backward translation, consensus conference and cognitive interviews (n = 15). Psychometric properties were examined based on data collected by face-to-face interview (n = 481). Tests included difficulty level, composite scale reliability and confirmatory factor analysis (CFA). Cognitive testing revealed that only minor re-wording was required. The easiest scale to respond to positively was 'Social support for health', and the hardest were 'Navigating the healthcare system' and 'Appraisal of health information'. CFA of the individual scales showed acceptably high loadings (range 0.49-0.93). CFA fit statistics after including correlated residuals were good for seven scales, acceptable for one. Composite reliability and Cronbach's α were >0.8 for all but one scale. A nine-factor CFA model was fitted to items with no cross-loadings or correlated residuals allowed. Given this restricted model, the fit was satisfactory. The HLQ appears robust for its intended application of assessing health literacy in a range of settings. Further work is required to demonstrate sensitivity to measure changes.

TeeVML : tool support for semi-automatic integration testing environment emulation

Software environment emulation provides a means for simulating an operational environment of a system. This process involves approximation of systems’ external behaviors and their communications with a system to be tested in the environment. Development of such an environment is a tedious task and involves complex low level coding. Model driven engineering is an avenue to raise the level of abstraction beyond programming by specifying solution directly using problem domain concepts. In this paper we propose a novel domain-specific modeling tool to generate complex testing environments. Our tool employs a suite of domain-specific visual modeling languages for modelling emulation environment at a high level of abstraction. These high level specifications are then automatically transformed to runtimeenvironment for application integration testing, boosting development productivity and ease of use.