Proving My Competency One Test at a Time: Internationally Educated Nurses and the Canadian English Language Benchmark Assessment for Nurses

Abstract Professional language assessment is a new concept that has great potential to benefit Internationally Educated Professionals and the communities they serve. This thesis reports on a qualitative study that examined the responses of 16 Canadian English Language Benchmark Assessment for Nurses (CELBAN) test-takers on the topic of their perceptions of the CELBAN test-taking experience in Ontario in the winter of 2015. An Ontario organization involved in registering participants distributed an e-mail through their listserv. Thematic analyses of focus group and interview transcripts identified 7 themes from the data. These themes were used to inform conclusions to the following questions: (1) How do IENs characterize their assessment experience? (2) How do IENs describe the testing constructs measured by the CELBAN? (3) What, if any, potential sources of construct irrelevant variance (CIV) do the test-takers describe based on their assessment experience? (4) Do IENs feel that the CELBAN tasks provide a good reflection of the types of communicative tasks required of a nurse? Overall, participants reported positive experiences with the CELBAN as an assessment of their language skills, and noted some instances in which they felt some factors external to the assessment impacted their demonstration of their knowledge and skill. Lastly, some test-takers noted the challenge of completing the CELBAN where the types of communicative nursing tasks included in the assessment differed from nursing tasks typical of an IENs country or origin. The findings are discussed in relation to literature on high-stakes large-scale assessment and IEPs, and a set of recommendations are offered to future CELBAN administration. These recommendations include (1) the provision of a webpage listing all licensure requirements (2) monitoring of CELBAN location and dates in relation to the wider certification timeline for applicants (3) The provision of additional CELBAN preparatory materials (4) Minor changes to the CELBAN administrative protocols. Given that the CELBAN is a relatively new assessment format and its widespread use for high-stakes decisions (a component of nursing certification and licensure), research validating IEN-test-taker responses to construct representation and construct irrelevant variance is critical to our understanding of the role of competency testing for IENs.

UNDERSTANDING ESSAY RATING AS A SOCIALLY MEDIATED ACTIVITY: THE CASE OF A HIGH-STAKES ENGLISH TEST IN CHINA

Most essay rating research in language assessment has examined human raters’ essay rating as a cognitive process, thus overlooking or oversimplifying the interaction between raters and sociocultural contexts. Given that raters are social beings, their practices have social meanings and consequences. Hence it is important to situate essay rating within its sociocultural context for a more meaningful understanding. Drawing on Engeström’s (1987, 2001) cultural-historical activity theory (CHAT) framework with a sociocultural perspective, this study reconceptualized essay rating as a socially mediated activity with both cognitive (individual raters’ goal-directed decision-making actions) and social layers (raters’ collective object-oriented essay rating activity at related settings). In particular, this study explored raters’ essay rating at one provincial rating centre in China within the context of a high-stakes university entrance examination, the National Matriculation English Test (NMET). This study adopted a multiple-method multiple-perspective qualitative case study design. Think-aloud protocols, stimulated recalls, interviews, and documents served as the data sources. This investigation involved 25 participants at two settings (rating centre and high schools), including rating centre directors, team leaders, NMET essay raters who were high school teachers, and school principals and teaching colleagues of these essay raters. Data were analyzed using Strauss and Corbin’s (1990) open and axial coding techniques, and CHAT for data integration. The findings revealed the interaction between raters and the NMET sociocultural context. Such interaction can be understood through a surface structure (cognitive layer) and a deep structure (social layer) concerning how raters assessed NMET essays, where the surface structure reflected the “what” and the deep structure explained the “how” and “why” in raters’ decision-making. This study highlighted the roles of goals and rules in rater decision-making, rating tensions and raters’ solutions, and the relationship between essay rating and teaching. This study highlights the value of a sociocultural view to essay rating research, demonstrates CHAT as a sociocultural approach to investigate essay rating, and proposes a direction for future washback research on the effect of essay rating. This study also provides support for NMET rating practices that can potentially bring positive washback to English teaching in Chinese high schools.

An Objective Framework to Test the Quality of Candidate Indicators of Good Environmental Status

Large efforts are on-going within the EU to prepare the Marine Strategy Framework Directive’s (MSFD) assessment of the environmental status of the European seas. This assessment will only be as good as the indicators chosen to monitor the eleven descriptors of good environmental status (GEnS). An objective and transparent framework to determine whether chosen indicators actually support the aims of this policy is, however, not yet in place. Such frameworks are needed to ensure that the limited resources available to this assessment optimize the likelihood of achieving GEnS within collaborating states. Here, we developed a hypothesis-based protocol to evaluate whether candidate indicators meet quality criteria explicit to the MSFD, which the assessment community aspires to. Eight quality criteria are distilled from existing initiatives, and a testing and scoring protocol for each of them is presented. We exemplify its application in three worked examples, covering indicators for three GEnS descriptors (1, 5 and 6), various habitat components (seaweeds, seagrasses, benthic macrofauna and plankton), and assessment regions (Danish, Lithuanian and UK waters). We argue that this framework provides a necessary, transparent and standardized structure to support the comparison of candidate indicators, and the decision-making process leading to indicator selection. Its application could help identify potential limitations in currently available candidate metrics and, in such cases, help focus the development of more adequate indicators. Use of such standardized approaches will facilitate the sharing of knowledge gained across the MSFD parties despite context-specificity across assessment regions, and support the evidence-based management of European seas.

An Objective Framework to Test the Quality of Candidate Indicators of Good Environmental Status

Large efforts are on-going within the EU to prepare the Marine Strategy Framework Directive’s (MSFD) assessment of the environmental status of the European seas. This assessment will only be as good as the indicators chosen to monitor the eleven descriptors of good environmental status (GEnS). An objective and transparent framework to determine whether chosen indicators actually support the aims of this policy is, however, not yet in place. Such frameworks are needed to ensure that the limited resources available to this assessment optimize the likelihood of achieving GEnS within collaborating states. Here, we developed a hypothesis-based protocol to evaluate whether candidate indicators meet quality criteria explicit to the MSFD, which the assessment community aspires to. Eight quality criteria are distilled from existing initiatives, and a testing and scoring protocol for each of them is presented. We exemplify its application in three worked examples, covering indicators for three GEnS descriptors (1, 5 and 6), various habitat components (seaweeds, seagrasses, benthic macrofauna and plankton), and assessment regions (Danish, Lithuanian and UK waters). We argue that this framework provides a necessary, transparent and standardized structure to support the comparison of candidate indicators, and the decision-making process leading to indicator selection. Its application could help identify potential limitations in currently available candidate metrics and, in such cases, help focus the development of more adequate indicators. Use of such standardized approaches will facilitate the sharing of knowledge gained across the MSFD parties despite context-specificity across assessment regions, and support the evidence-based management of European seas.

Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements

The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.

Sepsis: the LightCycler SeptiFast Test MGRADE®, SepsiTest™ and IRIDICA BAC BSI assay for rapidly identifying bloodstream bacteria and fungi – a systematic review and economic evaluation

Background: Sepsis can lead to multiple organ failure and death. Timely and appropriate treatment can reduce in-hospital mortality and morbidity. Objectives: To determine the clinical effectiveness and cost-effectiveness of three tests [LightCycler SeptiFast Test MGRADE® (Roche Diagnostics, Risch-Rotkreuz, Switzerland); SepsiTest™ (Molzym Molecular Diagnostics, Bremen, Germany); and the IRIDICA BAC BSI assay (Abbott Diagnostics, Lake Forest, IL, USA)] for the rapid identification of bloodstream bacteria and fungi in patients with suspected sepsis compared with standard practice (blood culture with or without matrix-absorbed laser desorption/ionisation time-offlight mass spectrometry). Data sources: Thirteen electronic databases (including MEDLINE, EMBASE and The Cochrane Library) were searched from January 2006 to May 2015 and supplemented by hand-searching relevant articles. Review methods: A systematic review and meta-analysis of effectiveness studies were conducted. A review of published economic analyses was undertaken and a de novo health economic model was constructed. A decision tree was used to estimate the costs and quality-adjusted life-years (QALYs) associated with each test; all other parameters were estimated from published sources. The model was populated with evidence from the systematic review or individual studies, if this was considered more appropriate (base case 1). In a secondary analysis, estimates (based on experience and opinion) from seven clinicians regarding the benefits of earlier test results were sought (base case 2). A NHS and Personal Social Services perspective was taken, and costs and benefits were discounted at 3.5% per annum. Scenario analyses were used to assess uncertainty. Results: For the review of diagnostic test accuracy, 62 studies of varying methodological quality were included. A meta-analysis of 54 studies comparing SeptiFast with blood culture found that SeptiFast had an estimated summary specificity of 0.86 [95% credible interval (CrI) 0.84 to 0.89] and sensitivity of 0.65 (95% CrI 0.60 to 0.71). Four studies comparing SepsiTest with blood culture found that SepsiTest had an estimated summary specificity of 0.86 (95% CrI 0.78 to 0.92) and sensitivity of 0.48 (95% CrI 0.21 to 0.74), and four studies comparing IRIDICA with blood culture found that IRIDICA had an estimated summary specificity of 0.84 (95% CrI 0.71 to 0.92) and sensitivity of 0.81 (95% CrI 0.69 to 0.90). Owing to the deficiencies in study quality for all interventions, diagnostic accuracy data should be treated with caution. No randomised clinical trial evidence was identified that indicated that any of the tests significantly improved key patient outcomes, such as mortality or duration in an intensive care unit or hospital. Base case 1 estimated that none of the three tests provided a benefit to patients compared with standard practice and thus all tests were dominated. In contrast, in base case 2 it was estimated that all cost per QALY-gained values were below £20,000; the IRIDICA BAC BSI assay had the highest estimated incremental net benefit, but results from base case 2 should be treated with caution as these are not evidence based. Limitations: Robust data to accurately assess the clinical effectiveness and cost-effectiveness of the interventions are currently unavailable. Conclusions: The clinical effectiveness and cost-effectiveness of the interventions cannot be reliably determined with the current evidence base. Appropriate studies, which allow information from the tests to be implemented in clinical practice, are required.

Response process and test-retest reliability of the Context Assessment for Community Health tool in Vietnam

BACKGROUND: The recently developed Context Assessment for Community Health (COACH) tool aims to measure aspects of the local healthcare context perceived to influence knowledge translation in low- and middle-income countries. The tool measures eight dimensions (organizational resources, community engagement, monitoring services for action, sources of knowledge, commitment to work, work culture, leadership, and informal payment) through 49 items. OBJECTIVE: The study aimed to explore the understanding and stability of the COACH tool among health providers in Vietnam. DESIGNS: To investigate the response process, think-aloud interviews were undertaken with five community health workers, six nurses and midwives, and five physicians. Identified problems were classified according to Conrad and Blair's taxonomy and grouped according to an estimation of the magnitude of the problem's effect on the response data. Further, the stability of the tool was examined using a test-retest survey among 77 respondents. The reliability was analyzed for items (intraclass correlation coefficient (ICC) and percent agreement) and dimensions (ICC and Bland-Altman plots). RESULTS: In general, the think-aloud interviews revealed that the COACH tool was perceived as clear, well organized, and easy to answer. Most items were understood as intended. However, seven prominent problems in the items were identified and the content of three dimensions was perceived to be of a sensitive nature. In the test-retest survey, two-thirds of the items and seven of eight dimensions were found to have an ICC agreement ranging from moderate to substantial (0.5-0.7), demonstrating that the instrument has an acceptable level of stability. CONCLUSIONS: This study provides evidence that the Vietnamese translation of the COACH tool is generally perceived to be clear and easy to understand and has acceptable stability. There is, however, a need to rephrase and add generic examples to clarify some items and to further review items with low ICC.

Graph analysis of verbal fluency test discriminate between patients with Alzheimer’s disease, Mild Cognitive Impairment and normal elderly controls

Verbal fluency is the ability to produce a satisfying sequence of spoken words during a given time interval. The core of verbal fluency lies in the capacity to manage the executive aspects of language. The standard scores of the semantic verbal fluency test are broadly used in the neuropsychological assessment of the elderly, and different analytical methods are likely to extract even more information from the data generated in this test. Graph theory, a mathematical approach to analyze relations between items, represents a promising tool to understand a variety of neuropsychological states. This study reports a graph analysis of data generated by the semantic verbal fluency test by cognitively healthy elderly (NC), patients with Mild Cognitive Impairment – subtypes amnestic(aMCI) and amnestic multiple domain (a+mdMCI) - and patients with Alzheimer’s disease (AD). Sequences of words were represented as a speech graph in which every word corresponded to a node and temporal links between words were represented by directed edges. To characterize the structure of the data we calculated 13 speech graph attributes (SGAs). The individuals were compared when divided in three (NC – MCI – AD) and four (NC – aMCI – a+mdMCI – AD) groups. When the three groups were compared, significant differences were found in the standard measure of correct words produced, and three SGA: diameter, average shortest path, and network density. SGA sorted the elderly groups with good specificity and sensitivity. When the four groups were compared, the groups differed significantly in network density, except between the two MCI subtypes and NC and aMCI. The diameter of the network and the average shortest path were significantly different between the NC and AD, and between aMCI and AD. SGA sorted the elderly in their groups with good specificity and sensitivity, performing better than the standard score of the task. These findings provide support for a new methodological frame to assess the strength of semantic memory through the verbal fluency task, with potential to amplify the predictive power of this test. Graph analysis is likely to become clinically relevant in neurology and psychiatry, and may be particularly useful for the differential diagnosis of the elderly.

Valutazione del fondo naturale di argon-37 a fini di supporto alle ispezioni in situ nell'ambito del Trattato Internazionale per il Bando dei Test Nucleari

Il CTBT (Trattato di Bando Complessivo dei Test Nucleari) è un accordo multilaterale tra Stati, stilato nel 1996, che impone il divieto assoluto di effettuare esplosioni di tipo nucleare, sia per scopi civili che per scopi militari, in qualsiasi ambiente. Esso prevede l'istituzione di un sistema di verifica e monitoraggio che si prefigge lo scopo di rilevare tempestivamente se in una qualsiasi parte del mondo sia stato clandestinamente effettuato un test. Tra le varie tecniche utilizzate, la più decisiva per l'identificazione di un'esplosione è data dal monitoraggio di radionuclidi, che permette di fare ipotesi su di un eventuale evento illecito di origine nucleare se in una certa regione si rileva la presenza nell'ambiente di particolari isotopi radioattivi in quantità anomale. Attualmente, il metodo più efficace consiste nell'eventuale identificazione della presenza di quattro radioisotopi di gas nobili, ovvero xeno-131m, xeno-133, xeno-133m e xeno-135. Di recente, però, gli esperti hanno cominciato a valutare l'ipotesi di effettuare misurazioni anche di un altro radioisotopo di gas nobili, ovvero l'argon-37. L'efficacia nell'utilizzo di quest'ultimo è superiore a quella che caratterizza il monitoraggio dello xeno, anche se il motivo per cui il metodo di rilevazione dell'argon non è ancora sfruttato è dato dall'estrema difficoltà che la sua misurazione comporta. Obiettivo di questo lavoro è proprio quello di analizzare le potenzialità dell'utilizzo di tale radioisotopo dell'argon per gli scopi di verifica del CTBT, di descrivere l'attuale stato dell'arte delle tecnologie disponibili per la sua misurazione e di valutare quantitativamente il fondo di argon-37 naturalmente presente, in modo da stabilire i criteri con cui si possa dire se le quantità rilevate siano compatibili con la normale presenza di tale radioisotopo oppure se siano senza dubbio dovute a un'avvenuta esplosione.

Modelling and Verification of a Correction Factor to Evaluate the Efficiency of Solar Thermal Collectors with the Rapid Test Method

A new method for the evaluation of the efficiency of parabolic trough collectors, called Rapid Test Method, is investigated at the Solar Institut Jülich. The basic concept is to carry out measurements under stagnation conditions. This allows a fast and inexpensive process due to the fact that no working fluid is required. With this approach, the temperature reached by the inner wall of the receiver is assumed to be the stagnation temperature and hence the average temperature inside the collector. This leads to a systematic error which can be rectified through the introduction of a correction factor. A model of the collector is simulated with COMSOL Multipyisics to study the size of the correction factor depending on collector geometry and working conditions. The resulting values are compared with experimental data obtained at a test rig at the Solar Institut Jülich. These results do not match with the simulated ones. Consequentially, it was not pos-sible to verify the model. The reliability of both the model with COMSOL Multiphysics and of the measurements are analysed. The influence of the correction factor on the rapid test method is also studied, as well as the possibility of neglecting it by measuring the receiver’s inner wall temperature where it receives the least amount of solar rays. The last two chapters analyse the specific heat capacity as a function of pressure and tem-perature and present some considerations about the uncertainties on the efficiency curve obtained with the Rapid Test Method.

Analsi biomeccanica di protesi transtibiali: dai test sulle componenti all'analisi del movimento in pazienti durante postura, cammino e corsa

L’elaborato propone una descrizione generale dei dispositivi protesici transtibiali, dalla storia, al processo di protesizzazione dei pazienti amputati alla struttura costitutiva modulare. Particolare importanza viene, inoltre, conferita all’analisi biomeccanica del movimento, della quale è data descrizione delle tecniche, della strumentazione, e dei risultati che vengono ottenuti nelle indagini di soggetti amputati rispetto alle persone normodotate. Considerazioni sulla biomeccanica di pazienti protesizzati transtibialmente offrono interessanti spunti di riflessione sulle componenti meccaniche, idrauliche ed elettroniche dei dispositivi modulari, delle quali si descrive in modo completo la struttura e il funzionamento, facendo riferimento ai brevetti tecnici più noti. Verranno, infine, riportati risultati di recenti studi e ricerche che evidenziano come l’utilizzo nei moduli prostesici transtibiali di caviglie idrauliche a controllo elettronico dello smorzamento sia la soluzione ottimale per ottenere risultati biomeccanicamente migliori, sia dal punto di vista dell’efficienza del movimento che per quanto riguarda l’equilibrio.

Considerations in the use of high intensity leg cycle ergometry as a test of muscular performance

High intensity leg cycle ergometry is a widely used method of measuring muscular performance during maximal exercise. Until recently, it was deemed to be a predominantly lower body activity; however, there is now evidence to suggest that the upper body could be making a significant contribution to power output, as demonstrated by the intense electrical activity of the forearm musculature. As high intensity cycle ergometry often is used to measure performance in untrained cyclists it is important they are given at least two familiarisation trials to ensure results are both reliable and reproducible. In addition, diurnal variations exist during a single high intensity bout of exercise. It is likely these daily fluctuations are influenced by a number of biochemical and physiological variables. The purpose of this article is to outline factors that contribute to our interpretation of data following high intensity cycle ergometry.