Measuring nursing competencies in the operating theatre: instrument development and psychometric analysis using Item Response Theory

BACKGROUND: Concern about the process of identifying underlying competencies that contribute to effective nursing performance has been debated with a lack of consensus surrounding an approved measurement instrument for assessing clinical performance. Although a number of methodologies are noted in the development of competency-based assessment measures, these studies are not without criticism. RESEARCH AIM: The primary aim of the study was to develop and validate a Performance Based Scoring Rubric, which included both analytical and holistic scales. The aim included examining the validity and reliability of the rubric, which was designed to measure clinical competencies in the operating theatre. RESEARCH METHOD: The fieldwork observations of 32 nurse educators and preceptors assessing the performance of 95 instrument nurses in the operating theatre were used in the calibration of the rubric. The Rasch model, a particular model among Item Response Models, was used in the calibration of each item in the rubric in an attempt at improving the measurement properties of the scale. This is done by establishing the 'fit' of the data to the conditions demanded by the Rasch model. RESULTS: Acceptable reliability estimates, specifically a high Cronbach's alpha reliability coefficient (0.940), as well as empirical support for construct and criterion validity for the rubric were achieved. Calibration of the Performance Based Scoring Rubric using Rasch model revealed that the fit statistics for most items were acceptable. CONCLUSION: The use of the Rasch model offers a number of features in developing and refining healthcare competency-based assessments, improving confidence in measuring clinical performance. The Rasch model was shown to be useful in developing and validating a competency-based assessment for measuring the competence of the instrument nurse in the operating theatre with implications for use in other areas of nursing practice.

HyperCard test engine using Rasch Analysis

This paper will focus on the development of an interactive test engine using Rasch analysis of item responses for question selection and reporting of results. The Rasch analysis is used to determine student ability and question difficulty. This model is widely used in the preparation of paper-based tests and has been the subject of particular use and development at the Australian Council for Education Research (ACER). This paper presents an overview of an interactive implementation of the Rasch analysis model in HyperCard, where student ability estimates are generated 'on the fly' and question difficulty values updated from time to time. The student ability estimates are used to determine question selection and are the basis of scoring and reporting schemes.

Prédire le professionnalisme : une exploration des modèles de la famille de Rasch et de la régression par classes latentes (Étude PERSONA)

L’étude cherche à provoquer la convergence des regards sur des enjeux méthodologiques fondamentaux, soit les enjeux de mesure, de décision et d’impact inhérents à toute démarche de sélection académique. À cet effet, elle explorera la capacité de prédiction de certaines variables non cognitives envers la compétence de professionnalisme observée chez les étudiants du doctorat professionnel de premier cycle en pharmacie. La sélection des candidats au sein des programmes académiques en santé repose en grande partie sur une évaluation de la capacité cognitive des étudiants. Tenant compte du virage compétence pris par la majorité des programmes en santé, la pertinence et la validité des critères traditionnels de sélection sont remises en question. La présente étude propose de valider l’utilisation des échelles de mesure de la personnalité, des valeurs et de l’autodétermination pour guider l’optimalité et l’équité des décisions de sélection. Les enjeux de mesure de ces variables seront abordés principalement par la modélisation dichotomique et polytomique de Rasch. L’application de la méthode des strates permettra, par la suite, de répondre aux enjeux de décision en procédant à une différenciation et un classement des étudiants. Puis, les enjeux d’impact seront, à leur tour, explorés par le modèle de régression par classes latentes. L’étude démontre notamment que le recours à la modélisation a permis une différenciation précise des étudiants. Cependant, la violation de certaines conditions d’application des modèles et la faible différenciation établie entre les étudiants sur la base des critères de professionnalisme, rendent l’évaluation de la capacité de prédiction de la personnalité, des valeurs et de l’autodétermination hasardeuse. À cet effet, les modèles identifiés par les analyses de régression par classes latentes s’avèrent peu concluants. Les classes latentes ainsi identifiées ne présentent pas de distinctions marquées et utiles à la sélection. Bien que les diverses procédures de modélisation proposées présentent des avantages intéressants pour une utilisation en contexte de sélection académique, des recherches additionnelles sur la qualité des critères de professionnalisme et sur la qualité des échelles de mesure des variables non cognitives demeurent nécessaires.

Aerosol indirect effects – general circulation model intercomparison and evaluation with satellite data

Aerosol indirect effects continue to constitute one of the most important uncertainties for anthropogenic climate perturbations. Within the international AEROCOM initiative, the representation of aerosol-cloud-radiation interactions in ten different general circulation models (GCMs) is evaluated using three satellite datasets. The focus is on stratiform liquid water clouds since most GCMs do not include ice nucleation effects, and none of the model explicitly parameterises aerosol effects on convective clouds. We compute statistical relationships between aerosol optical depth (τa) and various cloud and radiation quantities in a manner that is consistent between the models and the satellite data. It is found that the model-simulated influence of aerosols on cloud droplet number concentration (Nd ) compares relatively well to the satellite data at least over the ocean. The relationship between �a and liquid water path is simulated much too strongly by the models. This suggests that the implementation of the second aerosol indirect effect mainly in terms of an autoconversion parameterisation has to be revisited in the GCMs. A positive relationship between total cloud fraction (fcld) and �a as found in the satellite data is simulated by the majority of the models, albeit less strongly than that in the satellite data in most of them. In a discussion of the hypotheses proposed in the literature to explain the satellite-derived strong fcld–�a relationship, our results indicate that none can be identified as a unique explanation. Relationships similar to the ones found in satellite data between �a and cloud top temperature or outgoing long-wave radiation (OLR) are simulated by only a few GCMs. The GCMs that simulate a negative OLR - �a relationship show a strong positive correlation between �a and fcld. The short-wave total aerosol radiative forcing as simulated by the GCMs is strongly influenced by the simulated anthropogenic fraction of �a, and parameterisation assumptions such as a lower bound on Nd . Nevertheless, the strengths of the statistical relationships are good predictors for the aerosol forcings in the models. An estimate of the total short-wave aerosol forcing inferred from the combination of these predictors for the modelled forcings with the satellite-derived statistical relationships yields a global annual mean value of −1.5±0.5Wm−2. In an alternative approach, the radiative flux perturbation due to anthropogenic aerosols can be broken down into a component over the cloud-free portion of the globe (approximately the aerosol direct effect) and a component over the cloudy portion of the globe (approximately the aerosol indirect effect). An estimate obtained by scaling these simulated clearand cloudy-sky forcings with estimates of anthropogenic �a and satellite-retrieved Nd–�a regression slopes, respectively, yields a global, annual-mean aerosol direct effect estimate of −0.4±0.2Wm−2 and a cloudy-sky (aerosol indirect effect) estimate of −0.7±0.5Wm−2, with a total estimate of −1.2±0.4Wm−2.

Using Rasch modeling to re-evaluate three scales related to physical activity: Enjoyment, perceived benefits and perceived barriers

Studies suggest that enjoyment, perceived benefits and perceived barriers may be important mediators of physical activity. However, the psychometric properties of these scales have not been assessed using Rasch modeling. The purpose of this study was to use Rasch modeling to evaluate the properties of three scales commonly used in physical activity studies: the Physical Activity Enjoyment Scale, the Benefits of Physical Activity Scale and the Barriers to Physical Activity Scale. The scales were administered to 378 healthy adults, aged 25–75 years (50% women, 62% Whites), at the baseline assessment for a lifestyle physical activity intervention trial. The ConQuest software was used to assess model fit, item difficulty, item functioning and standard error of measurement. For all scales, the partial credit model fit the data. Item content of one scale did not adequately cover all respondents. Response options of each scale were not targeting respondents appropriately, and standard error of measurement varied across the total score continuum of each scale. These findings indicate that each scale's effectiveness at detecting differences among individuals may be limited unless changes in scale content and response format are made.

Improving the Psychometric Utility of the Hypomania Checklist (HCL-32): a Rasch Analysis Approach

Background The HCL-32 is a widely-used screening questionnaire for hypomania. We aimed to use a Rasch analysis approach to (i) evaluate the measurement properties, principally unidimensionality, of the HCL-32, and (ii) generate a score table to allow researchers to convert raw HCL-32 scores into an interval-level measurement which will be more appropriate for statistical analyses. Methods Subjects were part of the Bipolar Disorder Research Network (BDRN) study with DSM-IV bipolar disorder (n=389). Multidimensionality was assessed using the Rasch fit statistics and principle components analysis of the residuals (PCA). Item invariance (differential item functioning, DIF) was tested for gender, bipolar diagnosis and current mental state. Item estimates and reliabilities were calculated. Results Three items (29, 30, 32) had unacceptable fit to the Rasch unidimensional model. Item 14 displayed significant DIF for gender and items 8 and 17 for current mental state. Item estimates confirmed that not all items measure hypomania equally. Limitations This sample was recruited as part of a large ongoing genetic epidemiology study of bipolar disorder and may not be fully representative of the broader clinical population of individuals with bipolar disorder. Conclusion The HCL-32 is unidimensional in practice, but measurements may be further strengthened by the removal of four items. Re-scored linear measurements may be more appropriate for clinical research.

A competence model for environmental education

The goal of environmental education is ultimately to enable a person to strive for and to attain a more ecological way of life. In this article, we begin by distinguishing three forms of environmental knowledge and go on to predict that people's attitude toward nature represents the force that drives their ecological behavioral engagement. Based on data from 1,907 students, we calibrated previously established instruments to measure ecological behavior, environmental knowledge, and attitude toward nature with Rasch-type models. Using path modeling, we corroborated our theoretically anticipated competence structure. While environmental knowledge revealed a modest behavioral effect, attitude toward nature turned out to be, as expected, the stronger determinant of behavior. Overall, we propose a competence model that has the potential to guide us into more evidence-based ways of promoting the overall ecological engagement of individuals. (DIPF/Orig.)

An approximate method for the solution of an influence function foil rolling model

Fleck and Johnson (Int. J. Mech. Sci. 29 (1987) 507) and Fleck et al. (Proc. Inst. Mech. Eng. 206 (1992) 119) have developed foil rolling models which allow for large deformations in the roll profile, including the possibility that the rolls flatten completely. However, these models require computationally expensive iterative solution techniques. A new approach to the approximate solution of the Fleck et al. (1992) Influence Function Model has been developed using both analytic and approximation techniques. The numerical difficulties arising from solving an integral equation in the flattened region have been reduced by applying an Inverse Hilbert Transform to get an analytic expression for the pressure. The method described in this paper is applicable to cases where there is or there is not a flat region.