Evaluation of the Computer-based Battery for Oral Language (BILOv3) through the Rasch model for a Brazilian Sample

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Estimation of the Proportion of Underachieving Students in Compulsory Secondary Education in Spain: An Application of the Rasch Model

There are very few studies in Spain that treat underachievement rigorously, and those that do are typically related to gifted students. The present study examined the proportion of underachieving students using the Rasch measurement model. A sample of 643 first-year high school students (mean age = 12.09; SD = 0.47) from 8 schools in the province of Alicante (Spain) completed the Battery of Differential and General Skills (Badyg), and these students' General Points Average (GPAs) were recovered by teachers. Dichotomous and Partial credit Rasch models were performed. After adjusting the measurement instruments, the individual underachievement index provided a total sample of 181 underachieving students, or 28.14% of the total sample across the ability levels. This study confirms that the Rasch measurement model can accurately estimate the construct validity of both the intelligence test and the academic grades for the calculation of underachieving students. Furthermore, the present study constitutes a pioneer framework for the estimation of the prevalence of underachievement in Spain.

X-ray your data with Rasch

By using the Rasch model, much detailed diagnostic information is available to developers of survey and assessment instruments and to the researchers who use them. We outline an approach to the analysis of data obtained from the administration of survey instruments that can enable researchers to recognise and diagnose difficulties with those instruments and then to suggest remedial actions that can improve the measurement properties of the scales included in questionnaires. We illustrate the approach using examples drawn from recent research and demonstrate how the approach can be used to generate figures that make the results of Rasch analyses accessible to non-specialists.

Evaluation of a skin self examination attitude scale using an item response theory model approach

Introduction The Skin Self-Examination Attitude Scale (SSEAS) is a brief measure that allows for the assessment of attitudes in relation to skin self-examination. This study evaluated the psychometric properties of the SSEAS using Item Response Theory (IRT) methods in a large sample of men ≥ 50 years in Queensland, Australia. Methods A sample of 831 men (420 intervention and 411 control) completed a telephone assessment at the 13-month follow-up of a randomized-controlled trial of a video-based intervention to improve skin self-examination (SSE) behaviour. Descriptive statistics (mean, standard deviation, item–total correlations, and Cronbach’s alpha) were compiled and difficulty parameters were computed with Winsteps using the polytomous Rasch Rating Scale Model (RRSM). An item person (Wright) map of the SSEAS was examined for content coverage and item targeting. Results The SSEAS have good psychometric properties including good internal consistency (Cronbach’s alpha = 0.80), fit with the model and no evidence for differential item functioning (DIF) due to experimental trial grouping was detected. Conclusions The present study confirms the SSEA scale as a brief, useful and reliable tool for assessing attitudes towards skin self-examination in a population of men 50 years or older in Queensland, Australia. The 8-item scale shows unidimensionality, allowing levels of SSE attitude, and the item difficulties, to be ranked on a single continuous scale. In terms of clinical practice, it is very important to assess skin cancer self-examination attitude to identify people who may need a more extensive intervention to allow early detection of skin cancer.

Robustesse du modèle de Rasch unidimensionnel à la violation de l’hypothèse d’unidimensionnalité.

Le logiciel de simulation des données et d'analyse est Conquest V.3

Psychometric Properties of the Inventário Dimensional Clínico da Personalidade (IDCP) using the Rating Scale Model

The aim of this study was to evaluate the performance of the Dimensional Clinical Personality Inventory (DCPI) using Rasch-based person and item analysis. 1281 participants were recruited, between 18 and 90 years of age (M=26.64; SD=8.94), 431 men (33.6%) and 127 (9.9%) patients diagnosed with axis I disorders and/or axis II according to DSM-IV-TR. Results indicated the IDCP scales performed reasonably well, and the usefulness of the analyses presented, demonstrate the Rasch model’s applicability for clinical applications. Among the important tools offered by the Rasch model, we explore the use of the person-item map, which visually presents the intuitively understandable psychological construct along the dimensional scale of the instrument.

A Rasch analysis of the teachers music confidence scale

This article presents a new measure of teachers’ confidence to conduct musical activities with young children; Teachers Music Confidence Scale (TMCS). The TMCS was developed using a sample of 284 in-service and pre-service early childhood teachers in Hong Kong Special Administrative Region (HKSAR). The TMCS consisted of 10 musical activities. Teachers rated their confidence levels to conduct each activity on a scale from 1 (Not confident at all) to 5 (Very confident). An exploratory factor analysis retained a 10-item single factor that was replicated using confirmatory factor analysis procedures. All items of the TMCS fitted the Rasch model adequately. In-service teachers showed higher confidence levels to conduct several musical activities with young children than pre-service teachers. Implications of these findings for measuring teachers’ confidence to conduct musical activities with young children were discussed.

Development of a Rasch homogenous short form of the Pathological Narcissism Inventory

During the last decades, the narcissistic personality inventory (npi) was the most widely used questionnaire to measure narcissism as a personality trait. But the npi assesses grandiose narcissism only, while recent discussions emphasize the existence of vulnerable narcissism. The pathological narcissism inventory (pni, pincus et al., 2009) is a new questionnaire assessing these different aspects of narcissism. However, with 54 items on seven subscales, the pni is quite long to serve as a screening tool for narcissistic traits. We therefore developed a short form to facilitate its application in research and practice. Even though the pni covers different symptoms of narcissism, they are all expressions of the same underlying construct. We therefore used the rasch model to guide the item selection. Method and results: a sample of 1837 participants (67.5% female, mean age 26.8 years) was used to choose the items for the short form. Two criteria were adopted: all aspects, represented by the seven subscales in the original, should be retained, and items should be rasch homogenous. In a step-by-step procedure we excluded items successively until reaching a homogenous pool of 22 items. All remaining items had satisfactory fit indices and fitstatistics for the model were good. characteristics of the resulting short form were tested using a new independent validation sample (n=104, mean age = 32.8, 45% female). Correlations of the short pni with different validation measures were comparable to the correlations obtained with the original form, indicating that the two forms were equivalent. Conclusion: the resulting one-dimensional measure can be used as a screening questionnaire for pathological narcissism. The rasch homogeneity facilitates the comparison of narcissism scores among a variety of samples.

Many Facet, Common Latent Trait Polytomous IRT Model and EM Algorithm

2000 Mathematics Subject Classification: 91E45.

Développement d’un questionnaire cognitif bref, le FaCE (The Fast Cognitive Evaluation), selon la modélisation Rasch adapté aux patients avec cancer

Les déficits cognitifs sont présents chez les patients atteints de cancer. Les tests cognitifs tels que le Montreal Cognitive Assessment se sont révélés peu spécifiques, incapables de détecter des déficits légers et ne sont pas linéaires. Pour suppléer à ces limitations nous avons développé un questionnaire cognitif simple, bref et adapté aux dimensions cognitives atteintes chez les patients avec un cancer, le FaCE « The Fast Cognitif Evaluation », en utilisant la modélisation Rasch (MR). La MR est une méthode mathématique probabiliste qui détermine les conditions pour qu’un outil soit considéré une échelle de mesure et elle est indépendante de l’échantillon. Si les résultats s’ajustent au modèle, l’échelle de mesure est linéaire avec des intervalles égaux. Les réponses sont basées sur la capacité des sujets et la difficulté des items. La carte des items permet de sélectionner les items les plus adaptés pour l’évaluation de chaque aspect cognitif et d’en réduire le nombre au minimum. L’analyse de l’unidimensionnalité évalue si l’outil mesure une autre dimension que celle attendue. Les résultats d’analyses, conduites sur 165 patients, montrent que le FaCE distingue avec une excellente fiabilité et des niveaux suffisamment différents les compétences des patients (person-reliability-index=0.86; person-separation-index=2.51). La taille de la population et le nombre d’items sont suffisants pour que les items aient une hiérarchisation fiable et précise (item-reliability=0.99; item-séparation-index=8.75). La carte des items montre une bonne dispersion de ceux-ci et une linéarité du score sans effet plafond. Enfin, l’unidimensionnalité est respectée et le temps d’accomplissement moyen est d’environ 6 minutes. Par définition la MR permet d’assurer la linéarité et la continuité de l’échelle de mesure. Nous avons réussi à développer un questionnaire bref, simple, rapide et adapté aux déficits cognitifs des patients avec un cancer. Le FaCE pourrait, aussi, servir de mesure de référence pour les futures recherches dans le domaine.

Manual do software xcalibre para aplicação do modelo de rasch

Dissertação (mestrado)—Universidade de Brasília, Instituto de Física, Programa de Pós-Graduação de Mestrado Profissional em Ensino de Física, Mestrado Nacional Profissional em Ensino de Física, 2015.

Deriving a preference-based measure for cancer using the EORTC QLQ-C30 : a confirmatory versus exploratory approach

Background: To derive preference-based measures from various condition-specific descriptive health-related quality of life (HRQOL) measures. A general 2-stage method is evolved: 1) an item from each domain of the HRQOL measure is selected to form a health state classification system (HSCS); 2) a sample of health states is valued and an algorithm derived for estimating the utility of all possible health states. The aim of this analysis was to determine whether confirmatory or exploratory factor analysis (CFA, EFA) should be used to derive a cancer-specific utility measure from the EORTC QLQ-C30. Methods: Data were collected with the QLQ-C30v3 from 356 patients receiving palliative radiotherapy for recurrent or metastatic cancer (various primary sites). The dimensional structure of the QLQ-C30 was tested with EFA and CFA, the latter based on a conceptual model (the established domain structure of the QLQ-C30: physical, role, emotional, social and cognitive functioning, plus several symptoms) and clinical considerations (views of both patients and clinicians about issues relevant to HRQOL in cancer). The dimensions determined by each method were then subjected to item response theory, including Rasch analysis. Results: CFA results generally supported the proposed conceptual model, with residual correlations requiring only minor adjustments (namely, introduction of two cross-loadings) to improve model fit (increment χ2(2) = 77.78, p < .001). Although EFA revealed a structure similar to the CFA, some items had loadings that were difficult to interpret. Further assessment of dimensionality with Rasch analysis aligned the EFA dimensions more closely with the CFA dimensions. Three items exhibited floor effects (>75% observation at lowest score), 6 exhibited misfit to the Rasch model (fit residual > 2.5), none exhibited disordered item response thresholds, 4 exhibited DIF by gender or cancer site. Upon inspection of the remaining items, three were considered relatively less clinically important than the remaining nine. Conclusions: CFA appears more appropriate than EFA, given the well-established structure of the QLQ-C30 and its clinical relevance. Further, the confirmatory approach produced more interpretable results than the exploratory approach. Other aspects of the general method remain largely the same. The revised method will be applied to a large number of data sets as part of the international and interdisciplinary project to develop a multi-attribute utility instrument for cancer (MAUCa).

Development and validation of the ankle fracture outcome of rehabilitation measure (A-FORM)

Study Design Delphi panel and cohort study. Objective To develop and refine a condition-specific, patient-reported outcome measure, the Ankle Fracture Outcome of Rehabilitation Measure (A-FORM), and to examine its psychometric properties, including factor structure, reliability, and validity, by assessing item fit with the Rasch model. Background To our knowledge, there is no patient-reported outcome measure specific to ankle fracture with a robust content foundation. Methods A 2-stage research design was implemented. First, a Delphi panel that included patients and health professionals developed the items and refined the item wording. Second, a cohort study (n = 45) with 2 assessment points was conducted to permit preliminary maximum-likelihood exploratory factor analysis and Rasch analysis. Results The Delphi panel reached consensus on 53 potential items that were carried forward to the cohort phase. From the 2 time points, 81 questionnaires were completed and analyzed; 38 potential items were eliminated on account of greater than 10% missing data, factor loadings, and uniqueness. The 15 unidimensional items retained in the scale demonstrated appropriate person and item reliability after (and before) removal of 1 item (anxious about footwear) that had a higher-than-ideal outfit statistic (1.75). The “anxious about footwear” item was retained in the instrument, but only the 14 items with acceptable infit and outfit statistics (range, 0.5–1.5) were included in the summary score. Conclusion This investigation developed and refined the A-FORM (Version 1.0). The A-FORM items demonstrated favorable psychometric properties and are suitable for conversion to a single summary score. Further studies utilizing the A-FORM instrument are warranted. J Orthop Sports Phys Ther 2014;44(7):488–499. Epub 22 May 2014. doi:10.2519/jospt.2014.4980

Can population-level variation in early childhood development be measured in a manner that supports service planning and intervention?

Background: The Early Development Instrument (EDI) is a population-level measure of five developmental domains at school-entry age. The overall aim of this thesis was to explore the potential of the EDI as an indicator of early development in Ireland. Methods: A cross-sectional study was conducted in 47 primary schools in 2011 using the EDI and a linked parental questionnaire. EDI (teacher completed) scores were calculated for 1,344 children in their first year of full-time education. Those scoring in the lowest 10% of the sample population in one or more domains were deemed to be 'developmentally vulnerable'. Scores were correlated with contextual data from the parental questionnaire and with indicators of area and school-level deprivation. Rasch analysis was used to determine the validity of the EDI. Results: Over one quarter (27.5%) of all children in the study were developmentally vulnerable. Individual characteristics associated with increased risk of vulnerability were being male; under 5 years old; and having English as a second language. Adjusted for these demographics, low birth weight, poor parent/child interaction and mother’s lower level of education showed the most significant odds ratios for developmental vulnerability. Vulnerability did not follow the area-level deprivation gradient as measured by a composite index of material deprivation. Children considered by the teacher to be in need of assessment also had lower scores, which were not significantly different from those of children with a clinical diagnosis of special needs. all domains showed at least reasonable fit to the Rasch model supporting the validity of the instrument. However, there was a need for further refinement of the instrument in the Irish context. Conclusion: This thesis provides a unique snapshot of early development in Ireland. The EDI and linked parental questionnaires are promising indicators of the extent, distribution and determinants of developmental vulnerability.

Calibration of assessment instruments – classification in mathematics at the Universidad Jorge Tadeo Lozano

Item Response Theory, IRT, is a valuable methodology for analyzing the quality of the instruments utilized in assessment of academic achievement. This article presents an implementation of the mentioned theory, particularly of the Rasch model, in order to calibrate items and the instrument used in the classification test for the Basic Mathematics subject at Universidad Jorge Tadeo Lozano. 509 responses chains of students, obtained in the june 2011 application, were analyzed with a set of 45 items, through eight case studies that are showing progressive steps of calibration. Criteria of validity of items and of whole instrument were defined and utilized, to select groups of responses chains and items that were finally used in the determination of parameters which then allowed the classification of assessed students by the test.