Evaluation of the Computer-based Battery for Oral Language (BILOv3) through the Rasch model for a Brazilian Sample

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Estimation of the Proportion of Underachieving Students in Compulsory Secondary Education in Spain: An Application of the Rasch Model

There are very few studies in Spain that treat underachievement rigorously, and those that do are typically related to gifted students. The present study examined the proportion of underachieving students using the Rasch measurement model. A sample of 643 first-year high school students (mean age = 12.09; SD = 0.47) from 8 schools in the province of Alicante (Spain) completed the Battery of Differential and General Skills (Badyg), and these students' General Points Average (GPAs) were recovered by teachers. Dichotomous and Partial credit Rasch models were performed. After adjusting the measurement instruments, the individual underachievement index provided a total sample of 181 underachieving students, or 28.14% of the total sample across the ability levels. This study confirms that the Rasch measurement model can accurately estimate the construct validity of both the intelligence test and the academic grades for the calculation of underachieving students. Furthermore, the present study constitutes a pioneer framework for the estimation of the prevalence of underachievement in Spain.

Effectiveness of holistic neuropsychological rehabilitation for Spanish population with acquired brain injury measured using Rasch analysis.

INTRODUCTION The Rasch model is increasingly used in the field of rehabilitation because it improves the accuracy of measurements of patient status and their changes after therapy. OBJECTIVE To determine the long-term effectiveness of a holistic neuropsychological rehabilitation program for Spanish outpatients with acquired brain injury (ABI) using Rasch analysis. METHODS Eighteen patients (ten with long evolution - patients who started the program > 6 months after ABI- and eight with short evolution) and their relatives attended the program for 6 months. Patients' and relatives' answers to the European Brain Injury Questionnaire and the Frontal Systems Behavior Scale at 3 time points (pre-intervention. post-intervention and 12 month follow-up) were transformed into linear measures called logits. RESULTS The linear measures revealed significant improvements with large effects at the follow-up assessment on cognitive and executive functioning, social and emotional self-regulation, apathy and mood. At follow-up, the short evolution group achieved greater improvements in mood and cognitive functioning than the long evolution patients. CONCLUSIONS The program showed long-term effectiveness for most of the variables, and it was more effective for mood and cognitive functioning when patients were treated early. Relatives played a key role in the effectiveness of the rehabilitation program.

Robustesse du modèle de Rasch unidimensionnel à la violation de l’hypothèse d’unidimensionnalité.

Le logiciel de simulation des données et d'analyse est Conquest V.3

Psychometric Properties of the Inventário Dimensional Clínico da Personalidade (IDCP) using the Rating Scale Model

The aim of this study was to evaluate the performance of the Dimensional Clinical Personality Inventory (DCPI) using Rasch-based person and item analysis. 1281 participants were recruited, between 18 and 90 years of age (M=26.64; SD=8.94), 431 men (33.6%) and 127 (9.9%) patients diagnosed with axis I disorders and/or axis II according to DSM-IV-TR. Results indicated the IDCP scales performed reasonably well, and the usefulness of the analyses presented, demonstrate the Rasch model’s applicability for clinical applications. Among the important tools offered by the Rasch model, we explore the use of the person-item map, which visually presents the intuitively understandable psychological construct along the dimensional scale of the instrument.

Development of a Rasch homogenous short form of the Pathological Narcissism Inventory

During the last decades, the narcissistic personality inventory (npi) was the most widely used questionnaire to measure narcissism as a personality trait. But the npi assesses grandiose narcissism only, while recent discussions emphasize the existence of vulnerable narcissism. The pathological narcissism inventory (pni, pincus et al., 2009) is a new questionnaire assessing these different aspects of narcissism. However, with 54 items on seven subscales, the pni is quite long to serve as a screening tool for narcissistic traits. We therefore developed a short form to facilitate its application in research and practice. Even though the pni covers different symptoms of narcissism, they are all expressions of the same underlying construct. We therefore used the rasch model to guide the item selection. Method and results: a sample of 1837 participants (67.5% female, mean age 26.8 years) was used to choose the items for the short form. Two criteria were adopted: all aspects, represented by the seven subscales in the original, should be retained, and items should be rasch homogenous. In a step-by-step procedure we excluded items successively until reaching a homogenous pool of 22 items. All remaining items had satisfactory fit indices and fitstatistics for the model were good. characteristics of the resulting short form were tested using a new independent validation sample (n=104, mean age = 32.8, 45% female). Correlations of the short pni with different validation measures were comparable to the correlations obtained with the original form, indicating that the two forms were equivalent. Conclusion: the resulting one-dimensional measure can be used as a screening questionnaire for pathological narcissism. The rasch homogeneity facilitates the comparison of narcissism scores among a variety of samples.

Many Facet, Common Latent Trait Polytomous IRT Model and EM Algorithm

2000 Mathematics Subject Classification: 91E45.

Développement d’un questionnaire cognitif bref, le FaCE (The Fast Cognitive Evaluation), selon la modélisation Rasch adapté aux patients avec cancer

Les déficits cognitifs sont présents chez les patients atteints de cancer. Les tests cognitifs tels que le Montreal Cognitive Assessment se sont révélés peu spécifiques, incapables de détecter des déficits légers et ne sont pas linéaires. Pour suppléer à ces limitations nous avons développé un questionnaire cognitif simple, bref et adapté aux dimensions cognitives atteintes chez les patients avec un cancer, le FaCE « The Fast Cognitif Evaluation », en utilisant la modélisation Rasch (MR). La MR est une méthode mathématique probabiliste qui détermine les conditions pour qu’un outil soit considéré une échelle de mesure et elle est indépendante de l’échantillon. Si les résultats s’ajustent au modèle, l’échelle de mesure est linéaire avec des intervalles égaux. Les réponses sont basées sur la capacité des sujets et la difficulté des items. La carte des items permet de sélectionner les items les plus adaptés pour l’évaluation de chaque aspect cognitif et d’en réduire le nombre au minimum. L’analyse de l’unidimensionnalité évalue si l’outil mesure une autre dimension que celle attendue. Les résultats d’analyses, conduites sur 165 patients, montrent que le FaCE distingue avec une excellente fiabilité et des niveaux suffisamment différents les compétences des patients (person-reliability-index=0.86; person-separation-index=2.51). La taille de la population et le nombre d’items sont suffisants pour que les items aient une hiérarchisation fiable et précise (item-reliability=0.99; item-séparation-index=8.75). La carte des items montre une bonne dispersion de ceux-ci et une linéarité du score sans effet plafond. Enfin, l’unidimensionnalité est respectée et le temps d’accomplissement moyen est d’environ 6 minutes. Par définition la MR permet d’assurer la linéarité et la continuité de l’échelle de mesure. Nous avons réussi à développer un questionnaire bref, simple, rapide et adapté aux déficits cognitifs des patients avec un cancer. Le FaCE pourrait, aussi, servir de mesure de référence pour les futures recherches dans le domaine.

Manual do software xcalibre para aplicação do modelo de rasch

Dissertação (mestrado)—Universidade de Brasília, Instituto de Física, Programa de Pós-Graduação de Mestrado Profissional em Ensino de Física, Mestrado Nacional Profissional em Ensino de Física, 2015.

ADHD in a representative sample of the Brazilian population: estimated prevalence and comparative adequacy of criteria between adolescents and adults according to the item response theory

Background: Attention deficit hyperactivity disorder (ADHD) is a clinically significant disorder in adulthood, but current diagnostic criteria and instruments do not seem to adequately capture the complexity of the disorder in this developmental phase. Accordingly, there are limited data on the proportion of adults affected by the disorder, specially in developing countries. Method: We assessed a representative household sample of the Brazilian population for ADHD with the Adult ADHD Self-report Scale (ASRS) Screener, and evaluated the instrument according to the Rasch model of item response theory. Results: The sample was comprised by 3007 individuals, and the overal prevalence of positive screeners for ADHD was 5.8% [95% confidence interval (CI), 4.8-7.0]. Rasch analyses revealed the misfitt of the overall sample to expectations of the model. The evaluation of the sample stratified by age revealed that data for adolescents showed a signficant fittnes to the model expectations, while items completed by adults were not adequated. Conclusions: The lack of fitness to the model for adult respondents challenges the possibility of a linear transformation of the ordinal data into interval measures and the utilization of parametric analyses of data. This result suggests that diagnostic criteria and instruments for adult ADHD must take into account a developmental perspective. Moreover, it calls for further evaluation of currently employed research methods in light of modern theories of psychometrics. Copyright (C) 2010 John Wiley & Sons, Ltd.

Developing and validating a measure of the strength of the HRM system: Operationalizing the construct and relationships among its dimensions

The four studies in this article introduce a questionnaire to measure Strength of the HRM System (HRMSQ), a multidimensional construct, theoretically developed by Bowen and Ostroff (2004). Strength of the HRM System is a set of process characteristics that lead to effectiveness in conveying signals to employees that allow them to create a shared meaning of desired and appropriate work behaviours. Nine characteristics are suggested, grouped in three features: Distinctiveness, Consistency and Consensus. Study 1 developed and tested a questionnaire in a sample of workers from five different sectors. Study 2 cross-validated the measure in a sample of civil servants in a municipality. These two studies used performance appraisal as the reference HRM practice and led to a short version of the HRMSQ. Study 3 and Study 4 extend the HRMSQ to several common HRM practices. The HRMSQ is tested in two samples, of call center and several private and public organizations‟ workers (study 3). In study 4 the questionnaire is refined and tested with a sample from a hotel chain and finally cross-validated with two other samples, in the insurance and batteries sectors, leading to a longer version of the HRMSQ. Content analysis of several interviews with human resource managers and the Rasch model (1960, 1961, 1980), were used to define and select the indicators of the questionnaire. Convergent, discriminant and predictive validity of the measure are tested. The results of the four studies highlight the complexity of the relationships between the proposed characteristics and support the validity of a parsimonious measure of Strength of the HRM System.

Tradução e validação para Portugal do Patient Activation Measure 13 : em pessoas com Diabetes Mellitus tipo 2

RESUMO - Introdução: A Diabetes Mellitus Tipo 2 (DM2) tem uma elevada prevalência em todo o mundo, com impacto significativo a nível de Saúde Pública, na vida dos doentes e nos custos que lhe são associados. O Patiente Activation Measure 13 (PAM13) é um questionário que possibilita a avaliação das crenças, conhecimentos, motivação e capacidades de uma pessoa em relação à sua saúde; pelo que a sua utilização na DM2 é pertinente. Objetivos: Traduzir o PAM 13, versão curta, para Português de Portugal; Estabelecer as propriedades psicométricas da versão Portuguesa do PAM 13 (PAM13-P); Validar a PAM 13-P num grupo de pessoas com DM2. Material e métodos: O processo de tradução e adaptação cultural do questionário foi composto pelas fases: 1-Tradução, 2-Reconciliação e síntese, 3-Back translation, 4-Rever e sintetizar a Back translation, 5-Harmonização, 6-Revisão do comité de peritos, 7-Cognitive debriefing e 8-Avaliação final. Para validar o PAM13-P realizou-se um estudo observacional transversal analítico com uma amostra de conveniência, de pessoas com DM2, seguidas na Associação Protectora dos Diabéticos de Portugal. O questionário foi de autopreenchimento e foi consultado o processo clínico para obtenção da HbA1c. O tratamento estatístico foi realizado através do SPSS 21® e Winsteps v3.8.1®. Resultados e discussão: O processo de tradução e adaptação cultural foi realizado de acordo com as guidelines. Foram realizados 3 painéis de e-Delphi, com 21 participantes de áreas distintas, tendo-se obtido bons níveis de concordância. As principais modificações realizadas ao questionário foram a simplificação da linguagem e dos itens, obtendo-se as equivalências necessárias. O PAM13-P foi aplicado a 201 pessoas, sendo que a taxa de resposta foi de 83%. Na amostra analisada 57.3% eram homens. Obtiveram-se as médias de idade 67.1 anos e de duração de diabetes 17.3 anos. A média do score do PAM foi 58.5±10.1(41.8-90.5) pontos e 49,7% da amostra estava no nível 3 de ativação. Relativamente aos itens verificou-se que os itens mais difíceis foram o 13(56.1) e o 8(55.4) e o com menor dificuldade foi 4(38.5). As categorias de resposta tiveram um bom ajuste ao Modelo de Rasch. O ajustamento dos itens foi infit entre 0.779-1.177 e outfit entre 0.794-1.315. A fiabilidade dos indivíduos variou entre 0.77(real) e 0.83(modelo) e dos itens foi de 0.97 (real e modelo). O Alfa de Cronbach foi bom (α=0.82). Estas estatísticas foram semelhantes aos da validação do PAM13. Existiu uma relação entre o score do PAM e os itens de validação em 51%. Das variáveis analisadas, existia um relação do nível de ativação com a idade e com a HbA1c. Conclusões: O PAM13-P foi traduzido e adaptado culturalmente para Português e foi validado em pessoas com DM2, sendo as propriedades psicométricas boas.

Ascertaining late-life depressive symptoms in Europe: an evaluation of the survey version of the EURO-D scale in 10 nations. The SHARE project.

The reported prevalence of late-life depressive symptoms varies widely between studies, a finding that might be attributed to cultural as well as methodological factors. The EURO-D scale was developed to allow valid comparison of prevalence and risk associations between European countries. This study used Confirmatory Factor Analysis (CFA) and Rasch models to assess whether the goal of measurement invariance had been achieved; using EURO-D scale data collected in 10 European countries as part of the Survey of Health, Ageing and Retirement in Europe (SHARE) (n = 22,777). The results suggested a two-factor solution (Affective Suffering and Motivation) after Principal Component Analysis (PCA) in 9 of the 10 countries. With CFA, in all countries, the two-factor solution had better overall goodness-of-fit than the one-factor solution. However, only the Affective Suffering subscale was equivalent across countries, while the Motivation subscale was not. The Rasch model indicated that the EURO-D was a hierarchical scale. While the calibration pattern was similar across countries, between countries agreement in item calibrations was stronger for the items loading on the affective suffering than the motivation factor. In conclusion, there is evidence to support the EURO-D as either a uni-dimensional or bi-dimensional scale measure of depressive symptoms in late-life across European countries. The Affective Suffering sub-component had more robust cross-cultural validity than the Motivation sub-component.

Construction d'une échelle décrivant les niveaux de compétence de collaboration, à partir d'indicateurs validés par des enseignants cliniciens en médecine

La collaboration est une compétence essentielle que les futurs médecins doivent développer. La détermination des niveaux de compétence est cruciale dans la planification de cet apprentissage. Les échelles descriptives suscitent un intérêt croissant, car elles décrivent en termes qualitatifs les performances attendues. Nous inspirant de la méthodologie mixte de Blais, Laurier, & Rousseau (2009), nous avons construit en cinq étapes une échelle de niveau de compétence de collaboration: 1) formulation d’une liste d’indicateurs situés à quatre niveaux de la formation médicale (préclinique, externat, résidence junior et sénior) par les chercheurs (n= 3) et un groupe d’éducateurs (n=7), leaders pédagogiques possédant une expertise pour la compétence de collaboration; 2) sondage en ligne comprenant quatre questionnaires portant sur les niveaux de 118 indicateurs, auprès d’enseignants cliniciens représentant les différentes spécialités (n=277); 3) analyse, avec le modèle partial credit de Rasch, des réponses aux questionnaires appariés par calibration concurrente; 4) détermination des niveaux des indicateurs par les éducateurs et les chercheurs; et 5) rédaction de l’échelle à partir des indicateurs de chaque niveau. L’analyse itérative des réponses montre une adéquation au modèle de Rasch et répartit les indicateurs sur l’échelle linéaire aux quatre niveaux. Les éducateurs déterminent le niveau des 111 indicateurs retenus en tenant compte des résultats du sondage et de la cohérence avec le curriculum. L’échelle comporte un paragraphe descriptif par niveau, selon trois capacités : 1) participer au fonctionnement d’une équipe; 2) prévenir et gérer les conflits; et 3) planifier, coordonner et dispenser les soins en équipe. Cette échelle rend explicites les comportements collaboratifs attendus à la fin de chaque niveau et est utile à la planification de l’apprentissage et de l’évaluation de cette compétence. La discordance entre les niveaux choisis par les éducateurs et ceux issus de l’analyse des réponses des enseignants cliniciens est principalement due au faible choix de réponse du niveau préclinique par les enseignants et aux problèmes d’adéquation pour les indicateurs décrivant la gestion des conflits. Cette recherche marque une avan- cée dans la compréhension de la compétence de collaboration et démontre l’efficacité de la méthodologie de Blais (2009) dans un contexte de compétence transversale, en sciences de la santé. Cette méthodologie pourrait aider à approfondir les trajectoires de développement d’autres compétences.

Demi-vérités et vrais mensonges : une analyse des processus liés à la dissimulation dans les questionnaires informatisés

De nombreux chercheurs et cliniciens sont sceptiques quant à la validité des questionnaires autoadministrés, lorsqu’utilisés auprès d’une population carcérale (Gendreau, Irvine et Knight, 1973), surtout si celle-ci est composée de délinquants sexuels (Marshall et Hall, 1995). La sensibilité des sujets investigués jumelés à la transparence des questions expose l’évaluateur à la possibilité que le participant dissimule et modifie ses réponses (Tierney et McCabe, 2001). L’objectif de ce projet est de comprendre les processus impliqués dans la performance des participants à une évaluation autoadministrée. Les données de 282 délinquants sexuels ayant complétés le Multidimensional Inventory of Development, Sex, and Agression (MIDSA) ont été analysées afin de mieux comprendre l’interaction entre les échelles de désirabilité sociale, les temps de latence et les coefficients d’ajustement du modèle de Rasch. La convergence des analyses de temps de latence et des échelles de désirabilité sociale semble indiquer que certains participants dissimuleraient consciemment leurs réponses. Notamment, les participants détectés par les échelles de désirabilité sociales sembleraient répondre de manière de plus lente aux échelles d’évaluations, et certains d’entre eux offriraient des patrons de réponses incohérents à la prescription du modèle de Rasch. Les hypothèses permettant d’expliquer les potentiels mécanismes liés à la dissimulation seront discutées.