Item analysis /

Thesis (M. S.)--University of Illinois at Urbana-Champaign.

Evaluating the properties of a stage-specific self-efficacy scale for physical activity using classical test theory, confirmatory factor analysis and item response modeling

The purpose of this paper was to evaluate the psychometric properties of a stage-specific selfefficacy scale for physical activity with classical test theory (CTT), confirmatory factor analysis (CFA) and item response modeling (IRM). Women who enrolled in the Women On The Move study completed a 20-item stage-specific self-efficacy scale developed for this study [n = 226, 51.1% African-American and 48.9% Hispanic women, mean age = 49.2 (67.0) years, mean body mass index = 29.7 (66.4)]. Three analyses were conducted: (i) a CTT item analysis, (ii) a CFA to validate the factor structure and (iii) an IRM analysis. The CTT item analysis and the CFA results showed that the scale had high internal consistency (ranging from 0.76 to 0.93) and a strong factor structure. Results also showed that the scale could be improved by modifying or eliminating some of the existing items without significantly altering the content of the scale. The IRM results also showed that the scale had few items that targeted high self-efficacy and the stage-specific assumption underlying the scale was rejected. In addition, the IRM analyses found that the five-point response format functioned more like a four-point response format. Overall, employing multiple methods to assess the psychometric properties of the stage-specific self-efficacy scale demonstrated the complimentary nature of these methods and it highlighted the strengths and weaknesses of this scale.

Validation of the Edinburgh Postnatal Depression Scale for men, and comparison of item endorsement with their partners

Background: The Edinburgh Postnatal Depression Scale (EPDS) has been validated and used extensively in screening for depression in new mothers, both in English speaking and non-English speaking communities. While some studies have reported the use of the EPDS with Fathers, none have validated it for this group, and thus the appropriate cut-off score for screening for depression or anxiety caseness for this population is not known. Method: Couples were recruited antenatally and interviewed at six weeks postpartum. EPDS scores and distress caseness (depression or anxiety disorders) for 208 fathers and 230 mothers were determined using the Diagnostic Interview Schedule. Results: Analyses of the EPDS for fathers using distress caseness (depression or anxiety disorders) as the criterion shows that a cut-off of 5/6 has optimum receiver operating characteristics. Furthermore acceptable reliability (split-half and internal consistency) and validity (concurrent) coefficients were obtained. For mothers the optimum cut-off screening value to detect distress caseness was 7/8. Item analysis revealed that fathers endorsed seven of the ten items at lower rates to mothers, with the most significant being that referring to crying. Conclusions: The EPDS is a reliable and valid measure of mood in fathers. Screening for depression or anxiety disorders in fathers requires a two point lower cut-off than screening for depression or anxiety in mothers, and we recommend this cut-off to he 5/6. (C) 2001 Elsevier Science B.V. All rights reserved.

The hospital survey on patient safety culture in Portuguese hospitals: instrument validity and reliability

Purpose – Quantitative instruments to assess patient safety culture have been developed recently and a few review articles have been published. Measuring safety culture enables healthcare managers and staff to improve safety behaviours and outcomes for patients and staff. The study aims to determine the AHRQ Hospital Survey on Patient Safety Culture (HSPSC) Portuguese version's validity and reliability. Design/methodology/approach – A missing-value analysis and item analysis was performed to identify problematic items. Reliability analysis, inter-item correlations and inter-scale correlations were done to check internal consistency, composite scores. Inter-correlations were examined to assess construct validity. A confirmatory factor analysis was performed to investigate the observed data's fit to the dimensional structure proposed in the AHRQ HSPSC Portuguese version. To analyse differences between hospitals concerning composites scores, an ANOVA analysis and multiple comparisons were done. Findings – Eight of 12 dimensions had Cronbach's alphas higher than 0.7. The instrument as a whole achieved a high Cronbach's alpha (0.91). Inter-correlations showed that there is no dimension with redundant items, however dimension 10 increased its internal consistency when one item is removed. Originality/value – This study is the first to evaluate an American patient safety culture survey using Portuguese data. The survey has satisfactory reliability and construct validity.

Psychometric evaluation of the French version of the questionnaire attitudes towards morphine use; a cross-sectional study in Valais, Switzerland.

BACKGROUND: In Switzerland, nurses are allowed to prescribe and administer morphine in emergency situations without a doctor. Still, nurses and other health professionals are often reluctant to prescribe and administer morphine for pain management in patients. No valid French-speaking instrument is available in Switzerland to assess the attitudes of nurses and other health professionals towards the prescription and administration of morphine. In this study, we evaluated the psychometric properties of the French version of the questionnaire "Attitudes towards morphine use". METHODS: The instrument was derived from an Italian version. Forward and back translations of the questionnaire were performed. Item analysis and construct validity were assessed between April and December 2010 in a cross sectional study including five Swiss hospitals in a sample of 588 health professionals (533 nurses, mean age 38.3 ± 10.2 years). Thirty subjects participated in test-retest reliability. RESULTS: The time to complete the instrument ranged between 12 and 15 minutes and neither floor nor ceiling effect were found. The initial 24-item instrument showed an intraclass correlation (ICC) of 0.69 (95% CI: 0.64 to 0.73, P < 0.001), and a Cronbach's α of 0.700. Factor analysis led to a six-component solution explaining 52.4% of the total variance. After excluding five items, the shortened version showed an ICC of 0.74 (95% CI, 0.70 to 0.77, P < 0.001) and a Cronbach's α of 0.741. Factor analysis led to a five-component solution explaining 54.3% of the total variance. The five components were named "risk of addiction/dependence"; "operational reasons for not using morphine"; "risk of escalation"; "other (non-dependence) risks" and "external (non-operational) reasons". In test-retest, the shortened instrument showed an ICC of 0.797 (95% CI, 0.630 to 0.911, P < 0.001) and a Cronbach's α of 0.797. CONCLUSIONS: The 19-item shortened instrument assessing attitudes towards the prescription and administration of morphine showed adequate content and construct validity.

Identifying first graders at risk of reading and writing difficulties : creating a group-based screening tool in kiswahili in Tanzania

Early identification of beginning readers at risk of developing reading and writing difficulties plays an important role in the prevention and provision of appropriate intervention. In Tanzania, as in other countries, there are children in schools who are at risk of developing reading and writing difficulties. Many of these children complete school without being identified and without proper and relevant support. The main language in Tanzania is Kiswahili, a transparent language. Contextually relevant, reliable and valid instruments of identification are needed in Tanzanian schools. This study aimed at the construction and validation of a group-based screening instrument in the Kiswahili language for identifying beginning readers at risk of reading and writing difficulties. In studying the function of the test there was special interest in analyzing the explanatory power of certain contextual factors related to the home and school. Halfway through grade one, 337 children from four purposively selected primary schools in Morogoro municipality were screened with a group test consisting of 7 subscales measuring phonological awareness, word and letter knowledge and spelling. A questionnaire about background factors and the home and school environments related to literacy was also used. The schools were chosen based on performance status (i.e. high, good, average and low performing schools) in order to include variation. For validation, 64 children were chosen from the original sample to take an individual test measuring nonsense word reading, word reading, actual text reading, one-minute reading and writing. School marks from grade one and a follow-up test half way through grade two were also used for validation. The correlations between the results from the group test and the three measures used for validation were very high (.83-.95). Content validity of the group test was established by using items drawn from authorized text books for reading in grade one. Construct validity was analyzed through item analysis and principal component analysis. The difficulty level of most items in both the group test and the follow-up test was good. The items also discriminated well. Principal component analysis revealed one powerful latent dimension (initial literacy factor), accounting for 93% of the variance. This implies that it could be possible to use any set of the subtests of the group test for screening and prediction. The K-Means cluster analysis revealed four clusters: at-risk children, strugglers, readers and good readers. The main concern in this study was with the groups of at-risk children (24%) and strugglers (22%), who need the most assistance. The predictive validity of the group test was analyzed by correlating the measures from the two school years and by cross tabulating grade one and grade two clusters. All the correlations were positive and very high, and 94% of the at-risk children in grade two were already identified in the group test in grade one. The explanatory power of some of the home and school factors was very strong. The number of books at home accounted for 38% of the variance in reading and writing ability measured by the group test. Parents´ reading ability and the support children received at home for schoolwork were also influential factors. Among the studied school factors school attendance had the strongest explanatory power, accounting for 21% of the variance in reading and writing ability. Having been in nursery school was also of importance. Based on the findings in the study a short version of the group test was created. It is suggested for use in the screening processes in grade one aiming at identifying children at risk of reading and writing difficulties in the Tanzanian context. Suggestions for further research as well as for actions for improving the literacy skills of Tanzanian children are presented.

Psychometric Properties of the Inventário Dimensional Clínico da Personalidade (IDCP) using the Rating Scale Model

The aim of this study was to evaluate the performance of the Dimensional Clinical Personality Inventory (DCPI) using Rasch-based person and item analysis. 1281 participants were recruited, between 18 and 90 years of age (M=26.64; SD=8.94), 431 men (33.6%) and 127 (9.9%) patients diagnosed with axis I disorders and/or axis II according to DSM-IV-TR. Results indicated the IDCP scales performed reasonably well, and the usefulness of the analyses presented, demonstrate the Rasch model’s applicability for clinical applications. Among the important tools offered by the Rasch model, we explore the use of the person-item map, which visually presents the intuitively understandable psychological construct along the dimensional scale of the instrument.

Evaluación de la validez de la escala llanto para el dolor en neonatos y menores de cinco años

Introducción: La escala LLANTO para dolor es una escala que hasta la fecha ha sido solo validada en población infantil española, actualmente no se conocen datos en población colombiana. Se pretende validar la escala de dolor LLANTO en pacientes neonatos y menores de 5 años, a través de su aplicación en pacientes atendidos en una de tres instituciones, además comparándola con las escalas FLACC y PIPP dependiendo de edad del paciente. Metodología: Se incluyeron niños con cualquier tipo de dolor, clasificándolos en dos grupos por edad: 1) neonatos y 2) niños entre 1 mes y 5 años de edad, que asistieron a la Fundación Cardioinfantil, Clínica Infantil Colsubsidio o al Hospital Universitario Mayor. Las escalas fueron aplicadas por dos residentes de pediatría y una enfermera especializada en el cuidado de población infantil. Para la prueba piloto se diseñó un cuestionario determinar dificultades en la aplicación de la escala LLANTO. Una vez corregidos los problemas identificados se procederá a la validación de la escala. Resultados: Se presentan los datos de la prueba piloto. Se incluyeron 8 neonatos y 8 niños entre 1 mes y 5 años, esta muestra fue obtenida en un periodo de un mes, con la encuesta se evaluó la aceptación y entendimiento de la escala LLANTO por parte de los evaluadores. La prueba piloto mostró resultados favorables en el 100% de los encuestados. Discusión: Se considera que la escala LLANTO no requiere cambios para continuar con su validación.

A construção e validação de um teste com referência a critério para medir as competências básicas de leitura e escrita para o Programa de Alfabetização Funcional do MOBRAL

A necessidade de se desenvolverem instrumentos adequados a avaliação de programas educacionais é premente no momento atual. Segundo os estudos mais recentes no campo da medida educacional, os testes com referência a critério são os que atendem a esta necessidade. Este estudo teve por objetivo a construção de dois testes deste tipo para a avaliação do Programa de Alfabetização Funcional do MOBRAL nos dominios da aquisição das competências basicas em leitura e escrita. Inicialmente foram selecionadas e descritas quatro competências para serem medidas por cada teste. As competências foram selecionadas tendo como base os objetivos do programa apresentados no Roteiro de Orientação ao Alfabetizador (1977). A segunda etapa do estudo consistiu na criação das especificações para cada competência selecionadas nos dois dominios. A partir destas especificações foram construidos 32 itens para o teste de leitura e 40 para o de escrita. Estes itens foram, distribuidos randomicamente em 5 formas experimentais para cada teste. O teste de leitura é composto de itens de múltipla escolha e o teste de escrita de itens de resposta construida. Os criterios de correção e os padrões minimos de desempenho foram estabelecidos para cada teste. Para a analise destes padrões minimos foram calculados as medias e os desvios-padrão para cada forma aplicada. Estes resultados permitem a manutenção dos padrões minimos usados em estudos posteriores. A análise dos itens foi feita em duas etapas: na primeira foram calculados os indices de discriminância pelo processo tradicional e na segunda os indices de sensitivida de pelo processo desenvolvido por Gronlund (1976). Os resultados obtidos permitiram o aproveitamento de todos os itens. Os itens foram divididos em duas formas equivalentes para cada teste, cada um pode ser subdividido em três subtestes para a medida isolada das competências descritas. Para o exame de validade foram estabelecidos os indices de validade descritiva e de seleção de dominio. A fidedignidade dos testes ficou assegurada com a construção de mais de 5 itens para competência Gronlund (1976).

Refinement and initial validation of a multidimensional composite scale for use in assessing acute postoperative pain in cats

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Development of the Heat and Energy Concept Inventory: Preliminary Results on the Prevalence and Persistence of Engineering Students' Misconceptions

BACKGROUND Students frequently hold a number of misconceptions related to temperature, heat and energy. There is not currently a concept inventory with sufficiently high internal reliability to assess these concept areas for research purposes. Consequently, there is little data on the prevalence of these misconceptions amongst undergraduate engineering students. PURPOSE (HYPOTHESIS) This work presents the Heat and Energy Concept Inventory (HECI) to assess prevalent misconceptions related to: (1) Temperature vs. Energy, (2) Temperature vs. Perceptions of Hot and Cold, (3) Factors that affect the Rate vs. Amount of Heat Transfer and (4) Thermal Radiation. The HECI is also used to document the prevalence of misconceptions amongst undergraduate engineering students. DESIGN/METHOD Item analysis, guided by classical test theory, was used to refine individual questions on the HECI. The HECI was used in a one group, pre-test-post-test design to assess the prevalence and persistence of targeted misconceptions amongst a population of undergraduate engineering students at diverse institutions. RESULTS Internal consistency reliability was assessed using Kuder-Richardson Formula 20; values were 0.85 for the entire instrument and ranged from 0.59 to 0.76 for the four subcategories of the HECI. Student performance on the HECI went from 49.2% to 54.5% after instruction. Gains on each of the individual subscales of the HECI, while generally statistically significant, were similarly modest. CONCLUSIONS The HECI provides sufficiently high estimates of internal consistency reliability to be used as a research tool to assess students' understanding of the targeted concepts. Use of the instrument demonstrates that student misconceptions are both prevalent and resistant to change through standard instruction.

Testing the structural and cross-cultural validity of the KIDSCREEN-27 quality of life questionnaire

OBJECTIVES: The aim of this study is to assess the structural and cross-cultural validity of the KIDSCREEN-27 questionnaire. METHODS: The 27-item version of the KIDSCREEN instrument was derived from a longer 52-item version and was administered to young people aged 8-18 years in 13 European countries in a cross-sectional survey. Structural and cross-cultural validity were tested using multitrait multi-item analysis, exploratory and confirmatory factor analysis, and Rasch analyses. Zumbo's logistic regression method was applied to assess differential item functioning (DIF) across countries. Reliability was assessed using Cronbach's alpha. RESULTS: Responses were obtained from n = 22,827 respondents (response rate 68.9%). For the combined sample from all countries, exploratory factor analysis with procrustean rotations revealed a five-factor structure which explained 56.9% of the variance. Confirmatory factor analysis indicated an acceptable model fit (RMSEA = 0.068, CFI = 0.960). The unidimensionality of all dimensions was confirmed (INFIT: 0.81-1.15). Differential item functioning (DIF) results across the 13 countries showed that 5 items presented uniform DIF whereas 10 displayed non-uniform DIF. Reliability was acceptable (Cronbach's alpha = 0.78-0.84 for individual dimensions). CONCLUSIONS: There was substantial evidence for the cross-cultural equivalence of the KIDSCREEN-27 across the countries studied and the factor structure was highly replicable in individual countries. Further research is needed to correct scores based on DIF results. The KIDSCREEN-27 is a new short and promising tool for use in clinical and epidemiological studies.

Evaluation of an inventory for the assessment of critical incidents involving football fans

Introduction: Fan violence is a frequent occurrence in Swiss football (Bundesamt für Polizei, 2015) leading to high costs for prevention and control (Mensch & Maurer, 2014). Various theories put forward an explanation of fan violence, such as the Elaborated Social Identity Model (Drury & Reicher, 2000)and the Aggravation Mitigation Model (Hylander & Guvå, 2010). Important observations from these theories are the multi-dimensional understanding of fan violence and the Dynamics occurring in the fan group. Nevertheless, none of them deal with critical incidents (CIs) which involve a tense atmosphere combined with a higher risk of fan violence. Schumacher Dimech, Brechbühl and Seiler (2015) tackled this gap in research and explored CIs where 43 defining criteria were identified and compiled in an integrated model of CIs. The defining criteria were categorised in four higher-order themes “antecedents” (e.g. a documented history of fan rivalry), “triggers” (e.g. the arrest of a fan), “reactions” (e.g. fans masking themselves) and “consequences” (e.g. fans avoiding communication with fan social workers). Methods: An inventory based on this model is being developed including these 43 criteria. In an exploratory phase, this inventory was presented as an online questionnaire and was completed by 143 individuals. Three main questions are examined: Firstly, the individual items are tested using descriptive analyses. An item analysis is conducted to test reliability, item difficulty and discriminatory power. Secondly, the model’s four higher-order themes are tested using exploratory factor analysis (EFA). Thirdly, differences between sub -groups are explored, such as gender and age-related differences. Results: Respondents rated the items’ importance as high and the quota of incomplete responses was not systematic. Two items were removed from the inventory because of low mean or a high rate of “don’t know”-responses. EFA produced a six-factor solution grouping items into match-related factors, repressive measures, fans’ delinquent behaviour, intra-group behaviour, communication and control and inter-group factors. The item “fans consume alcohol” could not be ordered into any category but was retained since literature accentuates this factor’s influence on fan violence. Analyses examining possible differences between groups are underway. Discussion: Results exploring the adequacy of this inventory assessing defining criteria of CIs in football are promising and thus further evaluative investigation is recommended. This inventory can be used in two ways: as a standardised instrument of assessment for experts evaluating specific CIs and as an instrument for exploring differences in perception and assessment of a CI e.g. gender and age differences, differences between interest groups and stakeholders.

Evaluation of an inventory for the assessment of critical incidents involving football fans

Introduction: Fan violence is a frequent occurrence in Swiss football (Bundesamt für Polizei, 2015) leading to high costs for prevention and control (Mensch & Maurer, 2014). Various theories put forward an explanation of fan violence, such as the Elaborated Social Identity Model (Drury & Reicher, 2000) and the Aggravation Mitigation Model (Hylander & Guvå, 2010). Important observations from these theories are the multi-dimensional understanding of fan violence and the dynamics occurring in the fan group. Nevertheless, none of them deal with critical incidents (CIs) which involve a tense atmosphere combined with a higher risk of fan violence. Schumacher Dimech, Brechbühl and Seiler (2015) tackled this gap in research and explored CIs where 43 defining criteria were identified and compiled in an integrated model of CIs. The defining criteria were categorised in four higher-order themes “antecedents” (e.g. a documented history of fan rivalry), “triggers” (e.g. the arrest of a fan), “reactions” (e.g. fans masking themselves) and “consequences” (e.g. fans avoiding communication with fan social workers). Methods: An inventory based on this model is being developed including these 43 criteria. In an exploratory phase, this inventory was presented as an online questionnaire and was completed by 143 individuals. Three main questions are examined: Firstly, the individual items are tested using descriptive analyses. An item analysis is conducted to test reliability, item difficulty and discriminatory power. Secondly, the model’s four higher-order themes are tested using exploratory factor analysis (EFA). Thirdly, differences between sub-groups are explored, such as gender and agerelated differences. Results: Respondents rated the items’ importance as high and the quota of incomplete responses was not systematic. Two items were removed from the inventory because of low mean or a high rate of “don’t know”-responses. EFA produced a six-factor solution grouping items into match-related factors, repressive measures, fans’ delinquent behaviour, intra-group behaviour, communication and control and inter-group factors. The item “fans consume alcohol” could not be ordered into any category but was retained since literature accentuates this factor’s influence on fan violence. Analyses examining possible differences between groups are underway. Discussion: Results exploring the adequacy of this inventory assessing defining criteria of CIs in football are promising and thus further evaluative investigation is recommended. This inventory can be used in two ways: as a standardised instrument of assessment for experts evaluating specific CIs and as an instrument for exploring differences in perception and assessment of a CI e.g. gender and age differences, differences between interest groups and stakeholders. References: Bundesamt für Polizei. (2015). Jahresbericht 2014. Kriminalitätsbekämpfung Bund. Lage, Massnahmen und Mittel [Electronic Version]. Drury, J., & Reicher, S. (2000). Collective action and psychological change. The emergence of new social identities. British Journal of Social Psychology, 39, 579-604. Hylander, I., & Guvå, G. (2010). Misunderstanding of out-group behaviour: Different interpretations of the same crowd events among police officers and demonstrators. Nordic Psychology, 62, 25-47. Schumacher-Dimech, A., Brechbühl, A. &, Seiler, R. (2016). Dynamics of critical incidents with potentially violent outcomes involving ultra fans: an explorative study. Sport in Society. Advance online publication. doi: 10.1080/17430437.2015.1133597

THE MEASUREMENT OF HUMANISTIC AND DEHUMANISTIC BEHAVIORS IN HOSPITAL NURSING ENVIRONMENTS

The purpose of this investigation was to develop a reliable scale to measure the social environment of hospital nursing units according to the degree of humanistic and dehumanistic behaviors as perceived by nursing staff in hospitals. The study was based on a conceptual model proposed by Jan Howard, a sociologist. After reviewing the literature relevant to personalization of care, analyzing interviews with patients in various settings, and studying biological, psychological, and sociological frames of reference, Howard proposed the following necessary conditions for humanized health care. They were the dimensions of Irreplaceability, Holistic Selves, Freedom of Action, Status Equality, Shared Decision Making and Responsibility, Empathy, and Positive Affect.^ It was proposed that a scale composed of behaviors which reflected Howard's dimensions be developed within the framework of the social environment of nursing care units in hospitals. Nursing units were chosen because hospitals are traditionally organized around nursing care units and because patients spend the majority of their time in hospitals interacting with various levels of nursing personnel.^ Approximately 180 behaviors describing both patient and nursing staff behaviors which occur on nursing units were developed. Behaviors which were believed to be humanistic as well as dehumanistic were included. The items were classified under the dimensions of Howard's model by a purposively selected sample of 42 nurses representing a broad range of education, experience, and clinical areas. Those items with a high degree of agreement, at least 50%, were placed in the questionnaire. The questionnaire consisted of 169 items including six items from the Marlowe Crowne Social Desirability Scale (Short Form).^ The questionnaire, the Social Environment Scale, was distributed to the entire 7 to 3 shift nursing staff (603) of four hospitals including a public county specialty hospital, a public county general and acute hospital, a large university affiliated hospital with all services, and a small general community hospital. Staff were asked to report on a Likert type scale how often the listed behaviors occurred on their units. Three hundred and sixteen respondents (52% of the population) participated in the study.^ An item analysis was done in which each item was examined in relationship to its correlation to its own dimension total and to the totals of the other dimensions. As a result of this analysis, three dimensions, Positive Affect, Irreplaceability, and Freedom of Action were deleted from the scale. The final scale consisted of 70 items with 26 in Shared Decision Making and Responsibility, 25 in Holistic Selves, 12 in Status Equality, and seven in Empathy. The alpha coefficient was over .800 for all scales except Empathy which was .597.^ An analysis of variance by hospital was performed on the means of each dimension of the scale. There was a statistically significant difference between hospitals with a trend for the public hospitals to score lower on the scale than the university or community hospitals. That the scale scores should be lower in crowded, understaffed public hospitals was not unexpected and reflected that the scale had some discriminating ability. These differences were still observed after adjusting for the effect of Social Desirability.^ In summary, there is preliminary evidence based on this exploratory investigation that a reliable scale based on at least four dimensions from Howard's model could be developed to measure the concept of humanistic health care in hospital settings. ^