Validation of the Item-Attribute Matrix in TIMSS-Mathematics Using Multiple Regression and the LSDM

The purposes of this study were (1) to validate of the item-attribute matrix using two levels of attributes (Level 1 attributes and Level 2 sub-attributes), and (2) through retrofitting the diagnostic models to the mathematics test of the Trends in International Mathematics and Science Study (TIMSS), to evaluate the construct validity of TIMSS mathematics assessment by comparing the results of two assessment booklets. Item data were extracted from Booklets 2 and 3 for the 8th grade in TIMSS 2007, which included a total of 49 mathematics items and every student's response to every item. The study developed three categories of attributes at two levels: content, cognitive process (TIMSS or new), and comprehensive cognitive process (or IT) based on the TIMSS assessment framework, cognitive procedures, and item type. At level one, there were 4 content attributes (number, algebra, geometry, and data and chance), 3 TIMSS process attributes (knowing, applying, and reasoning), and 4 new process attributes (identifying, computing, judging, and reasoning). At level two, the level 1 attributes were further divided into 32 sub-attributes. There was only one level of IT attributes (multiple steps/responses, complexity, and constructed-response). Twelve Q-matrices (4 originally specified, 4 random, and 4 revised) were investigated with eleven Q-matrix models (QM1 ~ QM11) using multiple regression and the least squares distance method (LSDM). Comprehensive analyses indicated that the proposed Q-matrices explained most of the variance in item difficulty (i.e., 64% to 81%). The cognitive process attributes contributed to the item difficulties more than the content attributes, and the IT attributes contributed much more than both the content and process attributes. The new retrofitted process attributes explained the items better than the TIMSS process attributes. Results generated from the level 1 attributes and the level 2 attributes were consistent. Most attributes could be used to recover students' performance, but some attributes' probabilities showed unreasonable patterns. The analysis approaches could not demonstrate if the same construct validity was supported across booklets. The proposed attributes and Q-matrices explained the items of Booklet 2 better than the items of Booklet 3. The specified Q-matrices explained the items better than the random Q-matrices.

The Health Behavior Change Competency Checklist (HBCCC): Instrument development and psychometric evaluation.

More than half of morbidity and mortality in the United States can be attributed to behavior-related disease, such as tobacco use, physical inactivity, poor diet and alcohol consumption. Given the increased prevalence of behaviorally related medical health concerns, physician competence in the implementation of effective behavior change strategies is quickly becoming an essential skill. However, only recently have primary care residency programs begun to systematically teach and evaluate motivational interviewing skills critical to influencing health behavior, and use of standardized, objective assessment tools to assess skillfulness has been largely absent. This paper reports the development of a checklist, the Health Behavior Change Competency Checklist (HBCCC). The instrument captures the theoretical model of behavior change, motivational interviewing, in a practical and versatile manner. Psychometric evaluation demonstrated moderate efficacy. Namely, results indicated the HBCCC possesses good reliability, as evidenced by high internal consistency, and adequate construct validity. It also displayed considerable utility and practical application. While these results provide several reasons for confidence in the HBCCC, item revision and additional testing are required in order to establish it as a meaningful and valuable instrument.

Development of the DORI-TBI: An Investigation to Develop a New Screening Measure to Determine Consultation with a School-Based Brain Injury Resource Team

Brain injury is the leading cause of disability and death in children in the United States. Student re-entry into the school setting following a traumatic brain injury is crucial to student success. Multidisciplinary teams within the school district comprised of individuals with expertise in brain injury are ideal in implementing student specific treatment plans given their specialized training and wide range of expertise addressing student needs. Therefore, the purpose of this study is to develop and initially validate a quantitative instrument that school personnel can use to determine if a student, identified as having a traumatic brain injury, will benefit from district-level consultation from a brain injury team. Three studies were designed to investigate the research questions. In study one, the planning and construction of the DORI-TBI was completed. Study two addressed the content validity of the DORI-TBI through a comparison analysis with other referral forms, content review with experts in the field of TBI, and cognitive interviews with professionals to test the usability of the new screening tool. In study three, a field administration was conducted using vignettes to measure construct validity. Results produced a valid and reliable new screening instrument that can aid school-based teams to more efficiently utilize district level consultation with a brain injury support team.

Psychometric Investigation of the Check-In, Check-Out Fidelity of Implementation Measure

Suspension and expulsion are utilized frequently and disproportionality in schools in the United States. Many schools utilize Positive Behavioral Interventions and Supports (PBIS), a tiered framework to prevent problem behavior and reduce the use of discipline practices (Sugai et al., 2000). Check-In, Check-Out (CICO) is a targeted group behavioral intervention that is utilized within this framework in schools to prevent severe problem behavior in students that are beginning to exhibit externalizing and/or internalizing behavioral needs; thus, preventing the use of exclusionary discipline practices (Crone et al., 2010; Hawken & Horner, 2003). As the use of CICO in schools continues to grow, so too does the need for an instrument measuring its fidelity of implementation. The purpose of this study was to investigate the reliability and validity of the Check-In, Check-Out Fidelity of Implementation Measure (Crone et al., 2010), an instrument created to measure the fidelity of implementation of CICO intervention. This study assessed the psychometric properties of the instrument utilizing an archival data set collected by the statewide PBIS initiative in a western state in the U.S. The results demonstrated promising content validity, construct validity, internal consistency, and interrater reliability. A unidimensional structure was determined to be the best structure for the instrument based on parsimony and the strong results obtained from the item loadings, internal consistency, and interrater reliability. Implications for use and future research are discussed.

Validez de constructo de tres escalas de satisfacción del paciente mediante la estrategia de matrices multirrasgo-multimétodo

Se emplea el diseño de las matrices multirrasgo-multimétodo (MTMM) en la evaluación de la satisfacción del paciente. La muestra, extraída al azar simple, fue de 254 pacientes ingresados en tres hospitales del Servei Valencià de Salut de la provincia de Alicante, mayores de 16 años, conscientes y orientados. Los instrumentos de medida fueron tres escalas de satisfacción, dos de carácter general y una específica con los cuidados de enfermería, todas autoinformes. Los rasgos evaluados fueron varias dimensiones de satisfacción, y los métodos tres tipos de formulación de items y escalas de respuesta. Se ha empleado el análisis factorial confirmatorio, siguiéndose la estrategia de constrastar varios modelos alternativos (Widaman, 1985; Marsh, 1989). Los resulta dos indican que: la varianza de método es elevada, superior a la de rasgos; existe validez convergente; los rasgos están altamente correlacionados, pero hay evidencia de validez discriminante; dos métodos están altamente correlacionados; y no se ha podido estimar el modelo general de matrices MTMM.

Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: the eDESDE-LTC study

Background: The harmonization of European health systems brings with it a need for tools to allow the standardized collection of information about medical care. A common coding system and standards for the description of services are needed to allow local data to be incorporated into evidence-informed policy, and to permit equity and mobility to be assessed. The aim of this project has been to design such a classification and a related tool for the coding of services for Long Term Care (DESDE-LTC), based on the European Service Mapping Schedule (ESMS). Methods: The development of DESDE-LTC followed an iterative process using nominal groups in 6 European countries. 54 researchers and stakeholders in health and social services contributed to this process. In order to classify services, we use the minimal organization unit or “Basic Stable Input of Care” (BSIC), coded by its principal function or “Main Type of Care” (MTC). The evaluation of the tool included an analysis of feasibility, consistency, ontology, inter-rater reliability, Boolean Factor Analysis, and a preliminary impact analysis (screening, scoping and appraisal). Results: DESDE-LTC includes an alpha-numerical coding system, a glossary and an assessment instrument for mapping and counting LTC. It shows high feasibility, consistency, inter-rater reliability and face, content and construct validity. DESDE-LTC is ontologically consistent. It is regarded by experts as useful and relevant for evidence-informed decision making. Conclusion: DESDE-LTC contributes to establishing a common terminology, taxonomy and coding of LTC services in a European context, and a standard procedure for data collection and international comparison.

Psychometric properties of the Spanish adaptation of the School Attitude Assessment Survey-Revised

Background: The assessment of attitudes toward school with the objective of identifying adolescents who may be at risk of underachievement has become an important area of research in educational psychology, although few specific tools for their evaluation have been designed to date. One of the instruments available is the School Attitude Assessment Survey-Revised (SAAS-R). Method: The objective of the current research is to test the construct validity and to analyze the psychometric properties of the Spanish version of the SAAS-R. Data were collected from 1,398 students attending different high schools. Students completed the SAAS-R along with measures of the g factor, and academic achievement was obtained from school records. Results: Confirmatory factor analysis, multivariate analysis of variance and analysis of variance tests supported the validity evidence. Conclusions: The results indicate that the Spanish version of the SAAS-R is a useful measure that contributes to identification of underachieving students. Lastly, the results obtained and their implications for education are discussed.

Development and psychometric testing of the clinical learning environment, supervision and nurse teacher evaluation scale (CLES+T): The Spanish version

Background: The Clinical Learning Environment, Supervision and Nurse Teacher scale is a reliable and valid instrument to evaluate the quality of the clinical learning process in international nursing education contexts. Objectives: This paper reports the development and psychometric testing of the Spanish version of the Clinical Learning Environment, Supervision and Nurse Teacher scale. Design: Cross-sectional validation study of the scale. Setting: 10 public and private hospitals in the Alicante area, and the Faculty of Health Sciences (University of Alicante, Spain). Participants: 370 student nurses on clinical placement (January 2011–March 2012). Methods: The Clinical Learning Environment, Supervision and Nurse Teacher scale was translated using the modified direct translation method. Statistical analyses were performed using PASW Statistics 18 and AMOS 18.0.0 software. A multivariate analysis was conducted in order to assess construct validity. Cronbach’s alpha coefficient was used to evaluate instrument reliability. Results: An exploratory factorial analysis identified the five dimensions from the original version, and explained 66.4% of the variance. Confirmatory factor analysis supported the factor structure of the Spanish version of the instrument. Cronbach’s alpha coefficient for the scale was .95, ranging from .80 to .97 for the subscales. Conclusion: This version of the Clinical Learning Environment, Supervision and Nurse Teacher scale instrument showed acceptable psychometric properties for use as an assessment scale in Spanish-speaking countries.

Validación de la versión española del cuestionario Physician Readiness To Manage Intimate Partner Violence Survey (PREMIS)

Fundamentos: El Physician Readiness to Manage Intimate Partner Violence Survey (PREMIS) es uno de los cuestionarios más completos en el contexto internacional para la valoración de la capacidad de respuesta frente a la violencia del compañero íntimo por los profesionales de Atención Primaria de Salud. El objetivo de este estudio fue determinar la fiabilidad, consistencia interna y validez de constructo de la versión española de este cuestionario. Métodos: Tras la traducción, retrotraducción y valoración de la validez de contenido del cuestionario, se distribuyeron en una muestra de 200 profesionales de medicina y enfermería de 15 centros de atención primaria de 4 Comunidades Autónomas en 2013 (Comunidad Valenciana, Castilla León, Murcia y Cantabria). Se calcularon los coeficientes alfa de Cronbach, los de correlación intraclase y rho de Spearman (test-retest). Resultados: la versión española del PREMIS incluyó 64 ítems. El coeficiente α de Cronbach fue superior a 0,7 o muy cercano a ese valor en la mayoría de los índices. Se obtuvo un coeficiente de correlación intraclase de 0,87 y un coeficiente de Spearman de 0,67 que muestran una fiabilidad alta. Todas las correlaciones observadas para la escala de opiniones, la única tratada como estructura factorial en el cuestionario PREMIS, fueron superiores a 0,30. Conclusiones: el PREMIS en español obtuvo una buena validez interna, alta fiabilidad y capacidad predictiva de las prácticas auto-referidas por médicos(as) y enfermeros(as) frente a casos de violencia del compañero íntimo en centros de atención primaria.

Estimation of the Proportion of Underachieving Students in Compulsory Secondary Education in Spain: An Application of the Rasch Model

There are very few studies in Spain that treat underachievement rigorously, and those that do are typically related to gifted students. The present study examined the proportion of underachieving students using the Rasch measurement model. A sample of 643 first-year high school students (mean age = 12.09; SD = 0.47) from 8 schools in the province of Alicante (Spain) completed the Battery of Differential and General Skills (Badyg), and these students' General Points Average (GPAs) were recovered by teachers. Dichotomous and Partial credit Rasch models were performed. After adjusting the measurement instruments, the individual underachievement index provided a total sample of 181 underachieving students, or 28.14% of the total sample across the ability levels. This study confirms that the Rasch measurement model can accurately estimate the construct validity of both the intelligence test and the academic grades for the calculation of underachieving students. Furthermore, the present study constitutes a pioneer framework for the estimation of the prevalence of underachievement in Spain.

Construcción de un instrumento para medir la utilidad percibida de las TIC usadas en la docencia por estudiantes de Arquitectura Técnica

Se ha realizado una investigación acerca del uso del ordenador y de los objetos de aprendizaje que utilizan los estudiantes en una asignatura de Arquitectura Técnica de la Universidad de Alicante. Para ello, se ha creado un instrumento que analiza la utilidad percibida de los objetos de aprendizaje en la adquisición de las competencias y las actitudes de los estudiantes hacia el uso del ordenador. Los análisis realizados indican que el instrumento de medición elaborado es fiable y válido. La validez de contenido del instrumento se analizó a través del juicio de expertos (validez general del cuestionario = .912, p-valor = .000). La validez de constructo se estudió a través del análisis de su estructura interna, sometiendo a un análisis factorial los ítems de la versión definitiva del cuestionario (se identificaron cuatro factores que juntos explicaron el 45.65% de la varianza). La fiabilidad del instrumento se analizó calculando su consistencia interna por medio del coeficiente alpha de Cronbach (? para el total de la escala = .90).

Gray's model of personality and aggregate level factor analysis

Previous research shows that correlations tend to increase in magnitude when individuals are aggregated across groups. This suggests that uncorrelated constellations of personality variables (such as the primary scales of Extraversion and Neuroticism) may display much higher correlations in aggregate factor analysis. We hypothesize and report that individual level factor analysis can be explained in terms of Giant Three (or Big Five) descriptions of personality, whereas aggregate level factor analysis can be explained in terms of Gray's physiological based model. Although alternative interpretations exist, aggregate level factor analysis may correctly identify the basis of an individual's personality as a result of better reliability of measures due to aggregation. We discuss the implications of this form of analysis in terms of construct validity, personality theory, and its applicability in general. Copyright (C) 2003 John Wiley Sons, Ltd.

The sense of belonging to a neighbourhood: can it be measured and is it related to health and well being in older women?

This study investigates the sense of belonging to a neighbourhood among 9445 women aged 73-78 years participating in the Australian Longitudinal Study on Women's Health. Thirteen items designed to measure sense of neighbourhood were included in the survey of the older women in 1999. Survey data provided a range of measures of demographic, social and health-related factors to assess scale construct validity. Factor analysis showed that seven of the items loaded on one factor that had good face validity and construct validity as a measure of the sense of neighbourhood. Two of the remaining items related to neighbourhood safety and comprised a factor. A better sense of neighbourhood was associated with better physical and mental health, lower stress, better social support and being physically active. Women who had lived longer at their present address had a better sense of belonging to their neighbourhood, as did women living in non-urban areas and who were better able to manage on their income. Feeling safe in the neighbourhood was least likely in urban areas, increased in rural townships, and was most likely in rural and remote areas. Older women living alone felt less safe, as did women who were less able to manage on their income. This study has identified two sets of items that form valid measures of aspects of the social environment of older women, namely the sense of neighbourhood and feelings of safety. These findings make a contribution to our understanding of the relationship between feelings of belonging to a neighbourhood and health in older women. (C) 2004 Elsevier Ltd. All rights reserved.

The role of drinking restraint in alcohol dependence: validation of the temptation and restraint inventory in an alcohol dependent sample

Objective: The Temptation and Restraint Inventory (TRI) is commonly used to measure drinking restraint in relation to problem drinking behavior. However, as yet the TRI has not been validated in a clinical group with alcohol dependence. Method: Male (n = 111) and female (n = 57) inpatients with DSM-IV diagnosed alcohol dependence completed the TRI and measures of problem drinking severity, including the Alcohol Dependence Scale and the quantity, frequency and week total of alcohol consumed. Results: The factor structure of the TRI was replicated in the alcohol dependent sample. Cognitive Emotional Preoccupation (CEP), one of the two higher order factors of the TRI, demonstrated sound predictive power toward all dependence severity indices. The other higher order factor, Cognitive Behavioral Control (CBC), was related to frequency of drinking. There was limited support for the CEP/CBC interactional model of drinking restraint. Conclusions: Although the construct validity of the TRI was sound, the measure appears more useful in understanding the development, maintenance and severity of alcohol-related problems in nondependent drinkers. The TRI may show promise in detecting either continuous drinking or heavy episodic type dependent drinkers.

The Short-EMBU in Australia, Spain, and Venezuela - Factorial invariance, and associations with sex roles, self-esteem, and Eysenckian personality dimensions

The short(s)-EMBU (Swedish acronym for Egna Minnen Betraffande Uppfostran [My memories of upbringing]) consists of 23 items, is based on the early 81-item EMBU, and was developed out of the necessity of having a brief measure of perceived parental rearing practices when the clinical and/or research context does not adequately permit application of time-consuming test batteries. The s-EMBU comprises three subscales: Rejection., Emotional Warmth, and (Over)Protection. The factorial and/or construct validity and reliability of the s-EMBU were examined in samples comprising a total of 1950 students from Australia, Spain, and Venezuela. The data were presented for the three national groups separately. Findings confirmed the cross-national validity of the factorial structure underlying the s-EMBU. Rejection by fathers and mothers was consistently associated with high trait-neuroticism and low self-esteem in recipients of both sexes in each nation, as was high parental emotional warmth with high femininity (humility). The findings on factorial validity are in keeping with previous ones obtained in East Germany, Greece, Guatemala, Hungary, Italy, and Sweden. The s-EMBU is again recommended for use in several different countries as. a reliable, functional equivalent to the original 81-item EMBU.