The challenge of validation: assessing the performance of a test of productive vocabulary

This paper assesses the performance of a vocabulary test designed to measure second language productive vocabulary knowledge.The test, Lex30, uses a word association task to elicit vocabulary, and uses word frequency data to measure the vocabulary produced. Here we report firstly on the reliability of the test as measured by a test-retest study, a parallel test forms experiment and an internal consistency measure. We then investigate the construct validity of the test by looking at changes in test performance over time, analyses of correlations with scores on similar tests, and comparison of spoken and written test performance. Last, we examine the theoretical bases of the two main test components: eliciting vocabulary and measuring vocabulary. Interpretations of our findings are discussed in the context of test validation research literature. We conclude that the findings reported here present a robust argument for the validity of the test as a research tool, and encourage further investigation of its validity in an instructional context

Swedish translation and psychometric testing of the safety attitudes questionnaire (operating room version)

Background: Tens of millions of patients worldwide suffer from avoidable disabling injuries and death every year. Measuring the safety climate in health care is an important step in improving patient safety. The most commonly used instrument to measure safety climate is the Safety Attitudes Questionnaire (SAQ). The aim of the present study was to establish the validity and reliability of the translated version of the SAQ. Methods: The SAQ was translated and adapted to the Swedish context. The survey was then carried out with 374 respondents in the operating room (OR) setting. Data was received from three hospitals, a total of 237 responses. Cronbach's alpha and confirmatory factor analysis (CFA) was used to evaluate the reliability and validity of the instrument. Results: The Cronbach's alpha values for each of the factors of the SAQ ranged between 0.59 and 0.83. The CFA and its goodness-of-fit indices (SRMR 0.055, RMSEA 0.043, CFI 0.98) showed good model fit. Intercorrelations between the factors safety climate, teamwork climate, job satisfaction, perceptions of management, and working conditions showed moderate to high correlation with each other. The factor stress recognition had no significant correlation with teamwork climate, perception of management, or job satisfaction. Conclusions: Therefore, the Swedish translation and psychometric testing of the SAQ (OR version) has good construct validity. However, the reliability analysis suggested that some of the items need further refinement to establish sound internal consistency. As suggested by previous research, the SAQ is potentially a useful tool for evaluating safety climate. However, further psychometric testing is required with larger samples to establish the psychometric properties of the instrument for use in Sweden.

Teste Illinois de Habilidades Psicolinguísticas: analise critica do modelo mediacional e de diversos aspectos da validade do instrumento

Este trabalho investiga basicamente a validade do Teste Illinois de Habilidades Psicolinguísticas -ITPA -instrumento de avaliação do desenvolvimento da linguagem infantil. Seus autores, S. Kirk e J.J. McCarthy (1961), utilizam o referencial teórico proposto por C. Osgood (1957), a ele incorporando o modelo derivado da Teoria da Informação, o que permite que, na prática clínica, o ITPA possa ser incluído no processo psicodiagnóstico como instrumento de avaliação dos problemas da comunicação em crianças entre três e dez anos. Os objetivos que conduzem e orientam o trabalho apresentado podem ser definidos em três níveis: 1) O que trata dos constructos e suas interrelaç6es -análise crítica da validade teóricado ITPA; 2) O que avalia sua condição de instrumento diagnóstico do desempenho escolar -sensibilidade discriminante do rendimento acadêmico; 3) O que trata da eficácia da prática psicopedagógica proposta pelo mesmo instrumento. O estudo sobre a validade teórica foi realizado com 931 crianças entre três e dez anos de idade, em processo de escolarização, frequentando creches, jardins de infância ou classes regulares da Rede de Ensino do Primeiro Grau no Município do Rio de Janeiro. Utilizou-se a técnica da Análise Fatorial, complementada por uma abordagem lógica que comprovaram algumas das dimensões propostas pelo referencial teórico de Kirk e McCarthy. Para a validade diagnóstica foram avaliadas 71 crianças com dificuldades no desempenho acadêmico, expressas através de conceitos de insuficiência ou deficiência de rendimento e seus resultados foram comparados com os de um subgrupo, aleatoriamente constituído de crianças que participaram do estudo anterior. Utilizou-se a técnica da Análise Discriminante chegando-se à seguinte conclusão: embora a validade de constructo do IIPA não tenha sido completamente confirmada num nível diagnóstico os resultados permitem identificar, com baixa margem de erro, as crianças que pertencem a um ou outro dos grupos de contraste. Quanto ao terceiro nível, foi feita ampla-revelação bibliográfica sobre investigações efetuadas com este instrumento no Brasil e no Exterior. Visou-se avaliar a eficácia da prática psicopedagógica utilizada quando desenvolvida à luz dos recursos de intervenção que o IIPA propõe. Concluiu-se que as pesquisas, até o presente momento efetuadas, não são suficientes para formar um juízo mais seguro da praxis educativa destinada à reabilitação das crianças com problemas da comunicação - o que constitui impedimento a seu desempenho acadêmico - em função das controvérsias que tais pesquisas apresentam.

Construção do teste de aptidão criativa - TAC: forma experimental

Este trabalho tem por finalidade a construção de um instrumento psicológico de medida de comportamentos criativos, o Teste de Aptidão Criativa - TAC. A construção do TAC é justificada de um lado, por sua utilidade; sobretudo para o orientador profissional e psicólogo escolar e de outro; pela inexistência no Brasil; em particular no Nordeste, de instrumentos de medida similar. A formulação do problema e seu contexto são descritos logo no início do trabalho, bem como, a fundamentação teórica. Neste particular, enfatizou-se o enfoque psicológico da criatividade v em suas abordagens personológica e cognitiva. Atenção especial foi dada ao estudo do "modelo da estrutura do intelecto" de J. P. Guilford, suporte teórico do TAC. O teste de Aptidão Criativa é apresentado em todas as etapas de sua construção , desde a forma pré-piloto ç à piloto e à experimental, salientando-se seus respectivos subtestes figural e verbal. Ilustrações dos itens, bem como, das formas do teste são também apresentadas para melhor compreensão do texto. Em prosseguimento descreve- se uma pesquisa empírica realizada com o TAC, cujas hipóteses operacionais visam comprovar sua validade de constrito sua fidedignidade. A população foi constituída de alunos, de ambos os sexos, que cursaram em 1977 a 8a. série do 1º grau, de escolas públicas ou particulares; do município do Recife. O trabalho descreve detalhadamente a determinação da amostra, os instrumentos utilizados bem como os critérios de avaliação dos dados. O tratamento estatístico consistiu em medidas de tendência ceutra e de variabilidade; análise fatorial (técnica dos componentes principais-solução Varimax) para verificação da validade de construto. Os resultados obtidos, pela análise fatorial, demonstraram a presença de três fatores: um denominado C e interpretado como Convergente e dois, mensuráveis pelo TAC (conforme mostram as cargas fatoriais) que foram chamados de F (Fluência e Flexibilidade) e 0 Originalidade. O índice de homogeneidade calculado pelo “ de Cronbach" da característica originalidade entendida como essencial para a criatividade foi de 0,72. Tal resultado permite afirmar que na amostra testada o TAC , neste aspecto, apresenta fidedignidade significativa. Uma análise de regressão múltipla por passos foi levada a efeito, visando a identificação dos itens que mais contribuíram para explicação do escore total. Os resultados permitiram apontar uma melhor forma para o TAC figural constituída pelos itens I , II, V, VI , VII e VIII, os quais comporão a forma definitiva do TAC. Calculou-se também o estatístico Z do "Teste de urna Amostra de Kolmogorou para verificação da normalidade da distribuição das três características que o de medir: Fluência, Flexibilidade e Originalidade. Explicita-se ainda uma crítica à técnica de computação das respostas do TAC em face do índice de flexibilidade e propõe-se a realização de estudos, que permitam a formulação de novos procedimentos para um tratamento mais adequado das respostas aos estímulos do teste. Sugere-se também um estudo teórico sistemático de maior profundidade sobre os pontos de convergência entre as teorias psicológicas que tratam o tema para uma melhor compreensão da natureza e dinâmica do comportamento criativa. Propõe-se ainda a continuidade da pesquisa empírica que deu origem a este trabalho, tendo por objetivo imediato a padronização do TAC, de modo a poder ser ele instrumento útil ao psicólogo, na prática da psicologia aplicada.

Tradução, adaptação e validação da versão brasileira do questionário utian quality of life (uqol) para avaliação da qualidade de vida no climatério

The present study had as objective translates, to do equivalence and validation of the Utian Quality of Life (UQOL) for the Brazilian population through methods internationally accepted, in which the original questionnaire was translated for the Portuguese by three teachers and the consensual version was translated back for English by two American teachers (back translation). A multidisciplinary committee evaluated all versions and the final version in Portuguese was applied to climacteric women for the process of adaptation. Validation of the instrument was performed by measuring the reliability and validity properties. Construct validity was examined through the comparison between UQOL and the general measuring scale of quality of life Short Form-36 (SF-36). The final version of translation process was easily recognized by the target population, that didn't tell understanding problems. The results obtained for the reliability intra and interobserver showed significant agreement in all of the subjects. The construct validity was obtained through correlations statistically significant among the domains occupational, health and emotional of UQOL with the SF-36 domains. For the exploratory factorial analysis, it was verified that three factors explain 60% of the total variance of the data, the present study allowed concluding that UQOL was appropriately translated and adapted for applicability in Brazil, presenting high reliability and validity. In that way, the executed project provided the involvement of different areas as gynecology, psychology and physiotherapy (interdisciplinary). Thus, this instrument can be included and used in Brazilian studies to assessment the quality of life during the climacteric years

Validade e responsividade de uma escala multidimensional para avaliação de dor pós-operatória em gatos

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Number Processing and Calculation in Brazilian Children Aged 7-12 Years

Numerical cognition is based on two components - number processing and calculation. Its development is influenced by biological, cognitive, educational, and cultural factors. The objectives of the present study were to: i) assess number processing and calculation in Brazilian children aged 7-12 years from public schools using the Zareki-R (Battery of neuropsychological tests for number processing and calculation in children, Revised; von Aster & Dellatolas, 2006) in order to obtain normative data for Portuguese speakers; ii) identify how environment, age, and gender influences the development of these mathematical skills; iii) investigate the construct validity of the Zareki-R by the contrast with the Arithmetic subtest of WISC-III. The sample included 172 children, both genders, divided in two groups: urban (N= 119) and rural (N= 53) assessed by the Zareki-R. Rural children presented lower scores in one aspect of number processing; children aged 7-8 years demonstrated an inferior global score than older; boys presented a superior performance in both number processing and calculation. Construct validity of Zareki-R was demonstrated by high to moderate correlations with Arithmetic subtest of WISC-III. The Zareki-R therefore is a suitable instrument to assess the development of mathematical skills, which is influenced by factors such as environment, age, and gender.

Tradução, adaptação cultural e propriedades psicométricas da versão brasileira do Short-Form of the Mobility Assessment Tool (MAT-sf)

Objectives: to translate and assess the psychometric properties of the Brazilian short-form of the Mobility Assessment Tool (MAT-sf) in Brazilians community elderly. Methds: the 12 items MAT-sf went through the process of translation and back translation into Portuguese of the Brazil, considering the semantic and cultural adaptation. In a sample of 150 aged 65-74 years living in the community, the instrument's psychometric properties were evaluated by analysis of convergent and construct validity and test-retest reliability. The main outcomes measures used for validation included in the battery of tests of physical ability, self-repor measures of functional limitations, health, depression, cognitive and sex. The test-retest reliability of the instrument was assessed using the intra-class correlation coefficient (ICC), 40 subjects were reassessed after an interval of 14 days of assessment. Results: the concurrent validity for the MAT-sf was evidenced by significant correlations with SPPB (r = 0,53), number of functional limitations (r = -0,62) and depressive symptoms (r = -0,45). The construct validity of the instrument was measured by gradual and significant increase of the MAT-sf scores with high levels of physical performance and with positive self-reported health, also found that MAT-sf scores were statistically differents according to sex. The variation in MAT-sf scores (R2 = 0,41) was explained by SPPB, number of limitations for activities of daily life and depressive symptoms. High values for test-retest reliability was evidencend by ICC = 0,94, 95% CI = 0,90 0,97. Conclusions: the Brazilian version of the short-form of the Mobility Assessment Tool has values of validity and reliability to ensure its use in elderly populations living in communities

A new short and simple health-related quality of life measurement for paediatric rheumatic diseases: initial validation in juvenile idiopathic arthritis

Objective. To develop and validate a new short and simple measure of health-related quality of life (HRQL) in children with juvenile idiopathic arthritis (JIA).Methods. The Paediatric Rheumatology Quality of Life Scale (PRQL) is a 10-item questionnaire that explores HRQL in two domains: physical health (PhH) and psychosocial health (PsH). Validation of the parent proxy report and child self-report versions of the instrument was accomplished by evaluating 472 JIA patients and similar to 800 healthy children. Validation analyses included assessment of feasibility, face and content validity; construct and discriminative ability; internal structure and consistency; test-retest reliability; responsiveness to clinical change; and minimal clinically important difference.Results. The PRQL was found to be feasible and to possess both face and content validity. The PRQL score correlated in the predicted range with most of the other JIA outcome measures, thereby demonstrating good construct validity, and discriminated well between different levels of disease severity. Assessment of internal structure (factor analysis) revealed that the PhH and PsH subscales identify two unambiguously separated domains. The internal consistency (Cronbach's alpha) was 0.86. The intraclass correlation coefficient for test-retest reliability was 0.91. The PRQL revealed fair responsiveness, with a standardized response mean of 0.67 in improved patients. Overall, the PRQL appeared to be more able to capture physical HRQL than psychosocial HRQL.Conclusion. The PRQL was found to possess good measurement properties and is, therefore, a valid instrument for the assessment of HRQL in children with JIA. This tool is primarily proposed for use in standard clinical care.

Refinement and initial validation of a multidimensional composite scale for use in assessing acute postoperative pain in cats

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Validation of the Brazilian versions of two inventories for measuring oral health-related quality of life of edentulous subjects

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Validation of the English version of the UNESP-Botucatu multidimensional composite pain scale for assessing postoperative pain in cats

Background: A scale validated in one language is not automatically valid in another language or culture. The purpose of this study was to validate the English version of the UNESP-Botucatu multidimensional composite pain scale (MCPS) to assess postoperative pain in cats. The English version was developed using translation, back-translation, and review by individuals with expertise in feline pain management. In sequence, validity and reliability tests were performed.Results: Of the three domains identified by factor analysis, the internal consistency was excellent for 'pain expression' and 'psychomotor change' (0.86 and 0.87) but not for 'physiological variables' (0.28). Relevant changes in pain scores at clinically distinct time points (e.g., post-surgery, post-analgesic therapy), confirmed the construct validity and responsiveness (Wilcoxon test, p < 0.001). Favorable correlation with the IVAS scores (p < 0.001) and moderate to very good agreement between blinded observers and 'gold standard' evaluations, supported criterion validity. The cut-off point for rescue analgesia was > 7 (range 0-30 points) with 96.5% sensitivity and 99.5% specificity.Conclusions: The English version of the UNESP-Botucatu-MCPS is a valid, reliable and responsive instrument for assessing acute pain in cats undergoing ovariohysterectomy, when used by anesthesiologists or anesthesia technicians. The cut-off point for rescue analgesia provides an additional tool for guiding analgesic therapy. © 2013 Brondani et al.; licensee BioMed Central Ltd.

Versão brasileira do Chronic Respiratory Questionnaire: estudo da validade de constructo e reprodutibilidade

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Proposição de escala clínica para avaliação da dor em equinos

Pós-graduação em Medicina Veterinária - FMVZ

Translation, cross-cultural adaptation and validation of the Portuguese version of the DYMUS questionnaire for the assessment of dysphagia in multiple sclerosis

Translation, cross-cultural adaptation and validation of the DYMUS questionnaire for the assessment of dysphagia in multiple sclerosis. The original English version of the DYMUS was translated using the forward-backward technique, cross-culturally adaptated, pilot-tested in 40 patients, and then applied to 100 multiple sclerosis patients to assess the reliability and construct validity. Construct validity was assessed by Mann–Whitney test and Spearman’s correlation coefficient (rs). The internal consistency of the questionnaire was evaluated using Cronbach’s alpha coefficient and inter-item correlation. DYMUS-BR internal consistency was high (Cronbach’s alpha= 0.72); Cronbach’s alpha was 0.65 for the ‘dysphagia for solids’ subscale and 0.67 for the ‘dysphagia for liquids’ subscale and positive inter-item correlations was found between all items, except for weight loss question. Significant association (p<0.001) and correlation (rs = 0,357; p = 0.01) was found between DYMUS-BR and dysphagia self-assessment. The DYMUS-BR questionnaire maintained the characteristics of that originally described, demonstrating to be a reliable, valid, easy and consistent tool to be used by health professionals for preliminary selection of Brazilian MS patients who need more specific instrumental analyses of swallowing.