The inclusion of ‘then-test’ questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation

Objectives Program evaluations are frequently based on ‘then-test’ data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects’ responses to post-test.
Patients and methods To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
Results Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then test questions reporting significantly higher post-test scores than respondents of the other groups.
Conclusions This study provides evidence that the inclusion of then-test questions alters post-test responses,
suggesting that change scores based on then-test data be interpreted with care.

Using the Rose Angina Questionnaire cross-culturally : the importance of consulting lay people when translating epidemiological questionnaires

Objectives. The Rose Angina Questionnaire (RAQ) is an important measure of coronary heart disease prevalence. It has been shown to perform inconsistently across some ethnic groups in Britain. This study investigates whether the best available versions of the RAQ in Punjabi and Cantonese were linguistically equivalent to the English version.

Design. Interviews were carried out with lay people from the Pakistani, Chinese and European-origin communities in Scotland to assess the versions of the RAQ used in the Newcastle Heart Project (the best available versions). For each questionnaire item, participants were asked to elaborate on their understanding of the question and the meaning of keywords or phrases.

Results. Problems were discovered with the Punjabi and Cantonese translations of the RAQ. For example, the translation for ‘chest’ was interpreted by some Pakistani and Chinese women to mean ‘breasts’. ‘Walking uphill’ was translated in Chinese as ‘walking the hill’, without stipulation of the direction, so that some Cantonese speakers interpreted the question as pertaining to walking downhill. Many Chinese interpreted RAQ items to be referring to breathlessness rather than chest pain due to ambiguous wording.

Conclusion. Existing versions of the RAQ are unlikely to be yielding data that are cross-culturally valid or comparable. For robust health survey research in languages other than that in which the questionnaire was developed, lay assessment of questionnaires prior to and after translation is a necessity rather than a luxury.

Transition to motherhood in type 1 diabetes : design of the pregnancy and postnatal well-being in transition questionnaires

Background : Life transitions are associated with high levels of stress affecting health behaviours among people with Type 1 diabetes. Transition to motherhood is a major transition with potential complications accelerated by pregnancy with risks of adverse childbirth outcomes and added anxiety and worries about pregnancy outcomes. Further, preparing and going through pregnancy requires vigilant attention to a diabetes management regimen and detailed planning of everyday activities with added stress on women. Psychological and social well-being during and after pregnancy are integral for good pregnancy outcomes for both mother and baby. The aim of this study is to establish the face and content validity of two novel measures assessing the well-being of women with type 1 diabetes in their transition to motherhood, 1) during pregnancy and 2) during the postnatal period.

Methods : The approach to the development of the Pregnancy and Postnatal Well-being in T1DM Transition questionnaires was based on a four-stage pre-testing process; systematic overview of literature, items development, piloting testing of questionnaire and refinement of questionnaire. The questionnaire was reviewed at every stage by expert clinicians, researchers and representatives from consumer groups. The cognitive debriefing approach confirmed relevance of issues and identified additional items.

Results : The literature review and interviews identified three main areas impacting on the women’s postnatal self-management; (1) psychological well-being; (2) social environment, (3) physical (maternal and fetal) well-being. The cognitive debriefing in pilot testing of the questionnaire identified that immediate postnatal period was difficult, particularly when the women were breastfeeding and felt depressed.

Conclusions : The questionnaires fill an important gap by systematically assessing the psychosocial needs of women with type 1 diabetes during pregnancy and in the immediate postnatal period. The questionnaires can be used in larger data collection to establish psychometric properties. The questionnaires potentially play a key role in prospective research to determine the self-management and psychological needs of women with type 1 diabetes transitioning to motherhood and to evaluate health education interventions.

Internet administration of three commonly used questionnaires in panic research : equivalence to paper administration in Australian and Swedish samples of people with panic disorder

This study assessed the degree of equivalence between paper and Internet administration of three measures of panic and agoraphobia-related cognition and behavior: Body Sensations Questionnaire (BSQ), Agoraphobic Cognitions Questionnaire (ACQ), and Mobility Inventory (MI). Participants were 110 people with panic disorder who had registered for an Internet-based treatment program in Sweden (n = 54) or Australia (n = 56). Participants were randomly assigned to complete the questionnaires via the differing administration formats in a counterbalanced order. Results showed broadly equivalent psychometric properties across administrations, with strong significant intraclass correlations between them, and comparable Cronbach's alpha coefficients. A significant mean difference between administration formats was found for the BSQ only. In contrast to previous research, Internet administration did not generate higher scores than paper administration. No effect was found for order of administration. The findings suggest that each questionnaire can be validly administered via the Internet and used with confidence.

Reliability and vailidity of self-reported questionnaires related to adolescent violence and consequences, Thailand

In Thailand physical violence among male adolescents is considered a significant public health issue, although there has been little published research into the aetiology and functions of violence in Thai youth. Research in this area has been hampered by a lack of psychometrically sound tools that have been validated to assess problem behaviours in Asian youth. The purpose of this paper is to provide validity and reliability data on an instrument to measure violence in Thai youth. In this study, reliability and validity data for a sample of adolescent Thai youth are reported for the Communities That Care Youth Survey (CTC-YS), a measure of risk and protective factors for violent behaviour, and the STAXI-II, a measure of angry experience and expression. The findings showed overall high internal consistency for both questionnaires, and there was evidence of construct validity. It is concluded that these measures are appropriate for use in research that seeks to investigate youth violence among adolescents in Thailand.

User manual for the QUANTUM system: a system to exercise a step by step control over the receipt and key entry of survey questionnaires using a microcomputer, version 1.00

Manual del sistema QUANTUM desarrollado para controlar la recepción y entrega de cuestionarios de una encuesta, por microcomputador, usando dBASE III.

Estudo do "Ages and Stages Questionnaires" com cuidadores de crianças institucionalizadas

Este estudo explorou o conhecimento de cuidadoras sobre o desenvolvimento de crianças em acolhimento institucional com um instrumento de triagem. Participaram deste estudo quatro crianças na faixa etária de 5 anos de idade e as cuidadoras responsáveis. O instrumento utilizado foi o Ages and Stages Questionnaires, que contém 21 questionários que envolvem seis áreas de desenvolvimento. Os resultados revelaram que a comunicação foi uma das áreas pouco pontuadas pelas crianças. Suas principais dificuldades estão em verbalizar e se concentrar nas tarefas propostas. A área da coordenação motora ampla, que envolve, entre outras coisas, o correr e pular, incentivada pelo próprio ambiente da instituição, foi considerada dentro das expectativas para o desenvolvimento e recebeu pontuação máxima de acordo com o ASQ-3. As cuidadoras como pessoas de referência para as crianças, foram essenciais para aplicação do ASQ-3, que se mostrou sensível na identificação dos problemas do desenvolvimento.

Oral cleft research: intraobserver agreement in the use of questionnaires.

In epidemiological studies, when the data is collected by interview, it is of importance to analyse the reliability of the information. This study was carried out with the purpose of examining the self-agreement of mothers in answering questions about variables of interest in oral cleft researches on two different occasions. A sample of 91 mothers of oral cleft babies were interviewed on two different occasions. The capacity of giving the same answer to questions related to heredity, type of clefts, period of gestation of the mother and birthweight were studied. The statistics type kappa (k) and intraclass correlation coefficient (r) by point and by 95% of confidence interval were applied. The intra-observer agreement for the variables history of oral clefts in the family, type of cleft, period of gestation of the mother and birthweight of the newborn was, respectively, k=0.9492, k=1.0000, k=0.9281 and r=0.9996. We concluded that the background on oral cleft in the family history of patients with this anomaly is a variable with an excellent degree of reliability. Also, the information given by the mothers related to the period of gestation, type of the baby’s cleft and birthweight are reliable.

Cross-national differences in questionnaires do not necessarily reflect comparable differences in disorder prevalence

To examine whether the widely used Strengths and Difficulties Questionnaire (SDQ) can validly be used to compare the prevalence of child mental health problems cross nationally. We used data on 29,225 5- to 16-year olds in eight population-based studies from seven countries: Bangladesh, Brazil, Britain, India, Norway, Russia and Yemen. Parents completed the SDQ in all eight studies, teachers in seven studies and youth in five studies. We used these SDQ data to calculate three different sorts of "caseness indicators" based on (1) SDQ symptoms, (2) SDQ symptoms plus impact and (3) an overall respondent judgement of 'definite' or 'severe' difficulties. Respondents also completed structured diagnostic interviews including extensive open-ended questions (the Development and Well-Being Assessment, DAWBA). Diagnostic ratings were all carried out or supervised by the DAWBA's creator, working in conjunction with experienced local professionals. As judged by the DAWBA, the prevalence of any mental disorder ranged from 2.2% in India to 17.1% in Russia. The nine SDQ caseness indicators (three indicators times three informants) explained 8-56% of the cross-national variation in disorder prevalence. This was insufficient to make meaningful prevalence estimates since populations with a similar measured prevalence of disorder on the DAWBA showed large variations across the various SDQ caseness indicators. The relationship between SDQ caseness indicators and disorder rates varies substantially between populations: cross-national differences in SDQ indicators do not necessarily reflect comparable differences in disorder rates. More generally, considerable caution is required when interpreting cross-cultural comparisons of mental health, particularly when these rely on brief questionnaires.