The inclusion of ‘then-test’ questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation

Objectives Program evaluations are frequently based on ‘then-test’ data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects’ responses to post-test.
Patients and methods To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
Results Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then test questions reporting significantly higher post-test scores than respondents of the other groups.
Conclusions This study provides evidence that the inclusion of then-test questions alters post-test responses,
suggesting that change scores based on then-test data be interpreted with care.

An evaluation of the Shanghai mental health service schizophrenia rehabilitation program

This study aimed to evaluate the effectiveness of a rehabilitation program for people with schizophrenia in Shanghai, China. Thirty-five people with schizophrenia participated in an eight-module program that focused on a range of psychosocial skills, while 38 others received treatment as usual in the community. Participants were assessed at baseline and subsequently at 4-week intervals over 12 weeks using the Positive and Negative Syndrome Scale, Clinical Global Impression Scale, Drug Attitude Inventory, and Personal and Social Performance Scale. The rehabilitation program participants demonstrated significantly better improvement over the course of the program than did the control participants on all measures. The rehabilitation program is effective in addressing psychosocial deficits evident in many people who have schizophrenia, and it should be implemented more widely.

Increasing the sensitivity of the AQoL inventory for the evaluation of interventions affecting mental health

This paper presents the rationale and psychometric analysis for extending the inventory of the Assessment of Quality of Life (AQoL)-6D instrument. The resulting AQoL-8D has an 8 dimensional, 35 item inventory with greater sensitivity in the domain of mental health.The paper briefly reviews the existing QoL instruments used for economic evaluation of health programs. It outlines the steps adopted in developing the AQoL descriptive inventories and, specifically, the methods adopted for data collection and analysis for the AQoL-8D inventory.Three instruments are presented. The first, PsyQoL, is a 22 item instrument which represents the best statistical fit for the measurement of mental health related quality of life. The second, PsyQoL-Brief is a reduced form instrument which is combined with AQoL-6D as the basis for the third instrument, the AQoL-8D. Psychometric properties of the first instrument are excellent and the second are good. The full AQoL-8D has satisfactory properties. Results from a comparison with the original AQoL-6D are reported. The mental health content of AQoL-8D is unique amongst MAU instruments and, along with other AQoL instruments, unique in its derivation from psychometric analysis. Its application to mental health patients and the public demonstrates its ability to discriminate between the groups with greater sensitivity than the previous AQoL-6D instrument.

Diamond Dialogue method for the evaluation of personal well-being after a maternal health intervention in Lao PDR

This paper follows the genesis of an evolving tool to measure the impact of development interventions on personal well-being in non-literate and non-numerate communities. The Diamond Dialogue was adapted to evaluate the impact on women of the establishment of 17 one stop maternity waiting home/health promotion/income generating centres (Silk Homes) in Southern Lao PDR. The tool elicited data not accessible by research methods which assume literacy and numeracy and revealed that 86% of women who used the Silk Homes experienced substantial improvements in quality of life. The Diamond Dialogue can be modified to assess the outcomes of development interventions in a myriad of contexts.

Adaptation and validation of the Japanese version of the health education impact questionnaire (heiQ-J) for the evaluation of self-management education interventions

Aim:  

Assessment of proximal outcomes of self-management programs : translation and psychometric evaluation of a German version of the Health Education Impact Questionnaire (heiQtm)

Purpose This paper describes the translation, cultural adaption, and psychometric evaluation of a German version of the Health Education Impact Questionnaire (heiQ™), a widely used generic instrument assessing a wide range of proximal outcomes of self-management programs.

Methods The translation was carried out according to international standards and included forward and backward translations. Comprehensibility and content validity were tested using cognitive interviews with 10 rehabilitation inpatients. Psychometric properties were examined in rehabilitation inpatients (n = 1,202) with a range of chronic conditions. Factorial validity was assessed using confirmatory factor analysis; concurrent validity was explored by correlations with comparator scales.

Results The items of the German heiQ™ were well understood by rehabilitation inpatients. The structure of the eight heiQ™ scales was replicated after minor adjustment. heiQ™ scales had higher correlations with comparator scales with similar constructs, particularly mental health concepts than with physical health. Moreover, all heiQ™ scales differentiated between individuals across different levels of depression.

Conclusion The German heiQ™ is comprehensible for German-speaking patients suffering from different types of chronic conditions; it assesses relevant outcomes of self-management programs in a reliable and valid manner. Further studies involving its practical application are warranted.

Process evaluation of an integrated health promotion/occupational health model in WellWorks-2

Disparities in chronic disease risk by occupation call for newapproaches to health promotion. Well Works-2 was a randomized, controlled study comparing the effectiveness of a health promotion/occupational health program (HP/OHS) with a standard intervention (HP). Interventions in both studies were based on the same theoretical foundations. Results from process evaluation revealed that a similar number of activities were offered in both conditions and that in the HP/OHS condition there were higher levels of worker participation using three measures: mean participation per activity (HP: 14.2% vs. HP/OHS: 21.2%), mean minutes of worker exposure to the intervention/site (HP: 14.9 vs. HP/OHS: 33.3), and overall mean participation per site (HP: 34.4% vs. HP/ OHS: 45.8%). There were a greater number of contacts with management (HP: 8.8 vs. HP/OHS: 24.9) in the HP/ OHS condition. Addressing occupational health may have contributed to higher levels of worker and management participation and smoking cessation among blue-collar workers.

Evaluation of a filmed clinical scenario as a teaching resource for an introductory pharmacology unit for undergraduate health students: A pilot study.

BACKGROUND: Simulation is frequently being used as a learning and teaching resource for both undergraduate and postgraduate students, however reporting of the effectiveness of simulation particularly within the pharmacology context is scant. OBJECTIVES: The aim of this pilot study was to evaluate a filmed simulated pharmacological clinical scenario as a teaching resource in an undergraduate pharmacological unit. DESIGN: Pilot cross-sectional quantitative survey. SETTING: An Australian university. PARTICIPANTS: 32 undergraduate students completing a healthcare degree including nursing, midwifery, clinical science, health science, naturopathy, and osteopathy. METHODS: As a part of an undergraduate online pharmacology unit, students were required to watch a filmed simulated pharmacological clinical scenario. To evaluate student learning, a measurement instrument developed from Bloom's cognitive domains (knowledge, comprehension, application, analysis, synthesis and evaluation) was employed to assess pharmacological knowledge conceptualisation and knowledge application within the following fields: medication errors; medication adverse effects; medication interactions; and, general pharmacology. RESULTS: The majority of participants were enrolled in an undergraduate nursing or midwifery programme (72%). Results demonstrated that the majority of nursing and midwifery students (56.52%) found the teaching resource complementary or more useful compared to a lecture although less so compared to a tutorial. Students' self-assessment of learning according to Bloom's cognitive domains indicated that the filmed scenario was a valuable learning tool. Analysis of variance indicated that health science students reported higher levels of learning compared to midwifery and nursing. CONCLUSION: Students' self-report of the learning benefits of a filmed simulated clinical scenario as a teaching resource suggest enhanced critical thinking skills and knowledge conceptualisation regarding pharmacology, in addition to being useful and complementary to other teaching and learning methods.