Acceptability, impact and sustainability of the national pilot quality assurance system for self management programs

Chronic condition self-management education and training interventions such as the Stanford Self Management Programs (SMP) have the capacity to improve health and quality of life of people with chronic conditions whilst reducing the use of health services. This is in line with the outcomes from the recent Council of Australian Governments’ meeting where it was indicated that self-management will be a centrepiece in forthcoming chronic disease initiatives.
Aim: report on a large national pilot quality assurance program involving the implementation and of an evaluation and quality monitoring system for SMPs including the provision of structured feedback to courses course leaders and service providers. During 2005/06 the quality assurance program was implemented at 11 diverse organisations across Australia. The program involved assisting organisations apply the 42-item Health Education Impact Questionnaire (HEIQ), a chronic disease health education outcome measure, and then observe and evaluate the value and impact of the quality program. Interviews with course leaders (n=60) and course participants (n=35) have elicited views about course quality and feedback processes.
Results: The evaluation revealed enablers and barriers to effective implementation and sustainability. Important enablers were:
- Course Leaders and organisations valued an Australia-wide system that provided feedback on course
quality and the impact on participants.
- Course Leaders were strongly personally motivated to respond appropriately to HEI-Q course
report feedback.
- Completing the questionnaire provided participants with the opportunity to reflect on issues that
emerge in the course content and reflect on their progression at the end of the SSMP.
Sustainability issues included:
- Organisations and course leaders require support, training and flexibility on how to administer and
manage the use of the HEI-Q.
- Availability of administrative resources in organisations to support the quality assurance activities.
- The requirement that course leaders are trained in interpreting HEI-Q course report data.
A quality improvement framework was developed which identified the actions required of key stakeholders to
support effective implementation.
Discussion: With the increasing endorsement of SMP across sectors it is important that course quality is known, is acceptable, and is communicated to stakeholders to inform and engender confidence in the SSMP. To effectively implement and sustain a quality improvement program for SMP, the processes and tools for measuring outcomes need to be responsive, flexible and easily integrated into the organisation and delivery of programs.

HealthMap: a cluster randomised trial of interactive health plans and self-management support to prevent coronary heart disease in people with HIV

BACKGROUND: The leading causes of morbidity and mortality for people in high-income countries living with HIV are now non-AIDS malignancies, cardiovascular disease and other non-communicable diseases associated with ageing. This protocol describes the trial of HealthMap, a model of care for people with HIV (PWHIV) that includes use of an interactive shared health record and self-management support. The aims of the HealthMap trial are to evaluate engagement of PWHIV and healthcare providers with the model, and its effectiveness for reducing coronary heart disease risk, enhancing self-management, and improving mental health and quality of life of PWHIV.

METHODS/DESIGN: The study is a two-arm cluster randomised trial involving HIV clinical sites in several states in Australia. Doctors will be randomised to the HealthMap model (immediate arm) or to proceed with usual care (deferred arm). People with HIV whose doctors are randomised to the immediate arm receive 1) new opportunities to discuss their health status and goals with their HIV doctor using a HealthMap shared health record; 2) access to their own health record from home; 3) access to health coaching delivered by telephone and online; and 4) access to a peer moderated online group chat programme. Data will be collected from participating PWHIV (n = 710) at baseline, 6 months, and 12 months and from participating doctors (n = 60) at baseline and 12 months. The control arm will be offered the HealthMap intervention at the end of the trial. The primary study outcomes, measured at 12 months, are 1) 10-year risk of non-fatal acute myocardial infarction or coronary heart disease death as estimated by a Framingham Heart Study risk equation; and 2) Positive and Active Engagement in Life Scale from the Health Education Impact Questionnaire (heiQ).

DISCUSSION: The study will determine the viability and utility of a novel technology-supported model of care for maintaining the health and wellbeing of people with HIV. If shown to be effective, the HealthMap model may provide a generalisable, scalable and sustainable system for supporting the care needs of people with HIV, addressing issues of equity of access.

The extent and breadth of benefits from participating in chronic disease self-management courses: a national patient-reported outcomes survey

Objective
To quantify the benefits that people receive from participating in self-management courses and identify subgroups that benefit most.

Methods
People with a wide range of chronic conditions attending self-management courses (N = 1341 individuals) were administered the Health Education Impact Questionnaire (heiQ). Baseline and follow-up data were collected resulting in 842 complete responses. Outcomes were categorized as substantial improvement (effect size, ES ≥ 0.5), minimal/no change (ES −0.49 to 0.49) and substantial decline (ES ≤ −0.5).

Results
On average, one third of participants reported substantial benefits at the end of a course and this ranged from 49% in the heiQ subscale Skill and technique acquisition to 27% in the heiQ subscale Health service navigation. Stratification by gender, age and education showed that younger participants were more likely to benefit, particularly young women. No further subgroup differences were observed.

Conclusion
While the well-being of people with chronic diseases tends to decline, about one third of participants from a wide range of backgrounds show substantial improvements in a range of skills that enable them to self-manage.

Practice implications
These data support the application of self-management courses indicating that they are a useful adjunct to usual care for a modest proportion of attendees.

Ein Blick Down Under. Selbstmanagementinitiativen und Patientenschulungen in Australien

The growing burden of chronic disease and the current nature of healthcare systems which are structurally ill-equipped to cater to the complex needs of patients with chronic conditions has led to governments and healthcare providers seeking alternative ways to improve patients own capacity to actively self-manage their chronic condition. In Australia, there has been a focus on patient education and self-management programs within the healthcare system to achieve this. These programs aim to empower patients through providing information and teaching skills and techniques to improve self-care and doctor-patient interaction with the ultimate goal of improving quality of life. Patient education and self-management programs have been supported through several national government initiatives and implemented within the healthcare setting. This paper describes the current position of patient education and self-management within the Australian healthcare system. It further describes a new collaboration project between an Australian and a German research team which aims at translating an assessment questionnaire used in Australia for the evaluation of self-management programs, the “Health Education Impact Questionnaire” (“heiQ”); this instrument is expected to be of significant use in the German rehabilitative system.

Tests of measurement invariance failed to support the application of the "then-test"

Objective
The use of then-test (retrospective pre-test) scores has frequently been proposed as a solution to potential confounding of change scores because of response shift, as it is assumed that then-test and post-test responses are provided from the same perspective. However, this assumption has not been formally tested using robust quantitative methods. The aim of this study was to compare the psychometric performance of then-test/post-test with traditional pre-test/post-test data and assessing whether the resulting data structures support the application of the then-test for evaluations of chronic disease self-management interventions.

Study Design and Setting
Pre-test, post-test, and then-test data were collected from 314 participants of self-management courses using the Health Education Impact Questionnaire (heiQ). The derived change scores (pre-test/post-test; then-test/post-test) were examined for their psychometric performance using tests of measurement invariance.

Results
Few questionnaire items were noninvariant across pre-test/post-test, with four items identified and requiring removal to enable an unbiased comparison of factor means. In contrast, 12 items were identified and required removal in then-test/post-test data to avoid biased change score estimates.

Conclusion
Traditional pre-test/post-test data appear to be robust with little indication of response shift. In contrast, the weaker psychometric performance of then-test/post-test data suggests psychometric flaws that may be the result of implicit theory of change, social desirability, and recall bias.

Individual prevention courses for occupational skin diseases: changes in and relationships between proximal and distal outcomes

BACKGROUND: To treat people with occupational contact dermatitis, the German Accident Prevention and Insurance Association in the Health and Welfare Services offers 2-day individual prevention (IP) seminars. OBJECTIVES: We investigated whether there are short-term and medium-term changes in proximal (e.g. behaviour) and distal (e.g. symptoms) outcomes after an IP seminar, whether changes in proximal outcomes are associated with changes in distal outcomes, and whether subgroups can be identified that benefit in particular. PATIENTS/MATERIALS/METHODS: In a prospective study, 502 participants of 85 IP courses completed the health education impact questionnaire (heiQ™) and skin symptom questionnaire (Skindex-29) at the start of the course, immediately thereafter, and after 6 months. Change was assessed according to standardized effect size. Regression techniques were used to analyse associations between proximal and distal outcomes. RESULTS: After 6 months, participants showed improved self-management skills and preventive behaviour, and less fear of job loss, disease-related symptoms, and emotional distress. Significant associations between proximal and distal outcomes were found. Participants who felt more limited by their skin disease showed greater effects. CONCLUSIONS: The results are consistent with the assumption that IP courses provide a range of benefits for people with occupational contact dermatitis. Changes in distal outcomes may be influenced by changes in proximal outcomes.

Positive changes in self-management and disease severity following climate therapy in people with psoriasis

The aim of this study was to investigate the impact of climate therapy on self-management in people with psoriasis. This was a prospective study of 254 adults with chronic psoriasis who participated in a 3-week climate therapy (CT) programme. The 8-scale Health Education Impact Questionnaire (heiQ) was completed at baseline, after 3 weeks of CT, and 3 months later. Change was assessed using paired sample t-tests mean (95% confidence interval) change scores (range 1-4). All heiQ scales showed statistically significant improvement after 3 weeks of CT. The greatest improvement was in Health-directed activity, followed by Emotional distress, and Skill and technique acquisition. At the 3-month follow-up, only the Emotional distress scale remained improved. In addition, disease severity (self-administered PASI; SAPASI) improved significantly from before CT to 3 weeks and 3 months after CT. This study suggests that CT provides a range of benefits that are important to people with psoriasis, particularly in the short term. A challenge is how to achieve long-term benefits.

Can a disease-specific education program augment self-management skills and improve Health-Related Quality of Life in people with hip or knee osteoarthritis?

Background: Patient education and self-management programs are offered in many countries to people with chronic conditions such as osteoarthritis (OA). The most well-known is the disease-specific Stanford Arthritis Self-Management Program (ASMP). While Australian and international clinical guidelines promote the concept of self-management for OA, there is currently little evidence to support the use of the ASMP. Several meta-analyses have reported that arthritis self-management programs had minimal or no effect on reducing pain and disability. However, previous studies have had methodological shortcomings including the use of outcome measures which do not accurately reflect program goals. Additionally, limited cost-effectiveness analyses have been undertaken and the cost-utility of the program has not been explored.

Methods/design: This study is a randomised controlled trial to determine the efficacy (in terms of Health-Related Quality of Life and self-management skills) and cost-utility of a 6-week group-based Stanford ASMP for people with hip or knee OA.

Six hundred participants referred to an orthopaedic surgeon or rheumatologist for hip or knee OA will be recruited from outpatient clinics at 2 public hospitals and community-based private practices within 2 private hospital settings in Victoria, Australia. Participants must be 18 years or over, fluent in English and able to attend ASMP sessions. Exclusion criteria include cognitive dysfunction, previous participation in self-management programs and placement on a waiting list for joint replacement surgery or scheduled joint replacement.

Eligible, consenting participants will be randomised to an intervention group (who receive the ASMP and an arthritis self-management book) or a control group (who receive the book only). Follow-up will be at 6 weeks, 3 months and 12 months using standardised self-report measures. The primary outcome is Health-Related Quality of Life at 12 months, measured using the Assessment of Quality of Life instrument. Secondary outcome measures include the Health Education Impact Questionnaire, Western Ontario and McMaster Universities Osteoarthritis Index (pain subscale and total scores), Kessler Psychological Distress Scale and the Hip and Knee Multi-Attribute Priority Tool. Cost-utility analyses will be undertaken using administrative records and self-report data. A subgroup of 100 participants will undergo qualitative interviews to explore the broader potential impacts of the ASMP.

Discussion: Using an innovative design combining both quantitative and qualitative components, this project will provide high quality data to facilitate evidence-based recommendations regarding the ASMP.

The inclusion of ‘then-test’ questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation

Objectives Program evaluations are frequently based on ‘then-test’ data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects’ responses to post-test.
Patients and methods To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
Results Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then test questions reporting significantly higher post-test scores than respondents of the other groups.
Conclusions This study provides evidence that the inclusion of then-test questions alters post-test responses,
suggesting that change scores based on then-test data be interpreted with care.

Change of perspective: a measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/postintervention assessment

Objective: To determine if participants in chronic disease self-management courses have a change of perspective of their health status (a response shift), and if this is measurable with a paper-based questionnaire.

Methods: Nine items were developed to measure potential benefits of self-management courses. These were based on the constructs of a previous questionnaire, the Health Education Impact Questionnaire (HEI-Q). Cognitive interviews elicited spontaneous statements about the reasons for paper-based answers. Sensitivity, specificity, and overall accuracy of items were calculated using the interview as a relative gold standard. Response shift can be negative (i.e., after the course, participants realize that, before the course, they were worse than they thought they were), positive (i.e., participants now realize they were better than they thought they were), or absent (no change).

Results: Interviews (n = 39) reflected that true response shift occurred in approximately half the replies to questionnaire items. Of these, 31% were negative response shift, 20% were positive response shift. Response shift was absent in 32% of replies. Presence or absence of response shift could not be determined in 17% of replies across items. Significant concordance between questionnaires and cognitive interviews (average overall accuracy 0.79) indicated that the HEI-Q Perspective questionnaire detects response shift in participants of self-management courses. The questionnaire revealed that 87% of participants had response shift in at least 1 item.

Conclusion: This study suggests that preintervention/postintervention assessments of interventions such as self-management courses are confounded by a change in perspective of a large proportion of respondents. It also indicates response shift is a valuable outcome of self-management courses that can be measured with a paper-based questionnaire.

Getting your life back on track after stroke : a phase II multi-centered, single-blind, randomized, controlled trial of the Stroke Self-Management Program vs. the Stanford Chronic Condition Self-Management Program or standard care in stroke survivors

Background Self-management is seen as a primary mechanism to support the optimization of care for people with chronic diseases such as symptomatic vascular disease. There are no established and evidence-based stroke-specific chronic disease self-management programs. Our aim is to evaluate whether a stroke-specific program is safe and feasible as part of a Phase II randomized-controlled clinical trial.
Methods Stroke survivors are recruited from a variety of sources including: hospital stroke services, local paper advertisements, Stroke South Australia newsletter (volunteer peer support organization), Divisions of General Practice, and community service providers across Adelaide, South Australia. Subjects are invited to participate in a multi-center, single-blind, randomized, controlled trial. Eligible participants are randomized to either;
• standard care,
• standard care plus a six week generic chronic condition self-management group education program, or,
• standard care plus an eight week stroke specific self-management education group program.
Interventions are conducted after discharge from hospital. Participants are assessed at baseline, immediate post intervention and six months.
Study Outcomes The primary outcome measures determine study feasibility and safety, measuring, recruitment, participation, compliance and adverse events.
Secondary outcomes include:
• positive and active engagement in life measured by the Health Education Impact Questionnaire,
• improvements in quality of life measured by the Assessment of Quality of Life instrument,
• improvements in mood measured by the Irritability, Depression and Anxiety Scale,
• health resource utilization measured by a participant held diary and safety.

Conclusion The results of this study will determine whether a definitive Phase III efficacy trial is justified.

Who benefits most from participating in chronic disease self-management programs?

Objectives: To quantify the benefits that people receive from participating in self-management courses and identify subgroups that benefit most.
Research Design: People with a wide range of chronic conditions attending self-management courses (N=1,341 individuals) were administered the generic Health Education Impact Questionnaire (HEI-Q). Data were collected before the first session (baseline) and at the end of courses (follow-up) resulting in 842 complete responses. The median (interquartile range) age was 64 (54 to 73) years and most participants were female (75%). Outcomes were categorized as Substantial improvement (Effect Size, ES ≥ 0.5), Minimal/No change (ES -0.49 to 0.49) and Substantial decline (ES ≤ -0.5).
Results: On average, one third of participants reported substantial benefits after attending a self-management course. Proportions of participants reporting substantial benefits ranged from 49% in Skill and technique acquisition to 27% in Health service navigation. Stratification by gender, age and education showed that younger participants were more likely to benefit, particularly young women. No further subgroup differences were observed.
Conclusions: Given that the health of people with chronic diseases tends to decline, this evaluation is reassuring in that about one third of participants coming from a wide range of backgrounds receive substantial improvements in their self-management skills.

A phase II multicentered, single-blind, randomized, controlled trial of the stroke self-management program

Background and Purpose—The benefits of chronic disease self-management programs for stroke survivors are uncertain because individuals with severe impairments have been excluded from previous research. We undertook a phase II randomized controlled trial to determine whether a self-management program designed for survivors (SSMP; 8 weeks) was safe and feasible compared to standard care (control) or a generic self-management program (generic; 6 weeks).
Methods—Stroke survivors were recruited from 7 South Australian hospitals via a letter or indirectly (eg, newspapers). Eligible participants were randomized at a 1:1:1 ratio of 50 per group. Primary outcomes were recruitment, participation, and participant safety. Secondary outcomes were positive and active engagement in life using the Health Education Impact Questionnaire and characteristics of quality of life and mood at 6 months from program completion.
Results—Of 315 people screened, 149 were eligible and 143 were randomized (48 SSMP, 47 generic, 48 control); mean age was 69 years (SD, 11) and 59% were female. Demographic features were similar between groups and 41% had severe cognitive impairment; 57% accessed the interventions, with 52% SSMP and 38% generic completing >50% of sessions (P=0.18). Thirty-two participants reported adverse events (7 control, 12 generic, 13 SSMP; P=0.3; 34% severe); however, none was attributable to the interventions. Potential benefits for improved mood were found.
Conclusions—SSMP was safe and feasible. Benefits of the stroke-specific program over the generic program included greater participation and completion rates. An efficacy trial is warranted given the forecast growth in the stroke population and improved survival trends.

How do postgraduate qualifications in medical education impact on health professionals?

Background and Purpose: The number of degree-awarding programmes in medical education is steadily increasing. Despite the popularity and extensive investment in these courses, there is little research into their impact. This study investigated the perceived impact of an internationally-renowned postgraduate programme in medical education on health professionals’ development as educators.

Methods: An online survey of the 2008–12 graduates from the Centre for Medical Education, University of Dundee was carried out. Their self-reported shifts in various educational competencies and scholarship activities were analysed using non-parametric statistics. Qualitative data were also collected and analysed to add depth to the quantitative findings.

Results: Of the 504 graduates who received the online questionnaire 224 responded. Participants reported that a qualification in medical education had significantly (p < 0.001) improved their professional educational practices and engagement in scholarly activities. Masters graduates reported greater impact compared to Certificate graduates on all items, including ability to facilitate curriculum reforms, and in assessment and feedback practices. Masters graduates also reported more engagement in scholarship activities, with significantly greater contributions to journals. These qualifications equally benefited all participants regardless of age. International graduates reported greater impact of the qualification than their UK counterparts.

Conclusion: A postgraduate medical education programme can significantly impact on the practices and behaviours of health professionals in education, improving self-efficacy and instilling an increased sense of belonging to the educational community.

Comparison of the pre-post and transition question assessments in a health education setting

Objective : This study examined the classical pre-intervention/post-intervention assessment (pre–post) and the single post-intervention transition question assessment (transition question) to determine how well these methods reflected qualitative interview–based participant-reported outcomes from chronic disease self-management education programs (CDSMEPs).

Study Design and Setting : A mixed-method qualitative and quantitative approach was applied in 25 interviews with participants recruited from CDSMEPs within Australia. Qualitative interviews with participants were used as a relative “gold standard” and compared with questionnaire-based pre–post and transition question assessments.

Results : Comparison of the two questionnaire-based assessments showed that most of the individual paired responses were discordant (61%). Using participant's qualitative narratives as a “true” indicator, the pre–post assessment was found on more occasions to be discordant with participant-reported outcomes than the transition question. The origin of the inconsistency was largely because of a change in respondents' perspective that had occurred after pretest, which was mediated by CDSMEPs' experiences and insights.

Conclusion : This study suggests that the pre–post assessment has poor validity for the assessment of health education program outcomes. Alternative assessments, such as the transition question, may result in a more accurate reflection of the impact of such programs on participants.