Evaluation in der wissenschaftlichen Weiterbildung: Konzepte und Praxis

Dieser Beitrag befasst sich damit, • was Evaluation bedeutet, welche Schritte sie umfasst und welche möglichen Nutzungsformen von Evaluation es gibt • was mögliche «Gegenstände» der Evaluation im Bereich der wissenschaftlichen Weiterbildung sind und wie man an deren Evaluation herangehen kann • welche Erfahrungen Studienleiterinnen und Studienleiter sowie Weiterbildungsverantwortliche mit der Evaluation in ihren jeweiligen Tätigkeitsbereichen gemacht haben. Das Hauptthema der Herbsttagung 2014 des Zentrums für universitäre Weiterbildung ZUW, die Frage der Nützlichkeit und Nutzung von Evaluation, wird dabei unter verschiedenen Blickwinkeln vertieft.

How valuable are multiple treatment comparison methods in evidence-based health-care evaluation?

Objectives To compare the use of pair-wise meta-analysis methods to multiple treatment comparison (MTC) methods for evidence-based health-care evaluation to estimate the effectiveness and cost-effectiveness of alternative health-care interventions based on the available evidence. Methods Pair-wise meta-analysis and more complex evidence syntheses, incorporating an MTC component, are applied to three examples: 1) clinical effectiveness of interventions for preventing strokes in people with atrial fibrillation; 2) clinical and cost-effectiveness of using drug-eluting stents in percutaneous coronary intervention in patients with coronary artery disease; and 3) clinical and cost-effectiveness of using neuraminidase inhibitors in the treatment of influenza. We compare the two synthesis approaches with respect to the assumptions made, empirical estimates produced, and conclusions drawn. Results The difference between point estimates of effectiveness produced by the pair-wise and MTC approaches was generally unpredictable—sometimes agreeing closely whereas in other instances differing considerably. In all three examples, the MTC approach allowed the inclusion of randomized controlled trial evidence ignored in the pair-wise meta-analysis approach. This generally increased the precision of the effectiveness estimates from the MTC model. Conclusions The MTC approach to synthesis allows the evidence base on clinical effectiveness to be treated as a coherent whole, include more data, and sometimes relax the assumptions made in the pair-wise approaches. However, MTC models are necessarily more complex than those developed for pair-wise meta-analysis and thus could be seen as less transparent. Therefore, it is important that model details and the assumptions made are carefully reported alongside the results.

Psychometric properties of the MacNew heart disease health-related quality of life instrument in patients with heart failure

RATIONALE, AIMS AND OBJECTIVES: Heart failure (HF) is a severe chronic disease and impairs health-related quality of life (HRQL). While validated specific HRQL instruments are required for evaluation of treatment and rehabilitation in patients with HF, a single validated measure to document changes in HRQL for patients with different heart disease diagnoses would be invaluable. The purpose of this analysis was the psychometric analysis of the German MacNew Heart Disease Questionnaire (MacNew) in HF patients, which has previously been shown to be reliable and valid in patients with myocardial infarction, angina pectoris and arrhythmia. METHODS: We recruited 89 patients (61.7+/-11.5 years; 84.3% male) in two Austrian and one Swiss cardiology department with documented HF (effect sizes 28.9+/-10.1%). The self-administered MacNew, the Short Form-36 (SF-36) and the Hospital Anxiety and Depression Scale were completed. Internal consistency reliability (Cronbach's alpha), discriminative and evaluative validity were assessed. RESULTS: Cronbach's alpha exceeded 0.80. Each MacNew scale differentiated between patients with and without anxiety (3.9+/-1.0 vs. 5.3+/-0.8, all P<0.001), with and without depression (4.2+/-1.2 vs. 5.2+/-0.9 all P<0.03) and by the SF-36 health transition item (deteriorate=4.39, no change=4.95, improve=5.45, all P<0.02). Evaluative validity was demonstrated with effect sizes >0.70 for a subsample attending a 12-week outpatient rehabilitation programme. CONCLUSIONS: The German language version of the MacNew demonstrates consistently acceptable psychometric properties of reliability, validity and responsiveness in patients with documented HF. Together with previous documentation of reliability, validity and responsive, these findings strengthen the argument for the MacNew as a potential 'core' HRQL measure, at least in the German language.

Predictors of health care professionals' attitudes towards involvement in safety-relevant behaviours

BACKGROUND Patients can make valuable contributions towards promoting the safety of their health care. Health care professionals (HCPs) could play an important role in encouraging patient involvement in safety-relevant behaviours. However, to date factors that determine HCPs' attitudes towards patient participation in this area remain largely unexplored. OBJECTIVE To investigate predictors of HCPs' attitudes towards patient involvement in safety-relevant behaviours. DESIGN A 22-item cross-sectional fractional factorial survey that assessed HCPs' attitudes towards patient involvement in relation to two error scenarios relating to hand hygiene and medication safety. SETTING Four hospitals in London PARTICIPANTS   Two hundred sixteen HCPs (116 doctors; 100 nurses) aged between 21 and 60 years (mean: 32): 129 female. OUTCOME MEASURES Approval of patient's behaviour, HCP response to the patient, anticipated effects on the patient-HCP relationship, support for being asked as a HCP, affective rating response to the vignettes. RESULTS HCPs elicited more favourable attitudes towards patients intervening about a medication error than about hand sanitation. Across vignettes and error scenarios, the strongest predictors of attitudes were how the patient intervened and how the HCP responded to the patient's behaviour. With regard to HCP characteristics, doctors viewed patients intervening less favourably than nurses. CONCLUSIONS HCPs perceive patients intervening about a potential error less favourably if the patient's behaviour is confrontational in nature or if the HCP responds to the patient intervening in a discouraging manner. In particular, if a HCP responds negatively to the patient (irrespective of whether an error actually occurred), this is perceived as having negative effects on the HCP-patient relationship.

Acute cocaine-related health problems in patients presenting to an urban emergency department in Switzerland: a case series

BACKGROUND Emergency departments may be a useful information source to describe the demographics and clinical characteristics of patients with acute cocaine-related medical problems. We therefore conducted a retrospective analysis of 165 acute, laboratory-confirmed cocaine intoxications admitted to an urban emergency department in Switzerland between January 2007 and March 2011. RESULTS A total of 165 patients with a mean age of 32 years were included. Most patients were male (73%) and unemployed (65%). Only a minority (16%) had abused cocaine alone while 84% of the patients had used at least one additional substance, most commonly ethanol (41%), opioids (38%), or cannabis (36%) as confirmed by their detection in blood samples. The most frequently reported symptoms were chest pain (21%), palpitations (19%), anxiety (36%) and restlessness (36%). Psychiatric symptoms were present in 64%. Hypertension and tachycardia were observed in 53% and 44% of the patients, respectively. Severe poisonings only occurred in patients with multiple substance intoxication (15%). Severe intoxications were non-significantly more frequent with injected drug use compared to nasal, oral, or inhalational drug use. Severe complications included acute myocardial infarction (2 cases), stroke (one case), and seizures (3 cases). Most patients (75%) were discharged home within 24 h after admission. A psychiatric evaluation in the ED was performed in 24% of the patients and 19% were referred to a psychiatric clinic. CONCLUSIONS Patients with acute cocaine intoxication often used cocaine together with ethanol and opioids and presented with sympathomimetic toxicity and/or psychiatric disorders. Severe acute toxicity was more frequent with multiple substance use. Toxicity was typically short-lasting but psychiatric evaluation and referral was often needed.