I don't see me like you see me, but is that a problem? Cultural influences on rating discrepancy in 360-degree feedback instruments

360-degree feedback from a variety of rater sources yields important information about leaders' styles, strengths and weaknesses for development. Results where observer ratings are discrepant (i.e., different) from self-ratings are often seen as indicators of problematic leadership relationships, skills, or lack of self-awareness. Yet research into the antecedents of such self-observer rating discrepancy suggests the presence of systematic influences, such as cultural values. The present study investigates the variation of rating discrepancies on three leadership skills (decision making, leading employees, and composure) in dependence of one exemplary culture dimension (power distance) on data from 31 countries using multilevel structural equation modelling. Results show that cultural values indeed predict self-observer rating discrepancies. Thus, systemic and contextual influences such as culture need to be taken into consideration when interpreting the importance and meaning of self-observer rating discrepancies in 360-degree instruments.

Quality of life instruments in studies of menorrhagia: a systematic review

BACKGROUND: The use of quality of life (QoL) instruments in menorrhagia research is increasing but there is concern that not enough emphasis is placed on patient-focus in these measurements, i.e. on issues which are of importance to patients and reflect their experiences and concerns (clinical face validity). The objective was to assess the quality of QoL instruments in studies of menorrhagia. STUDY DESIGN: A systematic review of published research. Papers were identified through MEDLINE (1966-April 2000), EMBASE (1980-April 2000), Science Citation Index (1981-April 2000), Social Science Citation Index (1981-April 2000), CINAHL (1982-1999) and PsychLIT (1966-1999), and by manual searching of bibliographies of known primary and review articles. Studies were selected if they assessed women with menorrhagia for life quality, either developing QoL instruments or applying them as an outcome measure. Selected studies were assessed for quality of their QoL instruments, using a 17 items checklist including 10 items for clinical face validity (issues of relevance to patients' expectations and concerns) and 7 items for measurement properties (such as reliability, responsiveness, etc.). RESULTS: A total of 19 articles, 8 on instrument development and 11 on application, were included in the review. The generic Short Form 36 Health Survey Questionnaire (SF36) was used in 12/19 (63%) studies. Only two studies developed new specific QoL instruments for menorrhagia but they complied with 7/17 (41%) and 10/17 (59%) of the quality criteria. Quality assessment showed that only 7/19 (37%) studies complied with more than half the criteria for face validity whereas 17/19 (90%) studies complied with more than half of the criteria for measurement properties (P = 0.0001). CONCLUSION: Among existing QoL instruments, there is good compliance with the quality criteria for measurement properties but not with those for clinical face validity. There is a need to develop methodologically sound disease specific QoL instruments in menorrhagia focussing both on face validity and measurement properties.

Quality of life instruments in studies of menorrhagia: a systematic review

BACKGROUND: The use of quality of life (QoL) instruments in menorrhagia research is increasing but there is concern that not enough emphasis is placed on patient-focus in these measurements, i.e. on issues which are of importance to patients and reflect their experiences and concerns (clinical face validity). The objective was to assess the quality of QoL instruments in studies of menorrhagia. STUDY DESIGN: A systematic review of published research. Papers were identified through MEDLINE (1966-April 2000), EMBASE (1980-April 2000), Science Citation Index (1981-April 2000), Social Science Citation Index (1981-April 2000), CINAHL (1982-1999) and PsychLIT (1966-1999), and by manual searching of bibliographies of known primary and review articles. Studies were selected if they assessed women with menorrhagia for life quality, either developing QoL instruments or applying them as an outcome measure. Selected studies were assessed for quality of their QoL instruments, using a 17 items checklist including 10 items for clinical face validity (issues of relevance to patients' expectations and concerns) and 7 items for measurement properties (such as reliability, responsiveness, etc.). RESULTS: A total of 19 articles, 8 on instrument development and 11 on application, were included in the review. The generic Short Form 36 Health Survey Questionnaire (SF36) was used in 12/19 (63%) studies. Only two studies developed new specific QoL instruments for menorrhagia but they complied with 7/17 (41%) and 10/17 (59%) of the quality criteria. Quality assessment showed that only 7/19 (37%) studies complied with more than half the criteria for face validity whereas 17/19 (90%) studies complied with more than half of the criteria for measurement properties (P = 0.0001). CONCLUSION: Among existing QoL instruments, there is good compliance with the quality criteria for measurement properties but not with those for clinical face validity. There is a need to develop methodologically sound disease specific QoL instruments in menorrhagia focussing both on face validity and measurement properties.