Assessing the physical environment of older people’s residential care facilities: development of the Swedish version of the Sheffield Care Environment Assessment Matrix (S-SCEAM)

Background There is emerging evidence that the physical environment is important for health, quality of life and care, but there is a lack of valid instruments to assess health care environments. The Sheffield Care Environment Assessment Matrix (SCEAM), developed in the United Kingdom, provides a comprehensive assessment of the physical environment of residential care facilities for older people. This paper reports on the translation and adaptation of SCEAM for use in Swedish residential care facilities for older people, including information on its validity and reliability. Methods SCEAM was translated into Swedish and back-translated into English, and assessed for its relevance by experts using content validity index (CVI) together with qualitative data. After modification, the validity assessments were repeated and followed by test-retest and inter-rater reliability tests in six units within a Swedish residential care facility that varied in terms of their environmental characteristics. Results Translation and back translation identified linguistic and semantic related issues. The results of the first content validity analysis showed that more than one third of the items had item-CVI (I-CVI) values less than the critical value of 0.78.  After modifying the instrument, the second content validation analysis resulted in I-CVI scores above 0.78, the suggested criteria for excellent content validity. Test-retest reliability showed high stability (96% and 95% for two independent raters respectively), and inter-rater reliability demonstrated high levels of agreement (95% and 94% on two separate rating occasions). Kappa values were very good for test-retest (κ= 0.903 and 0.869) and inter-rater reliability (κ= 0.851 and 0.832). Conclusions Adapting an instrument to a domestic context is a complex and time-consuming process, requiring an understanding of the culture where the instrument was developed and where it is to be used. A team, including the instrument’s developers, translators, and researchers is necessary to ensure a valid translation and adaption. This study showed preliminary validity and reliability evidence for the Swedish version (S-SCEAM) when used in a Swedish context. Further, we believe that the S-SCEAM has improved compared to the original instrument and suggest that it can be used as a foundation for future developments of the SCEAM model.

Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings

Background: The gap between what is known and what is practiced results in health service users not benefitting from advances in healthcare, and in unnecessary costs. A supportive context is considered a key element for successful implementation of evidence-based practices (EBP). There were no tools available for the systematic mapping of aspects of organizational context influencing the implementation of EBPs in low- and middle-income countries (LMICs). Thus, this project aimed to develop and psychometrically validate a tool for this purpose. Methods: The development of the Context Assessment for Community Health (COACH) tool was premised on the context dimension in the Promoting Action on Research Implementation in Health Services framework, and is a derivative product of the Alberta Context Tool. Its development was undertaken in Bangladesh, Vietnam, Uganda, South Africa and Nicaragua in six phases: (1) defining dimensions and draft tool development, (2) content validity amongst in-country expert panels, (3) content validity amongst international experts, (4) response process validity, (5) translation and (6) evaluation of psychometric properties amongst 690 health workers in the five countries. Results: The tool was validated for use amongst physicians, nurse/midwives and community health workers. The six phases of development resulted in a good fit between the theoretical dimensions of the COACH tool and its psychometric properties. The tool has 49 items measuring eight aspects of context: Resources, Community engagement, Commitment to work, Informal payment, Leadership, Work culture, Monitoring services for action and Sources of knowledge. Conclusions: Aspects of organizational context that were identified as influencing the implementation of EBPs in high-income settings were also found to be relevant in LMICs. However, there were additional aspects of context of relevance in LMICs specifically Resources, Community engagement, Commitment to work and Informal payment. Use of the COACH tool will allow for systematic description of the local healthcare context prior implementing healthcare interventions to allow for tailoring implementation strategies or as part of the evaluation of implementing healthcare interventions and thus allow for deeper insights into the process of implementing EBPs in LMICs.

Response process and test-retest reliability of the Context Assessment for Community Health tool in Vietnam

BACKGROUND: The recently developed Context Assessment for Community Health (COACH) tool aims to measure aspects of the local healthcare context perceived to influence knowledge translation in low- and middle-income countries. The tool measures eight dimensions (organizational resources, community engagement, monitoring services for action, sources of knowledge, commitment to work, work culture, leadership, and informal payment) through 49 items. OBJECTIVE: The study aimed to explore the understanding and stability of the COACH tool among health providers in Vietnam. DESIGNS: To investigate the response process, think-aloud interviews were undertaken with five community health workers, six nurses and midwives, and five physicians. Identified problems were classified according to Conrad and Blair's taxonomy and grouped according to an estimation of the magnitude of the problem's effect on the response data. Further, the stability of the tool was examined using a test-retest survey among 77 respondents. The reliability was analyzed for items (intraclass correlation coefficient (ICC) and percent agreement) and dimensions (ICC and Bland-Altman plots). RESULTS: In general, the think-aloud interviews revealed that the COACH tool was perceived as clear, well organized, and easy to answer. Most items were understood as intended. However, seven prominent problems in the items were identified and the content of three dimensions was perceived to be of a sensitive nature. In the test-retest survey, two-thirds of the items and seven of eight dimensions were found to have an ICC agreement ranging from moderate to substantial (0.5-0.7), demonstrating that the instrument has an acceptable level of stability. CONCLUSIONS: This study provides evidence that the Vietnamese translation of the COACH tool is generally perceived to be clear and easy to understand and has acceptable stability. There is, however, a need to rephrase and add generic examples to clarify some items and to further review items with low ICC.