Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.

A qualitative approach to assessing the validity of the posttraumatic growth inventory

The Posttraumatic Growth Inventory (PTGI; Tedeschi & Calhoun, 1996) is the most commonly used measure of positive psychological change that can result from negotiating a traumatic experience. Whilst the PTGI has strong internal reliability, validity studies are still sparse. The presented research details trauma survivors’ understanding of items comprising the PTGI in order to qualitatively assess content validity. Participants were 14 trauma survivors who completed the PTGI and participated in a semi-structured interview. Thematic Analysis was conducted on participant’s transcribed interviews. One latent theme was identified reflecting that questions were consistently understood. A relationship was found between the constituent themes identified and the five factors of the PTGI. Participants answered the PTGI statements in a way that is consistent with the purpose of the instrument with only a small discrepancy found when some participants used the PTGI scale to indicate when a decrease in an element of the inventory had been experienced. Overall results supported the content validity of the PTGI.

Measuring the impact of burn scarring on health-related quality of life: Development and preliminary content validation of the Brisbane Burn Scar Impact Profile (BBSIP) for children and adults

INTRODUCTION No burn-scar specific, health-related quality of life (HRQOL) measure exists. This study aimed to develop a patient-reported, evaluative HRQOL measure to assess the impact of burn scarring in children and adults. METHOD Semi-structured interviews, content validation surveys, and cognitive interviews were used to develop and test content validity of a new measure - the Brisbane Burn Scar Impact Profile (BBSIP). RESULTS Participants comprised Australian adults (n=23) and children (n=19) with burn scarring; caregivers of children with burn scarring (n=28); and international scar management experts (n=14). Items distinct from other burn scar measures emerged. Four versions of the BBSIP were developed; one for children aged 8-18 years, one for adults, one for caregivers (as proxies for children aged less than 8-years), and one for caregivers of children aged 8-18 years. Preliminary content validity of the BBSIP was supported. Final items covered physical and sensory symptoms; emotional reactions; impact on social functioning and daily activities; impact of treatment; and environmental factors. CONCLUSION The BBSIP was developed to assess burn-scar specific HRQOL and will be available at http://www.coolburns.com.au under a creative commons license. Further testing is underway.

Development, reliability and validity of a tool, to measure emergency department clinicians' attitudes towards family presence (FP) during acute deterioration in adult patients.

Despite many studies of family presence during resuscitation, no validated tool exploring the attitudes and beliefs of healthcare staff towards family presence has been published. The aim of this paper is to describe the development of a tool to accurately measure the attitudes and beliefs of emergency department staff towards family presence in the deteriorating adult patient, present the results of validity and reliability testing, and present the final validated tool. Twenty-nine items were developed, informed by themes from the literature and unvalidated published tools related to family presence during resuscitation. The tool was piloted on a sample of 68 emergency nursing and medical staff. Content validity and face validity were established using feedback from participants. Reliability was established by unidimensionality, exploratory factor analysis and internal consistency. Sixteen items were deleted from the original tool due to low item-to-total correlations and low communalities. Exploratory factor analysis of the remaining items revealed four factors with acceptable correlation coefficients and appropriate explanation of variance. Cronbach's alpha for each factor was >0.7 indicating a high degree of internal consistency. The four factors were labelled and arranged in a logical order to form the final tool, the Emergency Department Family Presence Survey.

Validity and reliability of field-based measures for assessing movement skill competency in lifelong physical activities: a systematic review

Background: It has been suggested that young people should develop competence in a variety of ‘lifelong physical activities’ to ensure that they can be active across the lifespan. Objective: The primary aim of this systematic review is to report the methodological properties, validity, reliability, and test duration of field-based measures that assess movement skill competency in lifelong physical activities. A secondary aim was to clearly define those characteristics unique to lifelong physical activities. Data Sources: A search of four electronic databases (Scopus, SPORTDiscus, ProQuest, and PubMed) was conducted between June 2014 and April 2015 with no date restrictions. Study Selection: Studies addressing the validity and/or reliability of lifelong physical activity tests were reviewed. Included articles were required to assess lifelong physical activities using process-oriented measures, as well as report either one type of validity or reliability. Study Appraisal and Synthesis Methods: Assessment criteria for methodological quality were adapted from a checklist used in a previous review of sport skill outcome assessments. Results: Movement skill assessments for eight different lifelong physical activities (badminton, cycling, dance, golf, racquetball, resistance training, swimming, and tennis) in 17 studies were identified for inclusion. Methodological quality, validity, reliability, and test duration (time to assess a single participant), for each article were assessed. Moderate to excellent reliability results were found in 16 of 17 studies, with 71 % reporting inter-rater reliability and 41 % reporting intra-rater reliability. Only four studies in this review reported test–retest reliability. Ten studies reported validity results; content validity was cited in 41 % of these studies. Construct validity was reported in 24 % of studies, while criterion validity was only reported in 12 % of studies. Limitations: Numerous assessments for lifelong physical activities may exist, yet only assessments for eight lifelong physical activities were included in this review. Generalizability of results may be more applicable if more heterogeneous samples are used in future research. Conclusion: Moderate to excellent levels of inter- and intra-rater reliability were reported in the majority of studies. However, future work should look to establish test–retest reliability. Validity was less commonly reported than reliability, and further types of validity other than content validity need to be established in future research. Specifically, predictive validity of ‘lifelong physical activity’ movement skill competency is needed to support the assertion that such activities provide the foundation for a lifetime of activity.

Validity and reliability of the Portuguese version of Mandibular Function Impairment Questionnaire

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Design and validity of a questionnaire to assess sexuality in pregnant women

Background. A review of validated methods for assessing female sexual dysfunction and a review of male and female sexual dysfunction did not refer to any specific questionnaire for evaluating sexuality during pregnancy. A study was performed at the Obstetrics and Gynecology Department of Botucatu Medical School, São Paulo State University, Brazil to design and validate a pregnancy sexuality questionnaire, the Pregnancy Sexual Response Inventory (PSRI). Methods. Women with a singleton pregnancy between 10 and 35 weeks of gestation were randomly recruited. There were five phases in the development of the PSRI: (1) item selection; (2) item development; (3) determination of internal consistency, reliability and convergence; (4) content validity; and (5) determination of inter-interviewer reliability. Internal consistency and reliability were evaluated using Cronbach's alpha. Inter-interviewer reliability was assessed by evaluating the responses of 18 academics at various institutions, using Kappa Index and Student t test. Results. Good internal consistency and reliability were obtained (Cronbach's alpha coefficient = 0.79). Among the 18 academics, 13 totally agreed (K = 1.0), three partially agreed (K = 0.67) and two disagreed (K = 0.33) with the proposed questions. Comparisons of the mean PSRI domain scores made between the primary investigators and the other interviewers showed no significant differences in all domains (p > 0.05). Conclusion. PSRI is a new validated instrument for evaluating sexuality and sexual activity and related health concerns during pregnancy. © 2009 Rudge et al; licensee BioMed Central Ltd.

Reliability and validity of spirituality questionnaire by Parsian and Dunning in the Spanish version

The aim of this methodological study was to translate, culturally adapt and assess the internal consistency and validity of the Spanish version of the "Spirituality Questionnaire". The sample comprised 204 young people between 18 and 25 years of age from two universities in Bogota. Cronbach's Alpha was used for reliability, while groups of experts and young people were used for construct validity. The reliability score of the total instrument was 0.88. The overall index of content validity corresponded to 0.90. Exploratory factor analysis showed that four factors explain 52.60% of the variance. The originally proposed theoretical model was confirmed and, in two dimensions, a different structure was proposed. In conclusion, the instrument "Spirituality Questionnaire" by Parsian and Dunning is reliable and valid in the Spanish version.

Validity evidence and reliability of a Simulated Patient Feedback Instrument

In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Methods Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. Results All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. Conclusions The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

Item validity of the physical demands from the Dictionary of Occupational Titles for the functional capacity evaluation of clients with chronic back pain

This study assessed the item validity of 15 of the physical demands from the Dictionary of Occupational Titles (DOT), as evaluated in a new approach to functional capacity evaluation (FCE) for clients with chronic back pain, the Gibson Approach to FCE (GAPP FCE). Fifty-two occupational therapists were sent the specifications of the items in the GAPP FCE procedures and were asked to rate the items in terms of item-objective congruence, relevance and difficulty. A response rate of 59.2% was obtained. The majority of the therapists agreed that most of the items were congruent with the objectives based on the definition of the physical demands from the DOT. The items evaluating Balancing and Pushing and Pulling had the lowest item-objective congruence. The evaluation of Balancing and the Lifting, Carrying and Pushing and Pulling of loads greater than light-medium weight (10–16 kg) were not considered significantly relevant. Concerns were raised about the difficulty and safety of the evaluation of Lifting, Carrying and Pushing and Pulling with clients with chronic back pain, particularly if the therapist evaluates the manual handling of medium to heavy loads. These results may have implications for other FCEs, particularly those which are based on the DOT, or when assessing clients with chronic back pain.

Measuring mobility in frail older people: reliability and validity of the Physical Mobility Scale

Objectives: The study was designed to show the validity and reliability of scoring the Physical Mobility Scale (PMS). PMS was developed by physiotherapists working in residential aged care to specifically show resident functional mobility and to provide information regarding each resident's need for supervision or assistance from one or two staff members and equipment during position changes, transfers, mobilising and personal care. Methods: Nineteen physiotherapists of varying backgrounds and experience scored the performances of nine residents of care facilities from video recordings. The performances were compared to scores on two 'gold standard' assessment tools. Four of the physiotherapists repeated the evaluations. Results: The PAIS showed excellent content validity and reliability. Conclusions: The PAIS provides graded performance of physical mobility, including level of dependency on staff and equipment. This is a major advantage over existing functional assessment tools. There is no need for specific training for physiotherapists to use the tool.

Participation in a positive youth development program and multi -problem alternative high school students' transformative goals: An investigation of the theoretical and methodological utility and validity of the Transformative Goal Attainment (TGAS)

There is a growing body of literature that provides evidence for the efficacy of positive youth development programs in general and preliminary empirical support for the efficacy of the Changing Lives Program (CLP) in particular. This dissertation sought to extend previous efforts to develop and preliminarily examine the Transformative Goal Attainment Scale (TGAS) as a measure of participant empowerment in the promotion of positive development. Consistent with recent advances in the use of qualitative research methods, this dissertation sought to further investigate the utility of Relational Data Analysis (RDA) for providing categorizations of qualitative open-ended response data. In particular, a qualitative index of Transformative Goals, TG, was developed to complement the previously developed quantitative index of Transformative Goal Attainment (TGA), and RDA procedures for calculating reliability and content validity were refined. Second, as a Stage I pilot/feasibility study this study preliminarily examined the potentially mediating role of empowerment, as indexed by the TGAS, in the promotion of positive development. ^ Fifty-seven participants took part in this study, forty CLP intervention participants and seventeen control condition participants. All 57 participants were administered the study's measures just prior to and just following the fall 2003 semester. This study thus used a short-term longitudinal quasi-experimental research design with a comparison control group. ^ RDA procedures were refined and applied to the categorization of open-ended response data regarding participants' transformative goals (TG) and future possible selves (PSQ-QE). These analyses revealed relatively strong, indirect evidence for the construct validity of the categories as well as their theoretically meaningful structural organization, thereby providing sufficient support for the utility of RDA procedures in the categorization of qualitative open-ended response data. ^ In addition, transformative goals (TG) and future possible selves (PSQ-QE), and the quantitative index of perceived goal attainment (TGA) were evaluated as potential mediators of positive development by testing their relationships to other indices of positive intervention outcome within a four-step method involving both analysis of variance (ANOVA and RMANOVAs) and regression analysis. Though more limited in scope than the efforts at the development and refinement of the measures of these mediators, the results were also promising. ^

The Reliability and Validity of a Child and Adolescent Participation in Decision Making Questionnaire

Background
There is a growing impetus across the research, policy and practice communities for children and young people to participate in decisions that affect their lives. Furthermore, there is a dearth of general instruments that measure children and young people’s views on their participation in decision making. This paper presents the reliability and validity of the Child and Adolescent Participation in Decision Making Questionnaire (CAP-DMQ) and specifically looks at a population of looked-after children where a lack of participation in decision making is an acute issue.
Methods
The participants were 151 looked after children and adolescents between 10-23 years of age who completed the 10 item CAP-DMQ. Of the participants 113 were in receipt of an advocacy service that had an aim of increasing participation in decision-making with the remaining participants not having received this service.
Results
The results showed that the CAP-DMQ had good reliability (Cronbach’s alpha = .94) and showed promising uni-dimensional construct validity through an exploratory factor analysis. The items in the CAP-DMQ also demonstrated good content validity by overlapping with prominent models of child and adolescent participation (Lundy 2007) and decision making (Halpern 2014). A regression analysis showed that age and gender were not significant predictors of CAP-DMQ scores but receipt of advocacy was a significant predictor of scores (effect size d=.88), thus showing appropriate discriminant criterion validity.
Conclusion
Overall, the CAP-DMQ showed good reliability and validity. Therefore, the measure has excellent promise for theoretical investigation in the area of child and adolescent participation in decision making and equally shows empirical promise for use as a measure in evaluating services which have increasing the participation of children and adolescents in decision making as an intended outcome.

Neonatal palliative care attitude scale : development of an instrument to measure the barriers to and facilitators of palliative care in neonatal nursing

OBJECTIVE The aim of this research project was to obtain an understanding of the barriers to and facilitators of providing palliative care in neonatal nursing. This article reports the first phase of this research: to develop and administer an instrument to measure the attitudes of neonatal nurses to palliative care. METHODS The instrument developed for this research (the Neonatal Palliative Care Attitude Scale) underwent face and content validity testing with an expert panel and was pilot tested to establish temporal stability. It was then administered to a population sample of 1285 neonatal nurses in Australian NICUs, with a response rate of 50% (N 645). Exploratory factor-analysis techniques were conducted to identify scales and subscales of the instrument. RESULTS Data-reduction techniques using principal components analysis were used. Using the criteria of eigenvalues being 1, the items in the Neonatal Palliative Care Attitude Scale extracted 6 factors, which accounted for 48.1% of the variance among the items. By further examining the questions within each factor and the Cronbach’s of items loading on each factor, factors were accepted or rejected. This resulted in acceptance of 3 factors indicating the barriers to and facilitators of palliative care practice. The constructs represented by these factors indicated barriers to and facilitators of palliative care practice relating to (1) the organization in which the nurse practices, (2) the available resources to support a palliative model of care, and (3) the technological imperatives and parental demands. CONCLUSIONS The subscales identified by this analysis identified items that measured both barriers to and facilitators of palliative care practice in neonatal nursing. While establishing preliminary reliability of the instrument by using exploratory factor-analysis techniques, further testing of this instrument with different samples of neonatal nurses is necessary using a confirmatory factor-analysis approach.

Resilience in chronic disease : the relationships among risk factors, protective factors, adaptive outcomes, and the level of resilience in adults with diabetes

Background: There are innumerable diabetes studies that have investigated associations between risk factors, protective factors, and health outcomes; however, these individual predictors are part of a complex network of interacting forces. Moreover, there is little awareness about resilience or its importance in chronic disease in adulthood, especially diabetes. Thus, this is the first study to: (1) extensively investigate the relationships among a host of predictors and multiple adaptive outcomes; and (2) conceptualise a resilience model among people with diabetes. Methods: This cross-sectional study was divided into two research studies. Study One was to translate two diabetes-specific instruments (Problem Areas In Diabetes, PAID; Diabetes Coping Measure, DCM) into a Chinese version and to examine their psychometric properties for use in Study Two in a convenience sample of 205 outpatients with type 2 diabetes. In Study Two, an integrated theoretical model is developed and evaluated using the structural equation modelling (SEM) technique. A self-administered questionnaire was completed by 345 people with type 2 diabetes from the endocrine outpatient departments of three hospitals in Taiwan. Results: Confirmatory factor analyses confirmed a one-factor structure of the PAID-C which was similar to the original version of the PAID. Strong content validity of the PAID-C was demonstrated. The PAID-C was associated with HbA1c and diabetes self-care behaviours, confirming satisfactory criterion validity. There was a moderate relationship between the PAID-C and the Perceived Stress Scale, supporting satisfactory convergent validity. The PAID-C also demonstrated satisfactory stability and high internal consistency. A four-factor structure and strong content validity of the DCM-C was confirmed. Criterion validity demonstrated that the DCM-C was significantly associated with HbA1c and diabetes self-care behaviours. There was a statistical correlation between the DCM-C and the Revised Ways of Coping Checklist, suggesting satisfactory convergent validity. Test-retest reliability demonstrated satisfactory stability of the DCM-C. The total scale of the DCM-C showed adequate internal consistency. Age, duration of diabetes, diabetes symptoms, diabetes distress, physical activity, coping strategies, and social support were the most consistent factors associated with adaptive outcomes in adults with diabetes. Resilience was positively associated with coping strategies, social support, health-related quality of life, and diabetes self-care behaviours. Results of the structural equation modelling revealed protective factors had a significant direct effect on adaptive outcomes; however, the construct of risk factors was not significantly related to adaptive outcomes. Moreover, resilience can moderate the relationships among protective factors and adaptive outcomes, but there were no interaction effects of risk factors and resilience on adaptive outcomes. Conclusion: This study contributes to an understanding of how risk factors and protective factors work together to influence adaptive outcomes in blood sugar control, health-related quality of life, and diabetes self-care behaviours. Additionally, resilience is a positive personality characteristic and may be importantly involved in the adjustment process among people living with type 2 diabetes.