A systematic review of reliable and valid tools for the measurement of patient participation in healthcare

INTRODUCTION: Patient participation in healthcare is recognised internationally as essential for consumer-centric, high-quality healthcare delivery. Its measurement as part of continuous quality improvement requires development of agreed standards and measurable indicators. AIM: This systematic review sought to identify strategies to measure patient participation in healthcare and to report their reliability and validity. In the context of this review, patient participation was constructed as shared decision-making, acknowledging the patient as having critical knowledge regarding their own health and care needs and promoting self-care/autonomy. METHODS: Following a comprehensive search, studies reporting reliability or validity of an instrument used in a healthcare setting to measure patient participation, published in English between January 2004 and March 2014 were eligible for inclusion. RESULTS: From an initial search, which identified 1582 studies, 156 studies were retrieved and screened against inclusion criteria. Thirty-three studies reporting 24 patient participation measurement tools met inclusion criteria, and were critically appraised. The majority of studies were descriptive psychometric studies using prospective, cross-sectional designs. Almost all the tools completed by patients, family caregivers, observers or more than one stakeholder focused on aspects of patient-professional communication. Few tools designed for completion by patients or family caregivers provided valid and reliable measures of patient participation. There was low correlation between many of the tools and other measures of patient satisfaction. CONCLUSION: Few reliable and valid tools for measurement of patient participation in healthcare have been recently developed. Of those reported in this review, the dyadic Observing Patient Involvement in Decision Making (dyadic-OPTION) tool presents the most promise for measuring core components of patient participation. There remains a need for further study into valid, reliable and feasible strategies for measuring patient participation as part of continuous quality improvement.

Perioperative mortality rates in Australian public hospitals : the influence of age, gender and urgency

INTRODUCTION: A decline in surgical deaths has been observed in Australia since the introduction of the Australian and New Zealand Audit of Surgical Mortality (ANZASM). The current study was conducted to determine whether the perioperative mortality rate (POMR) has also declined. METHODS: This study is a retrospective review of the POMR for surgical procedures in Australian public hospitals between July 2009 and June 2013, using data obtained from the Australian Institute of Health and Welfare. Operative procedures contained in the Australian Refined Diagnosis Related Groups were selected and the POMR was modelled using urgency of admission, age and gender as explanatory covariates. RESULTS: The POMR in Australian public hospitals reduced by 15.4 % over the 4-year period. The emergency admissions POMR dropped from 1.40 to 1.12 %, and the elective admissions POMR from 0.09 to 0.08 %. The binary logistic regression model used to predict patient mortality showed emergency admissions to have a higher POMR than elective, being more evident at older ages. For emergency admissions, the difference in POMR between females and males increased with age, from about 55 years onwards, with females being lower. For elective surgeries, the difference between males and females was of little practical importance across ages. CONCLUSIONS: The reduction in the POMR in Australia confirms the reduction in surgical deaths reported to ANZASM. Continuing to monitor POMR will be important to ensure the safest surgery in Australia. Further investigations into case-mix will allow better risk adjustment and comparison between regions and time-periods, to facilitate continuous quality improvement.

Obesity prevention programs demand high-quality evaluations

Obesity prevention programs are at last underway or being planned in Australia and New Zealand. However, it is imperative that they are well-evaluated so that they can contribute to continuous program improvement and add much-needed evidence to the international literature on what works and does not work to prevent obesity. Three critical components of program evaluation are especially at risk when the funding comes from service delivery rather than research sources. These are: the need for comparison groups; the need for measured height and weight; and the need for sufficient process and context information. There is an important opportunity to build collaborative mechanisms across community-based obesity prevention sites to enhance the program and evaluation quality and to accelerate knowledge translation into practice and policy.

Towards development of a quality cost model for automotive stamping

The current work used discrete event simulation techniques to model the economics of quality within an actual automotive stamping plant. Automotive stamping is a complex, capital intensive process requiring part-specific tooling and specialised machinery. Quality control and quality improvement is difficult in the stamping environment due to the general lack of process understanding and the large number to interacting variables. These factors have prevented the widespread use of statistical process control. In this work, a model of the quality control techniques used at the Ford Geelong Stamping plant is developed and indirectly validated against results from production. To date, most discrete event models are of systems where the quality control process is clearly defined by the rules of statistical process control. However, the quality control technique used within the stamping plant is for the operator to perform a 100% visual inspection while unloading the finished panels. In the developed model, control is enacted after a cumulative count of defective items is observed, thereby approximating the operator who allows a number of defective panels to accumulate before resetting the line. Analysis of this model found that the cost sensitivity to inspection error is dependent upon the level of control and that the level of control determines line utilisation. Additional analysis of this model demonstrated that additional inspection processes would lead to more stable cost structures but these structures many not necessarily be lower cost. The model was subsequently applied to investigate the economics of quality improvement. The quality problem of panel blemishes, induced by slivers (small metal fragments), was chosen as a case stuffy. Errors of 20-30% were observed during direct validation of the cost model and it was concluded that the use of discrete event simulation models for applications requiring high accuracy would not be possible unless the production system was of low complexity. However, the model could be used to evaluate the sensitivity of input factors and investigating the effects of a number of potential improvement opportunities. Therefore, the research concluded that it is possible to use discrete event simulation to determine the quality economics of an actual stamping plant. However, limitations imposed by inability of the model to consider a number of external factors, such as continuous improvement, operator working conditions or wear and the lack of reliable quality data, result in low cost accuracy. Despite this, it still can be demonstrated that discrete event simulation has significant benefits over the alternate modelling methods.

From quality assurance to process improvement : an action research project

This action research project was undertaken at the Casino & District Memorial Hospital in northern N.S.W. during 1995 & 1996. The purpose was to utilise the Action Research frameork to enable the participants to improve their problem solving skills in relation to quality improvement strategies within the Nursing Division.

Towards a semantic quality based approach for business process models improvement

Business process (BP) modeling aims at a better understanding of processes, allowing deciders to improve them. We propose to support this modeling with an approach encompassing methods and tools for BP models quality measurement and improvement. In this paper we focus on semantic quality. The latter is evaluated by aligning BP model concepts with domain knowledge. The alignment is conducted thanks to meta-models. We also define validation rules for checking the completeness of BP models. A medical case study illustrates the main steps of our approach.

Another look at the culture-quality-performance link

Quality improvement is usually driven by quality, safety and risk agendas leading to a focus on measurements of the outputs of care; outputs such as fewer complaints, fewer accidents and adverse events. An oft-neglected theme is the impact of the quality improvement initiative within the organisation itself. This paper presents the findings of the first stage of an evaluation that has examined the changes which have occurred within organisations since participating in a quality improvement initiative. These findings indicate that engaging with a quality improvement program can change the nature of social interactions within the organisation. In this way, quality improvement programs can impact on organisational culture, particularly in relation to organisational learning. Thus, this paper argues that successful engagement with a quality improvement program can enhance organisational learning, and, in turn, build organisational capacity.

Public reporting of nursing home quality of care : Lessons from the United States experience for Canadian policy discussion

While the demand for continuing care services in Canada grows, the quality of such services has come under increasing scrutiny. Consideration has been given to the use of public reporting of quality data as a mechanism to stimulate quality improvement and promote public accountability for and transparency in service quality. The recent adoption of the Resident Assessment Instrument (RAI) throughout a number of Canadian jurisdictions means that standardized quality data are available for comparisons among facilities across regions, provinces and nationally. In this paper, we explore current knowledge on public reporting in nursing homes in the United States to identify what lessons may inform policy discussion regarding potential use of public reporting in Canada. Based on these findings, we make recommendations regarding how public reporting should be progressed and managed if Canadian jurisdictions were to implement this strategy.

Acceptability, impact and sustainability of the national pilot quality assurance system for self management programs

Chronic condition self-management education and training interventions such as the Stanford Self Management Programs (SMP) have the capacity to improve health and quality of life of people with chronic conditions whilst reducing the use of health services. This is in line with the outcomes from the recent Council of Australian Governments’ meeting where it was indicated that self-management will be a centrepiece in forthcoming chronic disease initiatives.
Aim: report on a large national pilot quality assurance program involving the implementation and of an evaluation and quality monitoring system for SMPs including the provision of structured feedback to courses course leaders and service providers. During 2005/06 the quality assurance program was implemented at 11 diverse organisations across Australia. The program involved assisting organisations apply the 42-item Health Education Impact Questionnaire (HEIQ), a chronic disease health education outcome measure, and then observe and evaluate the value and impact of the quality program. Interviews with course leaders (n=60) and course participants (n=35) have elicited views about course quality and feedback processes.
Results: The evaluation revealed enablers and barriers to effective implementation and sustainability. Important enablers were:
- Course Leaders and organisations valued an Australia-wide system that provided feedback on course
quality and the impact on participants.
- Course Leaders were strongly personally motivated to respond appropriately to HEI-Q course
report feedback.
- Completing the questionnaire provided participants with the opportunity to reflect on issues that
emerge in the course content and reflect on their progression at the end of the SSMP.
Sustainability issues included:
- Organisations and course leaders require support, training and flexibility on how to administer and
manage the use of the HEI-Q.
- Availability of administrative resources in organisations to support the quality assurance activities.
- The requirement that course leaders are trained in interpreting HEI-Q course report data.
A quality improvement framework was developed which identified the actions required of key stakeholders to
support effective implementation.
Discussion: With the increasing endorsement of SMP across sectors it is important that course quality is known, is acceptable, and is communicated to stakeholders to inform and engender confidence in the SSMP. To effectively implement and sustain a quality improvement program for SMP, the processes and tools for measuring outcomes need to be responsive, flexible and easily integrated into the organisation and delivery of programs.

Evaluation of individual program plan content and quality in a Victorian residential service

The study assesses the quality and content of Individual Program Plans for 284 Victorian adults with profound intellectual disabilities living in one residential service. A checklist was developed for improving plan quality, and measurement difficulties with checklists discussed. Recommends that a quality improvement process be implemented.

The resident assessment instrument-minimum data set 2.0 quality indicators : a systematic review

Background
The Resident Assessment Instrument-Minimum Data Set (RAI-MDS) 2.0 is designed to collect the minimum amount of data to guide care planning and monitoring for residents in long-term care settings. These data have been used to compute indicators of care quality. Use of the quality indicators to inform quality improvement initiatives is contingent upon the validity and reliability of the indicators. The purpose of this review was to systematically examine published and grey research reports in order to assess the state of the science regarding the validity and reliability of the RAI-MDS 2.0 Quality Indicators (QIs).

Methods
We systematically reviewed the evidence for the validity and reliability of the RAI-MDS 2.0 QIs. A comprehensive literature search identified relevant original research published, in English, prior to December 2008. Fourteen articles and one report examining the validity and/or reliability of the RAI-MDS 2.0 QIs were included.

Results
The studies fell into two broad categories, those that examined individual quality indicators and those that examined multiple indicators. All studies were conducted in the United States and included from one to a total of 209 facilities. The number of residents included in the studies ranged from 109 to 5758. One study conducted under research conditions examined 38 chronic care QIs, of which strong evidence for the validity of 12 of the QIs was found. In response to these findings, the 12 QIs were recommended for public reporting purposes. However, a number of observational studies (n=13), conducted in "real world" conditions, have tested the validity and/or reliability of individual QIs, with mixed results. Ten QIs have been studied in this manner, including falls, depression, depression without treatment, urinary incontinence, urinary tract infections, weight loss, bedfast, restraint, pressure ulcer, and pain. These studies have revealed the potential for systematic bias in reporting, with under-reporting of some indicators and over-reporting of others.

Conclusion
Evidence for the reliability and validity of the RAI-MDS QIs remains inconclusive. The QIs provide a useful tool for quality monitoring and to inform quality improvement programs and initiatives. However, caution should be exercised when interpreting the QI results and other sources of evidence of the quality of care processes should be considered in conjunction with QI results.

Discovering the value that the library can contribute to quality teaching and learning through student evaluation data

Large surveys of library user service quality perception are common. However, student evaluation of teaching (SET) data often show a disparity between ratings of library service quality and library resource quality. In this situation, perhaps SET data can also provide insights into what contributes to the perception of library resource quality, and hence identify leverage points for quality improvement interventions. This paper documents an analysis of available Deakin University SET data relating to student interaction with, and evaluation of, library resources. It highlights significant correlations associated with library-related SET items, and from them infers actions that the library could undertake to improve the value and perception of the quality of library resources. The following results were observed. High ratings for library resources were likely to be associated with high general ratings of teaching and unit quality. Postgraduate coursework students rated library resources significantly higher than students in the first three years of undergraduate programs. Students in one faculty (Health) rated library resources significantly higher than students in all other faculties. There was a strong correlation observed in Australasian Survey of Student Engagement data for both 2009 and 2010 between the two items “Used library resources on campus or online” and “Worked on an essay or assignment that required integrating ideas or information from various sources”. These findings suggest the following conclusions. Well-planned learning environments are likely to integrate meaningful student interaction with the library. Initiatives to improve the value and perception of the quality of library resources should be focussed on the specific characteristics and needs of particular student cohorts to have maximum impact. More sophisticated assessment tasks that require students to interact with the library have the potential to result in higher student ratings of the value of library resources.

Facilitating best practice : transferring the lessons of the Clinical Support Systems program

The Clinical Support Systems Program (CSSP) includes the management of clinical practice using clinical and consumer pathways, outcome and performance indicators, clinical measurement and review in a continuous improvement cycle using the best available extant evidence. The Royal Australasian College of Physicians is testing the CSSP model through four consortia around Australia. There are 17 project sites in three States. The funded projects address major clinical problems including congestive heart failure and acute coronary syndromes, acute stroke management, and colorectal cancer care. There is some early evidence of the CSSP influencing change in areas beyond the bounds of the project settings and the College has developed a plan to promote wider adoption of best practice. This approach recognises the College’s role in providing Fellows with the practical tools of quality improvement, the means to collect data and compare their practice to other clinicians, while traversing the appropriate educational framework.

Perceptions of quality in higher education : a comparative study of Turkish and Australian business academics

Finding a common definition of ‘quality’ in studies of quality and quality improvement in higher education institutions is very important. This study identifies the views of a key stakeholder group, academics, with reference to their beliefs (what is currently occurring) and their attitudes (what ought to be occurring) in relation to quality in their departments. The focus of this paper is on the collection of data from 64 business administration academics in Turkish universities. Face-to-face interviews were conducted using an instrument titled ‘Quality in Accounting Education Survey’. The questionnaire was developed by Watty and is based on the conceptions of quality framework, developed by Harvey and Green. The results are compared with the beliefs and attitudes of Australian accounting academics as reported in an earlier paper by Watty. The findings show that academics from Turkey adopt the perspective of quality as excellent or élitist, both in their beliefs (current situation perception) and in their attitudes (desired situation perception). This compares with the findings that Australian academics’ attitudes reflect a quality perspective as fitness for purpose in the current situation and beliefs that reflect a transformational quality perspective as the desired situation (what ought to be).