Formative and reflective measurement and validation mismatch in survey research : an archival analysis of information systems success constructs 1985-2007

The generic IS-success constructs first identified by DeLone and McLean (1992) continue to be widely employed in research. Yet, recent work by Petter et al (2007) has cast doubt on the validity of many mainstream constructs employed in IS research over the past 3 decades; critiquing the almost universal conceptualization and validation of these constructs as reflective when in many studies the measures appear to have been implicitly operationalized as formative. Cited examples of proper specification of the Delone and McLean constructs are few, particularly in light of their extensive employment in IS research. This paper introduces a four-stage formative construct development framework: Conceive > Operationalize > Respond > Validate (CORV). Employing the CORV framework in an archival analysis of research published in top outlets 1985-2007, the paper explores the extent of possible problems with past IS research due to potential misspecification of the four application-related success dimensions: Individual-Impact, Organizational-Impact, System-Quality and Information-Quality. Results suggest major concerns where there is a mismatch of the Respond and Validate stages. A general dearth of attention to the Operationalize and Respond stages in methodological writings is also observed.

Measuring quality : how does this improve construction performance?

The paper charts the history and development of the Hong Kong Housing Department (HKHD) Performance Assessment Scoring System (PASS) from 1990 to the present day and examines its effect on facilitating change to the quality of construction work of building contractors engaged in the production of public sector housing projects Hong Kong. The paper builds partly on empirical research carried out by the author as part of a doctoral thesis from 2000 to 2005, on experiential knowledge and also on some relevant case studies. The outcomes from this earlier research and validation of PASS based on results derived from the system since the research was originally undertaken are of benefit to practitioners and academics working and studying in the areas of performance assessment and organisational culture and change. The conclusions presented in the paper further underpin the connection established in previous research between strong organisational culture and project and corporate success. Organisational culture was measured using a survey instrument originally developed by Denison and Neale (1994), adapted for the environment of the study, and corporate success was measured by the PASS system mentioned above. The major results of the original study indicate that there is significant linkage between strong organisational cultures and business success and the detailed findings were that, (1) strong organisational culture was positively associated a high level of company effectiveness, (2) a high level of company effectiveness was positively associated with the cultural traits of ‘consistency’, ‘adaptability’ and ‘mission’, and (3) a high level of company effectiveness was positively associated with the combined cultural traits represented by the dimensions of ‘external focus’ and ‘stable culture’. Several opportunities to take forward this research have been identified, including extending the study to other countries and also longitudinally re-evaluating some of the original case studies to ascertain how organisational cultures have changed or further developed in relation to the changing construction climate in Hong Kong.

Uncertainty budget in the measurement of typical airborne number, surface area and mass particle distributions

The effects of particulate matter on environment and public health have been widely studied in recent years. A number of studies in the medical field have tried to identify the specific effect on human health of particulate exposure, but agreement amongst these studies on the relative importance of the particles’ size and its origin with respect to health effects is still lacking. Nevertheless, air quality standards are moving, as the epidemiological attention, towards greater focus on the smaller particles. Current air quality standards only regulate the mass of particulate matter less than 10 μm in aerodynamic diameter (PM10) and less than 2.5 μm (PM2.5). The most reliable method used in measuring Total Suspended Particles (TSP), PM10, PM2.5 and PM1 is the gravimetric method since it directly measures PM concentration, guaranteeing an effective traceability to international standards. This technique however, neglects the possibility to correlate short term intra-day variations of atmospheric parameters that can influence ambient particle concentration and size distribution (emission strengths of particle sources, temperature, relative humidity, wind direction and speed and mixing height) as well as human activity patterns that may also vary over time periods considerably shorter than 24 hours. A continuous method to measure the number size distribution and total number concentration in the range 0.014 – 20 μm is the tandem system constituted by a Scanning Mobility Particle Sizer (SMPS) and an Aerodynamic Particle Sizer (APS). In this paper, an uncertainty budget model of the measurement of airborne particle number, surface area and mass size distributions is proposed and applied for several typical aerosol size distributions. The estimation of such an uncertainty budget presents several difficulties due to i) the complexity of the measurement chain, ii) the fact that SMPS and APS can properly guarantee the traceability to the International System of Measurements only in terms of number concentration. In fact, the surface area and mass concentration must be estimated on the basis of separately determined average density and particle morphology. Keywords: SMPS-APS tandem system, gravimetric reference method, uncertainty budget, ultrafine particles.

Tear film surface quality with soft contact lenses using dynamic-area high-speed videokeratoscopy

Objectives. To evaluate the performance of the dynamic-area high-speed videokeratoscopy technique in the assessment of tear film surface quality with and without the presence of soft contact lenses on eye. Methods. Retrospective data from a tear film study using basic high-speed videokeratoscopy, captured at 25 frames per second, (Kopf et al., 2008, J Optom) were used. Eleven subjects had tear film analysis conducted in the morning, midday and evening on the first and seventh day of one week of no lens wear. Five of the eleven subjects then completed an extra week of hydrogel lens wear followed by a week of silicone hydrogel lens wear. Analysis was performed on a 6 second period of the inter-blink recording. The dynamic-area high-speed videokeratoscopy technique uses the maximum available area of Placido ring pattern reflected from the tear interface and eliminates regions of disturbance due to shadows from the eyelashes. A value of tear film surface quality was derived using image rocessing techniques, based on the quality of the reflected ring pattern orientation. Results. The group mean tear film surface quality and the standard deviations for each of the conditions (bare eye, hydrogel lens, and silicone hydrogel lens) showed a much lower coefficient of variation than previous methods (average reduction of about 92%). Bare eye measurements from the right and left eyes of eleven individuals showed high correlation values (Pearson’s correlation r = 0.73, p < 0.05). Repeated measures ANOVA across the 6 second period of measurement in the normal inter-blink period for the bare eye condition showed no statistically significant changes. However, across the 6 second inter-blink period with both contact lenses, statistically significant changes were observed (p < 0.001) for both types of contact lens material. Overall, wearing hydrogel and silicone hydrogel lenses caused the tear film surface quality to worsen compared with the bare eye condition (repeated measures ANOVA, p < 0.0001 for both hydrogel and silicone hydrogel). Conclusions. The results suggest that the dynamic-area method of high-speed videokeratoscopy was able to distinguish and quantify the subtle, but systematic worsening of tear film surface quality in the inter-blink interval in contact lens wear. It was also able to clearly show a difference between bare eye and contact lens wearing conditions.

Robust estimation of tear film surface quality in lateral shearing interferometry

Interferometry is a sensitive technique for recording tear film surface irregularities in a noninvasive manner. At the same time, the technique is hindered by natural eye movements resulting in measurement noise. Estimating tear film surface quality from interferograms can be reduced to a spatial-average-localized weighted estimate of the first harmonic of the interference fringes. However, previously reported estimation techniques proved to perform poorly in cases where the pattern fringes were significantly disturbed. This can occur in cases of measuring tear film surface quality on a contact lens on the eye or in a dry eye. We present a new estimation technique for extracting the first harmonic from the interference fringes that combines the traditional spectral estimation techniques with morphological image processing techniques. The proposed technique proves to be more robust to changes in interference fringes caused by natural eye movements and the degree of dryness of the contact lens and corneal surfaces than its predecessors, resulting in tear film surface quality estimates that are less noisy

Quality issues for the National Bowel Cancer Screening Program (NBCSP)

Purpose: This two-part research project was undertaken as part of the planning process by Queensland Health (QH), Cancer Screening Services Unit (CSSU), Queensland Bowel Cancer Screening Program (QBCSP), in partnership with the National Bowel Cancer Screening Program (NBCSP), to prepare for the implementation of the NBCSP in public sector colonoscopy services in QLD in late 2006. There was no prior information available on the quality of colonoscopy services in Queensland (QLD) and no prior studies that assessed the quality of colonoscopy training in Australia. Furthermore, the NBCSP was introduced without extra funding for colonoscopy service improvement or provision for increases in colonoscopic capacity resulting from the introduction of the NBCSP. The main purpose of the research was to record baseline data on colonoscopy referral and practice in QLD and current training in colonoscopy Australia-wide. It was undertaken from a quality improvement perspective. Implementation of the NBCSP requires that all aspects of the screening pathway, in particular colonoscopy services for the assessment of positive Faecal Occult Blood Tests (FOBTs), will be effective, efficient, equitable and evidence-based. This study examined two important aspects of the continuous quality improvement framework for the NBCSP as they relate to colonoscopy services: (1) evidence-based practice, and (2) quality of colonoscopy training. The Principal Investigator was employed as Senior Project Officer (Training) in the QBCSP during the conduct of this research project. Recommendations from this research have been used to inform the development and implementation of quality improvement initiatives for provision of colonoscopy in the NBCSP, its QLD counterpart the QBCSP and colonoscopy services in QLD, in general. Methods – Part 1 Chart audit of evidence-based practice: The research was undertaken in two parts from 2005-2007. The first part of this research comprised a retrospective chart audit of 1484 colonoscopy records (some 13% of all colonoscopies conducted in public sector facilities in the year 2005) in three QLD colonoscopy services. Whilst some 70% of colonoscopies are currently conducted in the private sector, only public sector colonoscopy facilities provided colonoscopies under the NBCSP. The aim of this study was to compare colonoscopy referral and practice with explicit criteria derived from the National Health & Medical Research Council (NHMRC) (1999) Clinical Practice Guidelines for the Prevention, Early Detection and Management of Colorectal Cancer, and describe the nature of variance with the guidelines. Symptomatic presentations were the most common indication for colonoscopy (60.9%). These comprised per rectal bleeding (31.0%), change of bowel habit (22.1%), abdominal pain (19.6%), iron deficiency anaemia (16.2%), inflammatory bowel disease (8.9%) and other symptoms (11.4%). Surveillance and follow-up colonoscopies accounted for approximately one-third of the remaining colonoscopy workload across sites. Gastroenterologists (GEs) performed relatively more colonoscopies per annum (59.9%) compared to general surgeons (GS) (24.1%), colorectal surgeons (CRS) (9.4%) and general physicians (GPs) (6.5%). Guideline compliance varied with the designation of the colonoscopist. Compliance was lower for CRS (62.9%) compared to GPs (76.0%), GEs (75.0%), GSs (70.9%, p<0.05). Compliance with guideline recommendations for colonoscopic surveillance for family history of colorectal cancer (23.9%), polyps (37.0%) and a past history of bowel cancer (42.7%), was by comparison significantly lower than for symptomatic presentations (94.4%), (p<0.001). Variation with guideline recommendations occurred more frequently for polyp surveillance (earlier than guidelines recommend, 47.9%) and follow-up for past history of bowel cancer (later than recommended, 61.7%, p<0.001). Bowel cancer cases detected at colonoscopy comprised 3.6% of all audited colonoscopies. Incomplete colonoscopies occurred in 4.3% of audited colonoscopies and were more common among women (76.6%). For all colonoscopies audited, the rate of incomplete colonoscopies for GEs was 1.6% (CI 0.9-2.6), GPs 2.0% (CI 0.6-7.2), GS 7.0% (CI 4.8-10.1) and CRS 16.4% (CI 11.2-23.5). 18.6% (n=55) of patients with a documented family history of bowel cancer had colonoscopy performed against guidelines recommendations (for general (category 1) population risk, for reasons of patient request or family history of polyps, rather than for high risk status for colorectal cancer). In general, family history was inadequately documented and subsequently applied to colonoscopy referral and practice. Methods - Part 2 Surveys of quality of colonoscopy training: The second part of the research consisted of Australia-wide anonymous, self-completed surveys of colonoscopy trainers and their trainees to ascertain their opinions on the current apprenticeship model of colonoscopy in Australia and to identify any training needs. Overall, 127 surveys were received from colonoscopy trainers (estimated response rate 30.2%). Approximately 50% of trainers agreed and 27% disagreed that current numbers of training places were adequate to maintain a skilled colonoscopy workforce in preparation for the NBCSP. Approximately 70% of trainers also supported UK-style colonoscopy training within dedicated accredited training centres using a variety of training approaches including simulation. A collaborative approach with the private sector was seen as beneficial by 65% of trainers. Non-gastroenterologists (non-GEs) were more likely than GEs to be of the opinion that simulators are beneficial for colonoscopy training (χ2-test = 5.55, P = 0.026). Approximately 60% of trainers considered that the current requirements for recognition of training in colonoscopy could be insufficient for trainees to gain competence and 80% of those indicated that ≥ 200 colonoscopies were needed. GEs (73.4%) were more likely than non-GEs (36.2%) to be of the opinion that the Conjoint Committee standard is insufficient to gain competence in colonoscopy (χ2-test = 16.97, P = 0.0001). The majority of trainers did not support training either nurses (73%) or GPs in colonoscopy (71%). Only 81 (estimated response rate 17.9%) surveys were received from GS trainees (72.1%), GE trainees (26.3%) and GP trainees (1.2%). The majority were males (75.9%), with a median age 32 years and who had trained in New South Wales (41.0%) or Victoria (30%). Overall, two-thirds (60.8%) of trainees indicated that they deemed the Conjoint Committee standard sufficient to gain competency in colonoscopy. Between specialties, 75.4% of GS trainees indicated that the Conjoint Committee standard for recognition of colonoscopy was sufficient to gain competence in colonoscopy compared to only 38.5% of GE trainees. Measures of competency assessed and recorded by trainees in logbooks centred mainly on caecal intubation (94.7-100%), complications (78.9-100%) and withdrawal time (51-76.2%). Trainees described limited access to colonoscopy training lists due to the time inefficiency of the apprenticeship model and perceived monopolisation of these by GEs and their trainees. Improvements to the current training model suggested by trainees included: more use of simulation, training tools, a United Kingdom (UK)-style training course, concentration on quality indicators, increased access to training lists, accreditation of trainers and interdisciplinary colonoscopy training. Implications for the NBCSP/QBCSP: The introduction of the NBCSP/QBCSP necessitates higher quality colonoscopy services if it is to achieve its ultimate goal of decreasing the incidence of morbidity and mortality associated with bowel cancer in Australia. This will be achieved under a new paradigm for colonoscopy training and implementation of evidence-based practice across the screening pathway and specifically targeting areas highlighted in this thesis. Recommendations for improvement of NBCSP/QBCSP effectiveness and efficiency include the following: 1. Implementation of NBCSP and QBCSP health promotion activities that target men, in particular, to increase FOBT screening uptake. 2. Improved colonoscopy training for trainees and refresher courses or retraining for existing proceduralists to improve completion rates (especially for female NBCSP/QBCSP participants), and polyp and adenoma detection and removal, including newer techniques to detect flat and depressed lesions. 3. Introduction of colonoscopy training initiatives for trainees that are aligned with NBCSP/QBCSP colonoscopy quality indicators, including measurement of training outcomes using objective quality indicators such as caecal intubation, withdrawal time, and adenoma detection rate. 4. Introduction of standardised, interdisciplinary colonoscopy training to reduce apparent differences between specialties with regard to compliance with guideline recommendations, completion rates, and quality of polypectomy. 5. Improved quality of colonoscopy training by adoption of a UK-style training program with centres of excellence, incorporating newer, more objective assessment methods, use of a variety of training tools such as simulation and rotations of trainees between metropolitan, rural, and public and private sector training facilities. 6. Incorporation of NHMRC guidelines into colonoscopy information systems to improve documentation, provide guideline recommendations at the point of care, use of gastroenterology nurse coordinators to facilitate compliance with guidelines and provision of guideline-based colonoscopy referral letters for GPs. 7. Provision of information and education about the NBCSP/QBCSP, bowel cancer risk factors, including family history and polyp surveillance guidelines, for participants, GPs and proceduralists. 8. Improved referral of NBCSP/QBCSP participants found to have a high-risk family history of bowel cancer to appropriate genetics services.

Social Participation of Community Dwelling Stroke Survivors [Published Abstract: From the Quality of Care and Outcomes Research in Cardiovascular Disease and Stroke Conference 2009]

Disability following a stroke can impose various restrictions on patients’ attempts at participating in life roles. The measurement of social participation, for instance, is important in estimating recovery and assessing quality of care at the community level. Thus, the identification of factors influencing social participation is essential in developing effective measures for promoting the reintegration of stroke survivors into the community. Data were collected from 188 stroke survivors (mean age 71.7 years) 12 months after discharge from a stroke rehabilitation hospital. Of these survivors, 128 (61 %) had suffered a first ever stroke, and 81 (43 %) had a right hemisphere lesion. Most (n = 156, 83 %) were living in their own home, though 32 (17 %) were living in residential care facilities. Path analysis was used to test a hypothesized model of participation restriction which included the direct and indirect effects between social, psychological and physical outcomes and demographic variables. Participation restriction was the dependent variable. Exogenous independent variables were age, functional ability, living arrangement and gender. Endogenous independent variables were depressive symptoms, state self-esteem and social support satisfaction. The path coefficients showed functional ability having the largest direct effect on participation restriction. The results also showed that more depressive symptoms, low state self-esteem, female gender, older age and living in a residential care facility had a direct effect on participation restriction. The explanatory variables accounted for 71% of the variance in explaining participation restriction. Prediction models have empirical and practical applications such as suggesting important factors to be considered in promoting stroke recovery. The findings suggest that interventions offered over the course of rehabilitation should be aimed at improving functional ability and promoting psychological aspects of recovery. These are likely to enhance stroke survivors resume or maximize their social participation so that they may fulfill productive and positive life roles.

Development of a Systematic Approach to Assessing Quality within Australian Residential Aged Care Facilities: The Clinical Care Indicators Tool

Recent years have seen the introduction of formalised accreditation processes in both community and residential aged care, but these only partially address quality assessment within this sector. Residential aged care in Australia does not yet have a standardised system of resident assessment related to clinical, rather than administrative, outcomes. This paper describes the development of a quality assessment tool aimed at addressing this gap. Utilising previous research and the results of nominal groups with experts in the field, the 21-item Clinical Care Indicators (CCI) Tool for residential aged care was developed and trialled nationally. The CCI Tool was found to be simple to use and an effective means of collecting data on the state of resident health and care, with potential benefits for resident care planning and continuous quality improvement within facilities and organisations. The CCI Tool was further refined through a small intervention study to assess its utility as a quality improvement instrument and to investigate its relationship with resident quality of life. The current version covers 23 clinical indicators, takes about 30 minutes to complete and is viewed favourably by nursing staff who use it. Current work focuses on psychometric analysis and benchmarking, which should enable the CCI Tool to make a positive contribution to the measurement of quality in aged care in Australia.

Perceptions of daylight quality delivered by light transport systems

Light Transport Systems (LTS) (e.g lightpipes, fibre optics) can illuminate core areas within buildings with great potential for energy savings. However, they do not provide a clear connection to the outside like windows do, and their effects on people’s physiological and psychological health are not well understood. Furthermore, how people perceive LTS affects users’ acceptance of the device and its performance. The purpose of this research is to understand how occupants perceive and experience spaces illuminated by LTS. Two case studies of commercial buildings with LTS, located in Brisbane, Australia are assessed by qualitative (focus group interviews) and quantitative (measurement of daylight illuminances and luminance) methods. The data from interviews with occupants provide useful insight into the aspects of LTS design that are most relevant to positive perception of the luminous environment. Luminance measurements of the occupied spaces support the perception of the LTS reported by occupants: designs that create high contrast luminous environments are more likely to be perceived negatively.

Widening the gap between measurement and modelling of secondary organic aerosol properties?

The link between measured sub-saturated hygroscopicity and cloud activation potential of secondary organic aerosol particles produced by the chamber photo-oxidation of α-pinene in the presence or absence of ammonium sulphate seed aerosol was investigated using two models of varying complexity. A simple single hygroscopicity parameter model and a more complex model (incorporating surface effects) were used to assess the detail required to predict the cloud condensation nucleus (CCN) activity from the subsaturated water uptake. Sub-saturated water uptake measured by three hygroscopicity tandem differential mobility analyser (HTDMA) instruments was used to determine the water activity for use in the models. The predicted CCN activity was compared to the measured CCN activation potential using a continuous flow CCN counter. Reconciliation using the more complex model formulation with measured cloud activation could be achieved widely different assumed surface tension behavior of the growing droplet; this was entirely determined by the instrument used as the source of water activity data. This unreliable derivation of the water activity as a function of solute concentration from sub-saturated hygroscopicity data indicates a limitation in the use of such data in predicting cloud condensation nucleus behavior of particles with a significant organic fraction. Similarly, the ability of the simpler single parameter model to predict cloud activation behaviour was dependent on the instrument used to measure sub-saturated hygroscopicity and the relative humidity used to provide the model input. However, agreement was observed for inorganic salt solution particles, which were measured by all instruments in agreement with theory. The difference in HTDMA data from validated and extensively used instruments means that it cannot be stated with certainty the detail required to predict the CCN activity from sub-saturated hygroscopicity. In order to narrow the gap between measurements of hygroscopic growth and CCN activity the processes involved must be understood and the instrumentation extensively quality assured. It is impossible to say from the results presented here due to the differences in HTDMA data whether: i) Surface tension suppression occurs ii) Bulk to surface partitioning is important iii) The water activity coefficient changes significantly as a function of the solute concentration.

Derivation of motor vehicle tailpipe particle emission factors suitable for modelling urban fleet emissions and air quality assessments

Background, aim, and scope Urban motor vehicle fleets are a major source of particulate matter pollution, especially of ultrafine particles (diameters < 0.1 µm), and exposure to particulate matter has known serious health effects. A considerable body of literature is available on vehicle particle emission factors derived using a wide range of different measurement methods for different particle sizes, conducted in different parts of the world. Therefore the choice as to which are the most suitable particle emission factors to use in transport modelling and health impact assessments presented as a very difficult task. The aim of this study was to derive a comprehensive set of tailpipe particle emission factors for different vehicle and road type combinations, covering the full size range of particles emitted, which are suitable for modelling urban fleet emissions. Materials and methods A large body of data available in the international literature on particle emission factors for motor vehicles derived from measurement studies was compiled and subjected to advanced statistical analysis, to determine the most suitable emission factors to use in modelling urban fleet emissions. Results This analysis resulted in the development of five statistical models which explained 86%, 93%, 87%, 65% and 47% of the variation in published emission factors for particle number, particle volume, PM1, PM2.5 and PM10 respectively. A sixth model for total particle mass was proposed but no significant explanatory variables were identified in the analysis. From the outputs of these statistical models, the most suitable particle emission factors were selected. This selection was based on examination of the statistical robustness of the statistical model outputs, including consideration of conservative average particle emission factors with the lowest standard errors, narrowest 95% confidence intervals and largest sample sizes, and the explanatory model variables, which were Vehicle Type (all particle metrics), Instrumentation (particle number and PM2.5), Road Type (PM10) and Size Range Measured and Speed Limit on the Road (particle volume). Discussion A multiplicity of factors need to be considered in determining emission factors that are suitable for modelling motor vehicle emissions, and this study derived a set of average emission factors suitable for quantifying motor vehicle tailpipe particle emissions in developed countries. Conclusions The comprehensive set of tailpipe particle emission factors presented in this study for different vehicle and road type combinations enable the full size range of particles generated by fleets to be quantified, including ultrafine particles (measured in terms of particle number). These emission factors have particular application for regions which may have a lack of funding to undertake measurements, or insufficient measurement data upon which to derive emission factors for their region. Recommendations and perspectives In urban areas motor vehicles continue to be a major source of particulate matter pollution and of ultrafine particles. It is critical that in order to manage this major pollution source methods are available to quantify the full size range of particles emitted for traffic modelling and health impact assessments.

Reference bias : presentation of extreme health states prior to EQ-VAS improves health-related quality of life scores. A randomised crossover trial.

Background: Clinical practice and clinical research has made a concerted effort to move beyond the use of clinical indicators alone and embrace patient focused care through the use of patient reported outcomes such as healthrelated quality of life. However, unless patients give consistent consideration to the health states that give meaning to measurement scales used to evaluate these constructs, longitudinal comparison of these measures may be invalid. This study aimed to investigate whether patients give consideration to a standard health state rating scale (EQ-VAS) and whether consideration of good and poor health state descriptors immediately changes their selfreport. Methods: A randomised crossover trial was implemented amongst hospitalised older adults (n = 151). Patients were asked to consider descriptions of extremely good (Description-A) and poor (Description-B) health states. The EQ-VAS was administered as a self-report at baseline, after the first descriptors (A or B), then again after the remaining descriptors (B or A respectively). At baseline patients were also asked if they had considered either EQVAS anchors. Results: Overall 106/151 (70%) participants changed their self-evaluation by ≥5 points on the 100 point VAS, with a mean (SD) change of +4.5 (12) points (p < 0.001). A total of 74/151 (49%) participants did not consider the best health VAS anchor, of the 77 who did 59 (77%) thought the good health descriptors were more extreme (better) then they had previously considered. Similarly 85/151 (66%) participants did not consider the worst health anchor of the 66 who did 63 (95%) thought the poor health descriptors were more extreme (worse) then they had previously considered. Conclusions: Health state self-reports may not be well considered. An immediate significant shift in response can be elicited by exposure to a mere description of an extreme health state despite no actual change in underlying health state occurring. Caution should be exercised in research and clinical settings when interpreting subjective patient reported outcomes that are dependent on brief anchors for meaning. Trial Registration: Australian and New Zealand Clinical Trials Registry (#ACTRN12607000606482) http://www.anzctr. org.au

Clinical indicators of quality for Australian residential aged care facilities : establishing reliability, validity, and quality thresholds

Background: In response to the need for more comprehensive quality assessment within Australian residential aged care facilities, the Clinical Care Indicator (CCI) Tool was developed to collect outcome data as a means of making inferences about quality. A national trial of its effectiveness and a Brisbane-based trial of its use within the quality improvement context determined the CCI Tool represented a potentially valuable addition to the Australian aged care system. This document describes the next phase in the CCI Tool.s development; the aims of which were to establish validity and reliability of the CCI Tool, and to develop quality indicator thresholds (benchmarks) for use in Australia. The CCI Tool is now known as the ResCareQA (Residential Care Quality Assessment). Methods: The study aims were achieved through a combination of quantitative data analysis, and expert panel consultations using modified Delphi process. The expert panel consisted of experienced aged care clinicians, managers, and academics; they were initially consulted to determine face and content validity of the ResCareQA, and later to develop thresholds of quality. To analyse its psychometric properties, ResCareQA forms were completed for all residents (N=498) of nine aged care facilities throughout Queensland. Kappa statistics were used to assess inter-rater and test-retest reliability, and Cronbach.s alpha coefficient calculated to determine internal consistency. For concurrent validity, equivalent items on the ResCareQA and the Resident Classification Scales (RCS) were compared using Spearman.s rank order correlations, while discriminative validity was assessed using known-groups technique, comparing ResCareQA results between groups with differing care needs, as well as between male and female residents. Rank-ordered facility results for each clinical care indicator (CCI) were circulated to the panel; upper and lower thresholds for each CCI were nominated by panel members and refined through a Delphi process. These thresholds indicate excellent care at one extreme and questionable care at the other. Results: Minor modifications were made to the assessment, and it was renamed the ResCareQA. Agreement on its content was reached after two Delphi rounds; the final version contains 24 questions across four domains, enabling generation of 36 CCIs. Both test-retest and inter-rater reliability were sound with median kappa values of 0.74 (test-retest) and 0.91 (inter-rater); internal consistency was not as strong, with a Chronbach.s alpha of 0.46. Because the ResCareQA does not provide a single combined score, comparisons for concurrent validity were made with the RCS on an item by item basis, with most resultant correlations being quite low. Discriminative validity analyses, however, revealed highly significant differences in total number of CCIs between high care and low care groups (t199=10.77, p=0.000), while the differences between male and female residents were not significant (t414=0.56, p=0.58). Clinical outcomes varied both within and between facilities; agreed upper and lower thresholds were finalised after three Delphi rounds. Conclusions: The ResCareQA provides a comprehensive, easily administered means of monitoring quality in residential aged care facilities that can be reliably used on multiple occasions. The relatively modest internal consistency score was likely due to the multi-factorial nature of quality, and the absence of an aggregate result for the assessment. Measurement of concurrent validity proved difficult in the absence of a gold standard, but the sound discriminative validity results suggest that the ResCareQA has acceptable validity and could be confidently used as an indication of care quality within Australian residential aged care facilities. The thresholds, while preliminary due to small sample size, enable users to make judgements about quality within and between facilities. Thus it is recommended the ResCareQA be adopted for wider use.

Non-invasive assessment of tear film surface quality

The tear film plays an important role preserving the health of the ocular surface and maintaining the optimal refractive power of the cornea. Moreover dry eye syndrome is one of the most commonly reported eye health problems. This syndrome is caused by abnormalities in the properties of the tear film. Current clinical tools to assess the tear film properties have shown certain limitations. The traditional invasive methods for the assessment of tear film quality, which are used by most clinicians, have been criticized for the lack of reliability and/or repeatability. A range of non-invasive methods of tear assessment have been investigated, but also present limitations. Hence no “gold standard” test is currently available to assess the tear film integrity. Therefore, improving techniques for the assessment of the tear film quality is of clinical significance and the main motivation for the work described in this thesis. In this study the tear film surface quality (TFSQ) changes were investigated by means of high-speed videokeratoscopy (HSV). In this technique, a set of concentric rings formed in an illuminated cone or a bowl is projected on the anterior cornea and their reflection from the ocular surface imaged on a charge-coupled device (CCD). The reflection of the light is produced in the outer most layer of the cornea, the tear film. Hence, when the tear film is smooth the reflected image presents a well structure pattern. In contrast, when the tear film surface presents irregularities, the pattern also becomes irregular due to the light scatter and deviation of the reflected light. The videokeratoscope provides an estimate of the corneal topography associated with each Placido disk image. Topographical estimates, which have been used in the past to quantify tear film changes, may not always be suitable for the evaluation of all the dynamic phases of the tear film. However the Placido disk image itself, which contains the reflected pattern, may be more appropriate to assess the tear film dynamics. A set of novel routines have been purposely developed to quantify the changes of the reflected pattern and to extract a time series estimate of the TFSQ from the video recording. The routine extracts from each frame of the video recording a maximized area of analysis. In this area a metric of the TFSQ is calculated. Initially two metrics based on the Gabor filter and Gaussian gradient-based techniques, were used to quantify the consistency of the pattern’s local orientation as a metric of TFSQ. These metrics have helped to demonstrate the applicability of HSV to assess the tear film, and the influence of contact lens wear on TFSQ. The results suggest that the dynamic-area analysis method of HSV was able to distinguish and quantify the subtle, but systematic degradation of tear film surface quality in the inter-blink interval in contact lens wear. It was also able to clearly show a difference between bare eye and contact lens wearing conditions. Thus, the HSV method appears to be a useful technique for quantitatively investigating the effects of contact lens wear on the TFSQ. Subsequently a larger clinical study was conducted to perform a comparison between HSV and two other non-invasive techniques, lateral shearing interferometry (LSI) and dynamic wavefront sensing (DWS). Of these non-invasive techniques, the HSV appeared to be the most precise method for measuring TFSQ, by virtue of its lower coefficient of variation. While the LSI appears to be the most sensitive method for analyzing the tear build-up time (TBUT). The capability of each of the non-invasive methods to discriminate dry eye from normal subjects was also investigated. The receiver operating characteristic (ROC) curves were calculated to assess the ability of each method to predict dry eye syndrome. The LSI technique gave the best results under both natural blinking conditions and in suppressed blinking conditions, which was closely followed by HSV. The DWS did not perform as well as LSI or HSV. The main limitation of the HSV technique, which was identified during the former clinical study, was the lack of the sensitivity to quantify the build-up/formation phase of the tear film cycle. For that reason an extra metric based on image transformation and block processing was proposed. In this metric, the area of analysis was transformed from Cartesian to Polar coordinates, converting the concentric circles pattern into a quasi-straight lines image in which a block statistics value was extracted. This metric has shown better sensitivity under low pattern disturbance as well as has improved the performance of the ROC curves. Additionally a theoretical study, based on ray-tracing techniques and topographical models of the tear film, was proposed to fully comprehend the HSV measurement and the instrument’s potential limitations. Of special interested was the assessment of the instrument’s sensitivity under subtle topographic changes. The theoretical simulations have helped to provide some understanding on the tear film dynamics, for instance the model extracted for the build-up phase has helped to provide some insight into the dynamics during this initial phase. Finally some aspects of the mathematical modeling of TFSQ time series have been reported in this thesis. Over the years, different functions have been used to model the time series as well as to extract the key clinical parameters (i.e., timing). Unfortunately those techniques to model the tear film time series do not simultaneously consider the underlying physiological mechanism and the parameter extraction methods. A set of guidelines are proposed to meet both criteria. Special attention was given to a commonly used fit, the polynomial function, and considerations to select the appropriate model order to ensure the true derivative of the signal is accurately represented. The work described in this thesis has shown the potential of using high-speed videokeratoscopy to assess tear film surface quality. A set of novel image and signal processing techniques have been proposed to quantify different aspects of the tear film assessment, analysis and modeling. The dynamic-area HSV has shown good performance in a broad range of conditions (i.e., contact lens, normal and dry eye subjects). As a result, this technique could be a useful clinical tool to assess tear film surface quality in the future.

Evaluation of financial incentives as a quality improvement strategy in the public hospital context : clinician attitudes, design variables, and economic costs

In 2008, a three-year pilot ‘pay for performance’ (P4P) program, known as ‘Clinical Practice Improvement Payment’ (CPIP) was introduced into Queensland Health (QHealth). QHealth is a large public health sector provider of acute, community, and public health services in Queensland, Australia. The organisation has recently embarked on a significant reform agenda including a review of existing funding arrangements (Duckett et al., 2008). Partly in response to this reform agenda, a casemix funding model has been implemented to reconnect health care funding with outcomes. CPIP was conceptualised as a performance-based scheme that rewarded quality with financial incentives. This is the first time such a scheme has been implemented into the public health sector in Australia with a focus on rewarding quality, and it is unique in that it has a large state-wide focus and includes 15 Districts. CPIP initially targeted five acute and community clinical areas including Mental Health, Discharge Medication, Emergency Department, Chronic Obstructive Pulmonary Disease, and Stroke. The CPIP scheme was designed around key concepts including the identification of clinical indicators that met the set criteria of: high disease burden, a well defined single diagnostic group or intervention, significant variations in clinical outcomes and/or practices, a good evidence, and clinician control and support (Ward, Daniels, Walker & Duckett, 2007). This evaluative research targeted Phase One of implementation of the CPIP scheme from January 2008 to March 2009. A formative evaluation utilising a mixed methodology and complementarity analysis was undertaken. The research involved three research questions and aimed to determine the knowledge, understanding, and attitudes of clinicians; identify improvements to the design, administration, and monitoring of CPIP; and determine the financial and economic costs of the scheme. Three key studies were undertaken to ascertain responses to the key research questions. Firstly, a survey of clinicians was undertaken to examine levels of knowledge and understanding and their attitudes to the scheme. Secondly, the study sought to apply Statistical Process Control (SPC) to the process indicators to assess if this enhanced the scheme and a third study examined a simple economic cost analysis. The CPIP Survey of clinicians elicited 192 clinician respondents. Over 70% of these respondents were supportive of the continuation of the CPIP scheme. This finding was also supported by the results of a quantitative altitude survey that identified positive attitudes in 6 of the 7 domains-including impact, awareness and understanding and clinical relevance, all being scored positive across the combined respondent group. SPC as a trending tool may play an important role in the early identification of indicator weakness for the CPIP scheme. This evaluative research study supports a previously identified need in the literature for a phased introduction of Pay for Performance (P4P) type programs. It further highlights the value of undertaking a formal risk assessment of clinician, management, and systemic levels of literacy and competency with measurement and monitoring of quality prior to a phased implementation. This phasing can then be guided by a P4P Design Variable Matrix which provides a selection of program design options such as indicator target and payment mechanisms. It became evident that a clear process is required to standardise how clinical indicators evolve over time and direct movement towards more rigorous ‘pay for performance’ targets and the development of an optimal funding model. Use of this matrix will enable the scheme to mature and build the literacy and competency of clinicians and the organisation as implementation progresses. Furthermore, the research identified that CPIP created a spotlight on clinical indicators and incentive payments of over five million from a potential ten million was secured across the five clinical areas in the first 15 months of the scheme. This indicates that quality was rewarded in the new QHealth funding model, and despite issues being identified with the payment mechanism, funding was distributed. The economic model used identified a relative low cost of reporting (under $8,000) as opposed to funds secured of over $300,000 for mental health as an example. Movement to a full cost effectiveness study of CPIP is supported. Overall the introduction of the CPIP scheme into QHealth has been a positive and effective strategy for engaging clinicians in quality and has been the catalyst for the identification and monitoring of valuable clinical process indicators. This research has highlighted that clinicians are supportive of the scheme in general; however, there are some significant risks that include the functioning of the CPIP payment mechanism. Given clinician support for the use of a pay–for-performance methodology in QHealth, the CPIP scheme has the potential to be a powerful addition to a multi-faceted suite of quality improvement initiatives within QHealth.