Routine Measurement of Health-Related Quality of Life in Assessing Cost-Effectiveness in Secondary Health Care

Aims: The aims of this study were 1) to identify and describe health economic studies that have used quality-adjusted life years (QALYs) based on actual measurements of patients' health-related quality of life (HRQoL); 2) to test the feasibility of routine collection of health-related quality of life (HRQoL) data as an indicator of effectiveness of secondary health care; and 3) to establish and compare the cost-utility of three large-volume surgical procedures in a real-world setting in the Helsinki University Central Hospital, a large referral hospital providing secondary and tertiary health-care services for a population of approximately 1.4 million. Patients and methods: So as to identify studies that have used QALYs as an outcome measure, a systematic search of the literature was performed using the Medline, Embase, CINAHL, SCI and Cochrane Library electronic databases. Initial screening of the identified articles involved two reviewers independently reading the abstracts; the full-text articles were also evaluated independently by two reviewers, with a third reviewer used in cases where the two reviewers could not agree a consensus on which articles should be included. The feasibility of routinely evaluating the cost-effectiveness of secondary health care was tested by setting up a system for collecting HRQoL data on approximately 4 900 patients' HRQoL before and after operative treatments performed in the hospital. The HRQoL data used as an indicator of treatment effectiveness was combined with diagnostic and financial indicators routinely collected in the hospital. To compare the cost-effectiveness of three surgical interventions, 712 patients admitted for routine operative treatment completed the 15D HRQoL questionnaire before and also 3-12 months after the operation. QALYs were calculated using the obtained utility data and expected remaining life years of the patients. Direct hospital costs were obtained from the clinical patient administration database of the hospital and a cost-utility analysis was performed from the perspective of the provider of secondary health care services. Main results: The systematic review (Study I) showed that although QALYs gained are considered an important measure of the effectiveness of health care, the number of studies in which QALYs are based on actual measurements of patients' HRQoL is still fairly limited. Of the reviewed full-text articles, only 70 reported QALYs based on actual before after measurements using a valid HRQoL instrument. Collection of simple cost-effectiveness data in secondary health care is feasible and could easily be expanded and performed on a routine basis (Study II). It allows meaningful comparisons between various treatments and provides a means for allocating limited health care resources. The cost per QALY gained was 2 770 for cervical operations and 1 740 for lumbar operations. In cases where surgery was delayed the cost per QALY was doubled (Study III). The cost per QALY ranges between subgroups in cataract surgery (Study IV). The cost per QALY gained was 5 130 for patients having both eyes operated on and 8 210 for patients with only one eye operated on during the 6-month follow-up. In patients whose first eye had been operated on previous to the study period, the mean HRQoL deteriorated after surgery, thus precluding the establishment of the cost per QALY. In arthroplasty patients (Study V) the mean cost per QALY gained in a one-year period was 6 710 for primary hip replacement, 52 270 for revision hip replacement, and 14 000 for primary knee replacement. Conclusions: Although the importance of cost-utility analyses has during recent years been stressed, there are only a limited number of studies in which the evaluation is based on patients own assessment of the treatment effectiveness. Most of the cost-effectiveness and cost-utility analyses are based on modeling that employs expert opinion regarding the outcome of treatment, not on patient-derived assessments. Routine collection of effectiveness information from patients entering treatment in secondary health care turned out to be easy enough and did not, for instance, require additional personnel on the wards in which the study was executed. The mean patient response rate was more than 70 %, suggesting that patients were happy to participate and appreciated the fact that the hospital showed an interest in their well-being even after the actual treatment episode had ended. Spinal surgery leads to a statistically significant and clinically important improvement in HRQoL. The cost per QALY gained was reasonable, at less than half of that observed for instance for hip replacement surgery. However, prolonged waiting for an operation approximately doubled the cost per QALY gained from the surgical intervention. The mean utility gain following routine cataract surgery in a real world setting was relatively small and confined mostly to patients who had had both eyes operated on. The cost of cataract surgery per QALY gained was higher than previously reported and was associated with considerable degree of uncertainty. Hip and knee replacement both improve HRQoL. The cost per QALY gained from knee replacement is two-fold compared to hip replacement. Cost-utility results from the three studied specialties showed that there is great variation in the cost-utility of surgical interventions performed in a real-world setting even when only common, widely accepted interventions are considered. However, the cost per QALY of all the studied interventions, except for revision hip arthroplasty, was well below 50 000, this figure being sometimes cited in the literature as a threshold level for the cost-effectiveness of an intervention. Based on the present study it may be concluded that routine evaluation of the cost-utility of secondary health care is feasible and produces information essential for a rational and balanced allocation of scarce health care resources.

Quality of life after early enteral feeding versus standard care for proven or suspected advanced epithelial ovarian cancer: Results from a randomised trial

Background Malnutrition is common in patients with advanced epithelial ovarian cancer (EOC), and is associated with impaired quality of life (QoL), longer hospital stay and higher risk of treatment-related adverse events. This phase III multi-centre randomised clinical trial tested early enteral feeding versus standard care on postoperative QoL. Methods From 2009 to 2013, 109 patients requiring surgery for suspected advanced EOC, moderately to severely malnourished were enrolled at five sites across Queensland and randomised to intervention (n = 53) or control (n = 56) groups. Intervention involved intraoperative nasojejunal tube placement and enteral feeding until adequate oral intake could be maintained. Despite being randomised to intervention, 20 patients did not receive feeds (13 did not receive the feeding tube; 7 had it removed early). Control involved postoperative diet as tolerated. QoL was measured at baseline, 6 weeks postoperatively and 30 days after the third cycle of chemotherapy. The primary outcome measure was the difference in QoL between the intervention and the control group. Secondary endpoints included treatment-related adverse event occurrence, length of stay, postoperative services use, and nutritional status. Results Baseline characteristics were comparable between treatment groups. No significant difference in QoL was found between the groups at any time point. There was a trend towards better nutritional status in patients who received the intervention but the differences did not reach statistical significance except for the intention-to-treat analysis at 7 days postoperatively (11.8 intervention vs. 13.8 control, p 0.04). Conclusion Early enteral feeding did not significantly improve patients' QoL compared to standard of care but may improve nutritional status.

Physical activity and health related quality of life

Background Research on the relationship between Health Related Quality of Life (HRQoL) and physical activity (PA), to date, have rarely investigated how this relationship differ across objective and subjective measures of PA. The aim of this paper is to explore the relationship between HRQoL and PA, and examine how this relationship differs across objective and subjective measures of PA, within the context of a large representative national survey from England. Methods Using a sample of 5,537 adults (40–60 years) from a representative national survey in England (Health Survey for England 2008), Tobit regressions with upper censoring was employed to model the association between HRQoL and objective, and subjective measures of PA controlling for potential confounders. We tested the robustness of this relationship across specific types of PA. HRQoL was assessed using the summary measure of health state utility value derived from the EuroQol-5 Dimensions (EQ-5D) whilst PA was assessed via subjective measure (questionnaire) and objective measure (accelerometer- actigraph model GT1M). The actigraph was worn (at the waist) for 7 days (during waking hours) by a randomly selected sub-sample of the HSE 2008 respondents (4,507 adults – 16 plus years), with a valid day constituting 10 hours. Analysis was conducted in 2010. Results Findings suggest that higher levels of PA are associated with better HRQoL (regression coefficient: 0.026 to 0.072). This relationship is consistent across different measures and types of PA although differences in the magnitude of HRQoL benefit associated with objective and subjective (regression coefficient: 0.047) measures of PA are noticeable, with the former measure being associated with a relatively better HRQoL (regression coefficient: 0.072). Conclusion Higher levels of PA are associated with better HRQoL. Using an objective measure of PA compared with subjective shows a relatively better HRQoL.

Multi-morbidity, obesity and quality of life among physically inactive Australians accessing physiotherapy clinics for musculoskeletal disorders

The benefits of physical activity are established and numerous, including improved musculoskeletal health and reduced risk of cardiovascular disease, diabetes, some cancers, and a range of other chronic conditions. While sedentary lifestyles are becoming increasingly prevalent among populations internationally, people with musculoskeletal disorders may face additional challenges to undertaking exercise and physically activities. Unfortunately, interventions in ambulatory hospital clinics for people with musculoskeletal disorders primarily focus on their presenting musculoskeletal complaint with cursory attention given to lifestyle risk factors; including physical inactivity.

Hormone therapy in elderly women; a comparative study with alendronate of the effects on bone, cardiovascular risk factors, periodontal conditions and quality of life

Thirty percent of 70-year-old women have osteoporosis; after age of 80 its prevalence is up to 70%. Postmenopausal women with osteoporosis seem to be at an increased risk for cardiovascular events, and deterioration of oral health, as shown by attachment loss of teeth, which is proportional to the severity of osteoporosis. Osteoporosis can be treated with many different medication, e.g. estrogen and alendronate. We randomized 90 elderly osteoporotic women (65-80 years of age) to receive hormone therapy (HT)(2mg E2+NETA), 10mg alendronate, and their combination for two years and compared their effects on bone mineral density (BMD) and turnover, two surrogate markers of the risk of cardiovascular diseases, C-reactive protein (CRP) and E-selectin, as well as oral health. The effect of HT on health-related quality of life (HRQoL) was studied in the population-based cohort of 1663 postmenopausal women (mean age 68 yr) (585 estrogen users and 1078 non-users). BMD was measured with dual-energy X-ray absorptiometry (DXA) at 0, 12 and 24 months. Urinary N-telopeptide (NTX) of type I collagen, a marker of bone resorption, and serum aminoterminal propeptide of human type I procollagen (PINP), a marker of bone formation, were measured every six months of treatment. Serum CRP and E-selectin, were measured at 0, 6, and 12 months. Dental, and periodontal conditions, and gingival crevicular fluid (GCF) matrix metalloproteinase (MMP)-8 levels were studied to evaluate the oral health status and for the mouth symptoms a structured questionnaire was used. The HRQoL was measured with 15D questionnaire. Lumbar spine BMD increased similarly in all treatment groups (6.8-8.4% and 9.1-11.2%). Only HT increased femoral neck BMD at both 12 (4.9%) and 24 months (5.8%), at the latter time point the HT group differed significantly from the other groups. HT reduced bone marker levels of NTX and PINP significantly less than other two groups.Oral HT significantly increased serum CRP level by 76.5% at 6 and by 47.1% (NS) at 12 months, and decreased serum E-selectin level by 24.3% and 30.0%. Alendronate had no effect on these surrogate markers. Alendronate caused a decrease in the resting salivary flow rate and tended to increase GCF MMP-8 levels. Otherwise, there was no effect on the parameters of oral health. HT improved the HRQoL of elderly women significantly on the dimensions of usual activities, vitality and sexual activity, but the overall improvement in HRQoL was neither statistically significant nor clinically important. In conclusion, bisphosphonates might be the first option to start the treatment of postmenopausal osteoporosis in the old age.

Health-Related Quality of Life and Mental Disorders in Finland

Quality of life (QoL) and Health-related quality of life (HRQoL) are becoming one of the key outcomes of health care due to increased respect for the subjective valuations and well-being of patients and an increasing part of the ageing population living with chronic, non-fatal conditions. Preference-based HRQoL measures enable estimation of health utility, which can be useful for rational rationing, evidence-based medicine and health policy. This study aimed to compare the individual severity and public health burden of major chronic conditions in Finland, including and focusing on reliably diagnosed psychiatric conditions. The study is based on the Health 2000 survey, a representative general population survey of 8028 Finns aged 30 and over. Depressive, anxiety and alcohol use disorders were diagnosed with the Composite International Diagnostic Interview (M-CIDI). HRQoL was measured with the 15D and the EQ-5D, with 83% response rate. This study found that people with psychiatric disorders had the lowest 15D HRQoL scores at all ages, in comparison to other main groups of chronic conditions. Considering 29 individual conditions, three of the four most severe (on 15D) were psychiatric disorders; the most severe was Parkinson s disease. Of the psychiatric disorders, chronic conditions that have sometimes been considered relatively mild - dysthymia, agoraphobia, generalized anxiety disorder and social phobia - were found to be the most severe. This was explained both by the severity of the impact of these disorders on mental health domains of HRQoL, and also by the fact that decreases were widespread on most dimensions of HRQoL. Considering the public health burden of conditions, musculoskeletal disorders were associated with the largest burden, followed by psychiatric disorders. Psychiatric disorders were associated with the largest burden at younger ages. Of individual conditions, the largest burden found was for depressive disorders, followed by urinary incontinence and arthrosis of the hip and knee. The public health burden increased greatly with age, so the ageing of the Finnish population will mean that the disease burden caused by chronic conditions will increase by a quarter up to year 2040, if morbidity patterns do not change. Investigating alcohol consumption and HRQoL revealed that although abstainers had poorer HRQoL than moderate drinkers, this was mainly due to many abstainers being former drinkers and having the poorest HRQoL. Moderate drinkers did not have significantly better HRQoL than abstainers who were not former drinkers. Psychiatric disorders are associated with a large part of the non-fatal disease burden in Finland. In particular anxiety disorders appear to be more severe and have a larger public health burden than previously thought.

Death or taxes? The political economy of sanitation expenditure in nineteenth-century Britain

This thesis consists of three papers studying the relationship between democratic reform, expenditure on sanitation public goods and mortality in Britain in the second half of the nineteenth century. During this period decisions over spending on critical public goods such as water supply and sewer systems were made by locally elected town councils, leading to extensive variation in the level of spending across the country. This dissertation uses new historical data to examine the political factors determining that variation, and the consequences for mortality rates.

The first substantive chapter describes the spread of government sanitation expenditure, and analyzes the factors that determined towns' willingness to invest. The results show the importance of towns' financial constraints, both in terms of the available tax base and access to borrowing, in limiting the level of expenditure. This suggests that greater involvement by Westminster could have been very effective in expediting sanitary investment. There is little evidence, however, that democratic reform was an important driver of greater expenditure.

Chapter 3 analyzes the effect of extending voting rights to the poor on government public goods spending. A simple model predicts that the rich and the poor will desire lower levels of public goods expenditure than the middle class, and so extensions of the right to vote to the poor will be associated with lower spending. This prediction is tested using plausibly exogenous variation in the extent of the franchise. The results strongly support the theoretical prediction: expenditure increased following relatively small extensions of the franchise, but fell once more than approximately 50% of the adult male population held the right to vote.

Chapter 4 tests whether the sanitary expenditure was effective in combating the high mortality rates following the Industrial Revolution. The results show that increases in urban expenditure on sanitation-water supply, sewer systems and streets-was extremely effective in reducing mortality from cholera and diarrhea.

A dual-band antenna enabling improved quality of service in multi-radio wireless sensor applications in indoor environments

Science Foundation Ireland (CSET - Centre for Science, Engineering and Technology, Grant No. 07/CE/11147)

The quality of diabetes care in the community: practice, policy and culture

Aim: Diabetes is an important barometer of health system performance. This chronic condition is a source of significant morbidity, premature mortality and a major contributor to health care costs. There is an increasing focus internationally, and more recently nationally, on system, practice and professional-level initiatives to promote the quality of care. The aim of this thesis was to investigate the ‘quality chasm’ around the organisation and delivery of diabetes care in general practice, to explore GPs’ attitudes to engaging in quality improvement activities and to examine efforts to improve the quality of diabetes care in Ireland from practice to policy. Methods: Quantitative and qualitative methods were used. As part of a mixed methods sequential design, a postal survey of 600 GPs was conducted to assess the organization of care. This was followed by an in-depth qualitative study using semi-structured interviews with a purposive sample of 31 GPs from urban and rural areas. The qualitative methodology was also used to examine GPs’ attitudes to engaging in quality improvement. Data were analysed using a Framework approach. A 2nd observation study was used to assess the quality of care in 63 practices with a special interest in diabetes. Data on 3010 adults with Type 2 diabetes from 3 primary care initiatives were analysed and the results were benchmarked against national guidelines and standards of care in the UK. The final study was an instrumental case study of policy formulation. Semi-structured interviews were conducted with 15 members of the Expert Advisory Group (EAG) for Diabetes. Thematic analysis was applied to the data using 3 theories of the policy process as analytical tools. Results: The survey response rate was 44% (n=262). Results suggested care delivery was largely unstructured; 45% of GPs had a diabetes register (n=157), 53% reported using guidelines (n=140), 30% had formal call recall system (n=78) and 24% had none of these organizational features (n=62). Only 10% of GPs had a formal shared protocol with the local hospital specialist diabetes team (n=26). The lack of coordination between settings was identified as a major barrier to providing optimal care leading to waiting times, overburdened hospitals and avoidable duplication. The lack of remuneration for chronic disease management had a ripple effect also creating costs for patients and apathy among GPs. There was also a sense of inertia around quality improvement activities particularly at a national level. This attitude was strongly influenced by previous experiences of change in the health system. In contrast GP’s spoke positively about change at a local level which was facilitated by a practice ethos, leadership and special interest in diabetes. The 2nd quantitative study found that practices with a special interest in diabetes achieved a standard of care comparable to the UK in terms of the recording of clinical processes of care and the achievement of clinical targets; 35% of patients reached the HbA1c target of <6.5% compared to 26% in England and Wales. With regard to diabetes policy formulation, the evolving process of action and inaction was best described by the Multiple Streams Theory. Within the EAG, the formulation of recommendations was facilitated by overarching agreement on the “obvious” priorities while the details of proposals were influenced by personal preferences and local capacity. In contrast the national decision-making process was protracted and ambiguous. The lack of impetus from senior management coupled with the lack of power conferred on the EAG impeded progress. Conclusions: The findings highlight the inconsistency of diabetes care in Ireland. The main barriers to optimal diabetes management center on the organization and coordination of care at the systems level with consequences for practice, providers and patients. Quality improvement initiatives need to stimulate a sense of ownership and interest among frontline service providers to address the local sense of inertia to national change. To date quality improvement in diabetes care has been largely dependent the “special interest” of professionals. The challenge for the Irish health system is to embed this activity as part of routine practice, professional responsibility and the underlying health care culture.

The quality of deliberation in Northern Ireland's district policing partnerships

This study uses theoretical based deliberative democratic dimensions to measure the deliberative quality of Northern Ireland’s District Policing Partnership (DPP) meetings in public. The study combines Habermasian, and Young’s deliberative concepts to create an Augmented Discourse Quality Index. This Augmented DQI is employed by this research as am empirical instrument to establish the true deliberative nature of these DPP meetings in public. The overall goal of this study is two-fold. First; to gain an in-depth understanding of Northern Ireland’s DPPs in relation to deliberative democratic theory, specifically regarding how these policing/public partnerships stand up under a deliberative democratic lens. The second goal is to provide a possible framework by which deliberative quality can be more accurately measured. In that frameworks which are designed to measure deliberative quality should include not only the dimensions for rational participation, but also include broader terms of communication such as greeting, rhetoric and story-telling.

Aging African American Women and the Impact of Positive Psychological Factors on Quality of Life

Aging African-American women are disproportionately affected by negative health outcomes and mortality. Life stress has strong associations with these health outcomes. The purpose of this research was to understand how aging African American women manage stress. Specifically, the effects of coping, optimism, resilience, and religiousness as it relates to quality of life were examined. This cross-sectional exploratory study used a self-administered questionnaire and examined quality of life in 182 African-American women who were 65 years of age or older living in senior residential centers in Baltimore using convenience sampling. The age range for these women was 65 to 94 years with a mean of 71.8 years (SD = 5.6). The majority (53.1%) of participants completed high school, with 23 percent (N = 42) obtaining college degrees and 19 percent (N = 35) holding advanced degrees. Nearly 58 percent of participants were widowed and 81 percent were retired. In addition to demographics, the questionnaire included the following reliable and valid survey instruments: The Brief Cope Scale (Carver, Scheier, & Weintraub, 1989), Optimism Questionnaire (Scheier, Carver, & Bridges, 1994), Resilience Survey (Wagnild & Young, 1987), Religiousness Assessment (Koenig, 1997), and Quality of Life Questionnaire (Cummins, 1996). Results revealed that the positive psychological factors examined were positively associated with and significant predictors of quality of life. The bivariate correlations indicated that of the six coping dimensions measured in this study, planning (r=.68) was the most positively associated with quality of life. Optimism (r=.33), resilience (=.48), and religiousness (r=.30) were also significantly correlated with quality of life. In the linear regression model, again the coping dimension of planning was the best predictor of quality of life (beta = .75, p <.001). Optimism (beta = .31, p <.001), resilience (beta = .34, p, .001) and religiousness (beta = .17, p <.01) were also significant predictors of quality of life. It appears as if positive psychology plays an important role in improving quality of life among aging African-American women.

Genomics research: world survey of public funding.

BACKGROUND: Over the past two decades, genomics has evolved as a scientific research discipline. Genomics research was fueled initially by government and nonprofit funding sources, later augmented by private research and development (R&D) funding. Citizens and taxpayers of many countries have funded much of the research, and have expectations about access to the resulting information and knowledge. While access to knowledge gained from all publicly funded research is desired, access is especially important for fields that have broad social impact and stimulate public dialogue. Genomics is one such field, where public concerns are raised for reasons such as health care and insurance implications, as well as personal and ancestral identification. Thus, genomics has grown rapidly as a field, and attracts considerable interest. RESULTS: One way to study the growth of a field of research is to examine its funding. This study focuses on public funding of genomics research, identifying and collecting data from major government and nonprofit organizations around the world, and updating previous estimates of world genomics research funding, including information about geographical origins. We initially identified 89 publicly funded organizations; we requested information about each organization's funding of genomics research. Of these organizations, 48 responded and 34 reported genomics research expenditures (of those that responded but did not supply information, some did not fund such research, others could not quantify it). The figures reported here include all the largest funders and we estimate that we have accounted for most of the genomics research funding from government and nonprofit sources. CONCLUSION: Aggregate spending on genomics research from 34 funding sources averaged around $2.9 billion in 2003-2006. The United States spent more than any other country on genomics research, corresponding to 35% of the overall worldwide public funding (compared to 49% US share of public health research funding for all purposes). When adjusted to genomics funding intensity, however, the United States dropped below Ireland, the United Kingdom, and Canada, as measured both by genomics research expenditure per capita and per Gross Domestic Product.

Generic, disease-specific and individualised approaches to measuring health-related quality of life among people with heart disease - a comparative analysis.

Increasing emphasis is being placed on the evaluation of health-related quality of life. However, there is no consensus on the definition of this concept and as a result there are a plethora of existing measurement instruments. Head-to-head comparisons of the psychometric properties of existing instruments are necessary to facilitate evidence-based decisions about which instrument should be chosen for routine use. Therefore, an individualised instrument (the modified Patient Generated Index), a generic instrument (the Short Form 36) and a disease-specific instrument (the Quality of Life after Myocardial Infarction questionnaire) were administered to patients with ischaemic heart disease (n=117) and the evidence for the validity, reliability and sensitivity of each instrument was examined and compared. The modified Patient Generated Index compared favourably with the other instruments but none of the instruments examined provided sound evidence for sensitivity to change. Therefore, any recommendation for the use of the individualised approach in the routine collection of health-related quality of life data in clinical practice must be conditional upon the submission of further evidence to support the sensitivity of such instruments.

Assessment of data quality in a multi-centre cross-sectional study of participation and quality of life of children with cerebral palsy

Background: SPARCLE is a cross-sectional survey in nine European regions, examining the relationship of the environment of children with cerebral palsy to their participation and quality of life. The objective of this report is to assess data quality, in particular heterogeneity between regions, family and item non-response and potential for bias. Methods: 1,174 children aged 8–12 years were selected from eight population-based registers of children with cerebral palsy; one further centre recruited 75 children from multiple sources. Families were visited by trained researchers who administered psychometric questionnaires. Logistic regression was used to assess factors related to family non-response and self-completion of questionnaires by children. Results: 431/1,174 (37%) families identified from registers did not respond: 146 (12%) were not traced; of the 1,028 traced families, 250 (24%) declined to participate and 35 (3%) were not approached. Families whose disabled children could walk unaided were more likely to decline to participate. 818 children entered the study of which 500 (61%) self-reported their quality of life; children with low IQ, seizures or inability to walk were less likely to self-report. There was substantial heterogeneity between regions in response rates and socio-demographic characteristics of families but not in age or gender of children. Item non-response was 2% for children and ranged from 0.4% to 5% for questionnaires completed by parents. Conclusion: While the proportion of untraced families was higher than in similar surveys, the refusal rate was comparable. To reduce bias, all analyses should allow for region, walking ability, age and socio-demographic characteristics. The 75 children in the region without a population based register are unlikely to introduce bias

Evaluation of the Q-method as a method of public participation in the selection of sustainable development indicators

The complexity of sustainable development means that it is often difficult to evaluate and communicate the concept effectively. One standard method to reduce complexity and improve Communication, while maintaining scientific objectivity, is to use selected indicators. The aim of this paper is to describe and evaluate a process Of public participation in the selection of sustainable development indicators that utilised the Q-method for discourse analysis. The Q-method was Utilised to combine public opinion with technical expertise to create a list of technically robust indicators that would be relevant to the public, The method comprises statement collection, statement analysis, Q-sorts and Q-sort analysis. The results of the Q-method generated a list of statements for which a preliminary list of indicators was then developed by a team of experts from the fields of environmental science, sustainable development and Psychology. Subsequently members of the public evaluated the preliminary list of indicators, to select a final list of indicators that were both technically sound and incorporated the views of the public. The Utilisation of the Q-method in this process was evaluated using previously published criteria. The application of the Q-method in this context needs to be considered not only by the quality of the indicators developed, but also from the perspective of the benefit of the process to the participants. it was concluded that the Q-method provided an effective framework for public participation in the selection of indicators as it allowed the public to discuss Sustainable development in familiar language and in the context of their daily lives. By combining this information with expert input, a list of technically robust indicators that resonate with the public was developed. The results demonstrated that many citizens are not aware Of Sustainable development, and if it is to be successfully communicated to them, then indicators and policy need to be couched in terms familiar and relevant to citizen and communities. (C) 2009 Elsevier Ltd. All rights reserved.