The International Universities Walking Project: Development of a Framework for Workplace Intervention Using the Delphi Technique

Background: This paper aimed to use the Delphi technique to develop a consensus framework for a multinational, workplace walking intervention. Methods: Ideas were gathered and ranked from eight recognized and emerging experts in the fields of physical activity and health, from universities in Australia, Canada, England, the Netherlands, Northern Ireland, and Spain. Members of the panel were asked to consider the key characteristics of a successful campus walking intervention. Consensus was reached by an inductive, content analytic approach, conducted through an anonymous, three-round, e-mail process. Results: The resulting framework consisted of three interlinking themes defined as “design, implementation, and evaluation.” Top-ranked subitems in these themes included the need to generate research capacity (design), to respond to group needs through different walking approaches (implementation), and to undertake physical activity assessment (evaluation). Themes were set within an underpinning domain, referred to as the “institution” and sites are currently engaging with subitems in this domain, to provide sustainable interventions that reflect the practicalities of local contexts and needs. Conclusions: Findings provide a unique framework for designing, implementing, and evaluating walking projects in universities and highlight the value of adopting the Delphi technique for planning international, multisite health initiatives.

European undergraduate curriculum in geriatric medicine developed using an international modified Delphi technique.

INTRODUCTION the rise in the number of older, frail adults necessitates that future doctors are adequately trained in the skills of geriatric medicine. Few countries have dedicated curricula in geriatric medicine at the undergraduate level. The aim of this project was to develop a consensus among geriatricians on a curriculum with the minimal requirements that a medical student should achieve by the end of medical school. METHODS a modified Delphi process was used. First, educational experts and geriatricians proposed a set of learning objectives based on a literature review. Second, three Delphi rounds involving a panel with 49 experts representing 29 countries affiliated to the European Union of Medical Specialists (UEMS) was used to gain consensus for a final curriculum. RESULTS the number of disagreements following Delphi Rounds 1 and 2 were 81 and 53, respectively. Complete agreement was reached following the third round. The final curriculum consisted of detailed objectives grouped under 10 overarching learning outcomes. DISCUSSION a consensus on the minimum requirements of geriatric learning objectives for medical students has been agreed by European geriatricians. Major efforts will be needed to implement these requirements, given the large variation in the quality of geriatric teaching in medical schools. This curriculum is a first step to help improve teaching of geriatrics in medical schools, and will also serve as a basis for advancing postgraduate training in geriatrics across Europe.

PIPc study: development of indicators of potentially inappropriate prescribing in children (PIPc) in primary care using a modified Delphi technique

Objective There is limited evidence regarding the quality of prescribing for children in primary care. Several prescribing criteria (indicators) have been developed to assess the appropriateness of prescribing in older and middle-aged adults but few are relevant to children. The objective of this study was to develop a set of prescribing indicators that can be applied to prescribing or dispensing data sets to determine the prevalence of potentially inappropriate prescribing in children (PIPc) in primary care settings.


Design Two-round modified Delphi consensus method.


Setting Irish and UK general practice.


Participants A project steering group consisting of academic and clinical general practitioners (GPs) and pharmacists was formed to develop a list of indicators from literature review and clinical expertise. 15 experts consisting of GPs, pharmacists and paediatricians from the Republic of Ireland and the UK formed the Delphi panel.


Results 47 indicators were reviewed by the project steering group and 16 were presented to the Delphi panel. In the first round of this exercise, consensus was achieved on nine of these indicators. Of the remaining seven indicators, two were removed following review of expert panel comments and discussion of the project steering group. The second round of the Delphi process focused on the remaining five indicators, which were amended based on first round feedback. Three indicators were accepted following the second round of the Delphi process and the remaining two indicators were removed. The final list consisted of 12 indicators categorised by respiratory system (n=6), gastrointestinal system (n=2), neurological system (n=2) and dermatological system (n=2).


Conclusions The PIPc indicators are a set of prescribing criteria developed for use in children in primary care in the absence of clinical information. The utility of these criteria will be tested in further studies using prescribing databases.

Identification of nurses` competencies in primary health care through a Delphi study in southern Brazil

Objective: This investigation aimed to identify and analyze the general and specific competencies of nurses in the primary health care practice of Brazil. Design: The Delphi Technique was used as the method of study. Sample: 2 groups of participants were selected: One contained primary health care nurses (n=52) and the other specialists (n=57), including public health nurses and public or community health faculty. Measurements: 3 questionnaires were developed for the study. The first asked participants to indicate general and specific competencies, which were compiled into a list for each group. A Likert scale of 1-5 was added to these 2 lists in the second and third questionnaires. A consensus criterion of 75% for score 4 or 5 was adopted. Results: In the nurses` group, 17 general and 8 specific competencies reached the consensus criterion; 19 general and 9 specific competencies reached the criterion in the specialists` group. These competencies were classified into 10 domains: professional values, communication, teamwork, management, community-oriented, health promotion, problem solving, health care, and education and basic public health sciences. Conclusions: These competencies reflect Brazilian health policy and constitute a reference for health professional practice and education.

Initiation of biological agents in patients with ankylosing spondylitis: results of a Delphi study by the ASAS Group

Background: There is ample evidence of important symptomatic efficacy of tumour necrosis factor alpha (TNFalpha) inhibition in ankylosing spondylitis (AS). Moreover, studies suggest that anti-TNF could be considered as the first disease controlling antirheumatic treatment (DC-ART) for AS. Objective: To determine precisely which patients with AS are most likely to benefit from anti-TNFalpha treatment because of the cost and possible long term side effects of such treatment. Methods: Assessment in Ankylosing Spondylitis (ASAS) members were asked to use a Delphi technique to name the characteristics of patients with AS for whom they would start DC-ART, in three different clinical presentations (isolated axial involvement, peripheral arthritis, enthesitis). Results: Among the 62 invited ASAS members, more than 50% actively participated in the four phases of definition according to the Delphi technique. For each of the three clinical presentations, a combination of five to six domains was proposed, with an evaluation instrument and a cut off point defining a minimum level of activity for each domain. Conclusion: This study provides a profile for a patient with AS for considering initiation of biological agents that reflects the opinion of the ASAS members, using a Delphi exercise. Further studies are required to assess their relevance and their consistency with clinical practice.

A delphi study to detect deficiencies and propose actions in real life treatment of neovascular age-related macular degeneration.

Purpose. Spanish retina specialists were surveyed in order to propose actions to decrease deficiencies in real-life neovascular age macular degeneration treatment (nv-AMD). Methods. One hundred experts, members of the Spanish Vitreoretinal Society (SERV), were invited to complete an online survey of 52 statements about nv-AMD management with a modified Delphi methodology. Four rounds were performed using a 5-point Linkert scale. Recommendations were developed after analyzing the differences between the results and the SERV guidelines recommendations. Results. Eighty-seven specialists completed all the Delphi rounds. Once major potential deficiencies in real-life nv-AMD treatment were identified, 15 recommendations were developed with a high level of agreement. Consensus statements to reduce the burden of the disease included the use of treat and extend regimen and to reduce the amount of diagnostic tests during the loading phase and training technical staff to perform these tests and reduce the time between relapse detection and reinjection, as well as establishing patient referral protocols to outside general ophthalmology clinics. Conclusion. The level of agreement with the final recommendations for nv-AMD treatment among Spanish retinal specialist was high indicating that some actions could be applied in order to reduce the deficiencies in real-life nv-AMD treatment.

Research priorities for nursing care of infants, children and adolescents: a West Australian Delphi study.

This paper describes a study that aimed to identify research priorities for the care of infants, children and adolescents at the sole tertiary referral hospital for children in Western Australia. The secondary aim was to stimulate nurses to explore clinical problems that would require further inquiry. Background. Planning for research is an essential stage of research development; involving clinicians in this exercise is likely to foster research partnerships that are pertinent to clinical practice. Nursing research priorities for the paediatric population have not previously been reported in Australia. Design. Delphi study. Method. Over 12 months in 2005-2006, a three-round questionnaire, using the Delphi technique, was sent to a randomly selected sample of registered nurses. This method was used to identify and prioritise nursing research topics relevant to the patient and the family. Content analysis was used to analyse Round I data and descriptive statistics for Round II and III data. Results. In Round I, 280 statements were identified and reduced to 37 research priorities. Analysis of data in subsequent rounds identified the top two priority research areas as (1) identification of strategies to reduce medication incidents (Mean = 6 center dot 47; SD 0 center dot 88) and (2) improvement in pain assessment and management (Mean = 6; SD 1 center dot 38). Additional comments indicated few nurses access the scientific literature or use research findings because of a lack of time or electronic access. Conclusions. Thirty-seven research priorities were identified. The identification of research priorities by nurses provided research direction for the health service and potentially other similar health institutions for children and adolescents in Australia and internationally. Relevance to clinical practice. The nurse participants showed concern about the safety of care and the well-being of children and their families. This study also enabled the identification of potential collaborative research and development of pain management improvement initiatives.

A Delphi study on National PICU nursing research priorities in Australia and New Zealand.

BACKGROUND: There is a lack of evidence to direct and support nursing practice in the specialty of paediatric intensive care (PIC). The development of national PIC nursing research priorities may facilitate the process of undertaking clinical research and translating evidence into practice. PURPOSE: To (a) identify research priorities for the care of patients and their family as well as for the professional needs of PIC nurses, (b) foster nursing research collaboration, (c) develop a research agenda for PIC nurses. METHODS: Over 13 months in 2007-2008, a three-round questionnaire, using the Delphi technique, was sent to all specialist level registered nurses working in Australian and New Zealand PICUs. This method was used to identify and prioritise nursing research topics. Content analysis was used to analyse Round I data and descriptive statistics for Round II and III data. RESULTS: In Round I, 132 research topics were identified, with 77 research priorities (mdn>6, mean MAD(median) 0.68±0.01) identified in subsequent rounds. The top nine priorities (mean>6 and median>6) included patient issues related to neurological care (n=2), pain/sedation/comfort (n=3), best practice at the end of life (n=1), and ventilation strategies (n=1), as well as two priorities related to professional issues about nurses' stress/burnout and professional development needs. CONCLUSION: The research priorities identified reflect important issues related to critically ill patients and their family as well as to the nurses caring for them. These priorities can be used for the development of a research agenda for PIC nursing in Australia and New Zealand.

Introducing a first AIDS vaccine in Switzerland: a Policy Delphi analysis.

This preliminary exploration was limited by a number of factors. The format of the study has necessarily induced some form of selection bias of the panelists, because of the complexity of some questions, and the time required to complete the questionnaires. Several issues have not been addressed. One example could be the response to HIV infection occurring in a vaccinee. The study also did not address the difficulties related to the licensing of the vaccine. Indeed, the proposed scenario assumed that the vaccine had been registered as a starting point for the analysis. Finally, it has not been possible to conduct a sensitivity analysis, in order to evaluate how the responses would have been modified if some important characteristics of the vaccine had been modified.Very diverse evaluations were given in response to questions related with attitudes and perception of AIDS and AIDS vaccine. The possibility that vaccine availability or usage can be associated with an increased frequency in risky behaviors was spontaneously mentioned by half of the panelists. The estimation of the proportion of persons at highest risk who would choose to use this vaccine also indicated a high degree of uncertainty. This study offers important lessons. According to a broad and diverse panel of individuals, an incompletely effective AIDS vaccine would result in an additional level of complexity for the AIDS prevention strategy, rather than a simplification. The use of such a vaccine would have to be coupled with counselling. This implies a sustained emphasis on the recommendations which have been central to the STOP AIDS campaigns until now. In addition, consensual issues, as well as other issues more likely to be controversial have been identified. This should greatly help focusing the work of any committee designated to develop and implement a vaccination policy if an AIDS vaccine became available. Finally, our experience with the Policy Delphi indicates that this mode of structured communication could be usefully applied to other public health issues presenting a high visibility as well as a complex relationship with public perception.

Malaria chemoprophylaxis recommendations for immigrants to Europe, visiting relatives and friends--a Delphi method study.

BACKGROUND: Numbers of travellers visiting friends and relatives (VFRs) from Europe to malaria endemic countries are increasing and include long-term and second generation immigrants, who represent the major burden of malaria cases imported back into Europe. Most recommendations for malaria chemoprophylaxis lack a solid evidence base, and often fail to address the cultural, social and economic needs of VFRs. METHODS: European travel medicine experts, who are members of TropNetEurop, completed a sequential series of questionnaires according to the Delphi method. This technique aims at evaluating and developing a consensus through repeated iterations of questionnaires. The questionnaires in this study included questions about professional experience with VFRs, controversial issues in malaria prophylaxis, and 16 scenarios exploring indications for prescribing and choice of chemoprophylaxis. RESULTS: The experience of participants was rather diverse as was their selection of chemoprophylaxis regimen. A significant consensus was observed in only seven of 16 scenarios. The analysis revealed a wide variation in prescribing choices with preferences grouped by region of practice and increased prescribing seen in Northern Europe compared to Central Europe. CONCLUSIONS: Improving the evidence base on efficacy, adherence to chemoprophylaxis and risk of malaria and encouraging discussion among experts, using techniques such as the Delphi method, may reduce the variability in prescription in European travel clinics.

Key elements for, and indicators of, a successful transition: an international Delphi study.

PURPOSE: The purpose of this study was to reach an international consensus to determine what key elements should be part of a transition program and what indicators could be used to assess its success. METHODS: For this purpose, a Delphi study including an international panel of 37 experts was carried out. The study consisted of three rounds, with response rates ranging from 86.5% to 95%. At each round, experts were asked to assess key elements (defined as the most important elements for the task) and indicators (defined as quantifiable characteristics). At each round, panelists were contacted via e-mail explaining them the tasks to be done and giving them the Web link where to complete the questionnaire. At Round 3, each key element and indicator was assessed as essential, very important, important, accessory, or unnecessary. A 70% agreement was used as cutoff. RESULTS: At Round 3, more than 70% of panelists agreed on six key elements being essential, with one of them (Assuring a good coordination between pediatric and adult professionals) reaching an almost complete consensus (97%). Additionally, 11 more obtained more than 70% agreement when combined with the Very important category. Among indicators, only one (Patient not lost to follow-up) was considered almost unanimously (91%) as essential by the panelists and seven others also reached consensus when the Very important category was included. CONCLUSIONS: Using these results as a framework to develop guidelines at local, national, and international levels would allow better assessing and comparing transition programs.

Major Sport Event Operational Planning Issues and Strategies: A Multi-Case Delphi Study

This study examined the operational planning, implementation and execution issues of major sport events, as well as the mitigation and management strategies used to address these issues, with the aim of determining best practices in sport event operational planning. The three Research Questions were: 1) What can previous major sport events provide to guide the operational management of future events? 2) What are the operational issues that arise in the planning and execution of a major sport event, how are they mitigated and what are the strategies used to deal with these issues? 3) What are the best practices for sport event operational planning and how can these practices aid future events? Data collection involved a modified Delphi technique that consisted of one round of in-depth interviews followed by two rounds of questionnaires. Both data collection and analysis were guided by an adaptation of the work of Parent, Rouillard & Leopkey (2011) with a focus on previously established issue and strategy categories. The results provided a list of Top 26 Prominent Issues and Top 17 Prominent Strategies with additional issue-strategy links that can be used to aid event managers producing future major sport events. The following issue categories emerged as having had the highest impact on previous major sport events that participants had managed: timing, funding and knowledge management. In addition, participants used strategies from the following categories most frequently: other, formalized agreements and communication.

Content validation for level of use of feature rich systems: a Delphi study of electronic medical records systems

Introduction. Feature usage is a pre-requisite to realising the benefits of investments in feature rich systems. We propose that conceptualising the dependent variable 'system use' as 'level of use' and specifying it as a formative construct has greater value for measuring the post-adoption use of feature rich systems. We then validate the content of the construct as a first step in developing a research instrument to measure it. The context of our study is the post-adoption use of electronic medical records (EMR) by primary care physicians. Method. Initially, a literature review of the empirical context defines the scope based on prior studies. Having identified core features from the literature, they are further refined with the help of experts in a consensus seeking process that follows the Delphi technique. Results.The methodology was successfully applied to EMRs, which were selected as an example of feature rich systems. A review of EMR usage and regulatory standards provided the feature input for the first round of the Delphi process. A panel of experts then reached consensus after four rounds, identifying ten task-based features that would be indicators of level of use. Conclusions. To study why some users deploy more advanced features than others, theories of post-adoption require a rich formative dependent variable that measures level of use. We have demonstrated that a context sensitive literature review followed by refinement through a consensus seeking process is a suitable methodology to validate the content of this dependent variable. This is the first step of instrument development prior to statistical confirmation with a larger sample.

Validação de constructo de tecnologia educativa para pacientes mediante aplicação da técnica Delphi

OBJETIVO: Descrever o processo de validação de uma tecnologia educativa para pacientes submetidos à cirurgia ortognática, mediante a aplicação da técnica Delphi. MÉTODOS: Participaram do estudo, dez juízes (quatro cirurgiões bucomaxilofaciais, duas enfermeiras, duas nutricionistas e duas fonoaudiólogas) que indicaram os assuntos a serem abordados no material destinado à primeira fase da técnica Delphi. Após a construção do material e a obtenção dos resultados da primeira fase, iniciou-se as segunda e terceira fases dessa técnica com validação da pertinência das informações contidas nessa tecnologia educativa. RESULTADOS: Na segunda rodada, obteve-se, pelo menos, 90,0% de concordância em todas as categorias. Na terceira, houve um aumento de 12% para o item conteúdo, melhora de 6,7% em linguagem, de 12,5% em ilustrações, de 3,3% no layout, de 10% em cultura, mantendo-se constante o item motivação. CONCLUSÃO: A técnica Delphi, pode ser uma importante ferramenta na construção e validação de tecnologias educativas.