Online-questionnaire design:establishing guidelines and evaluating existing eupport

As a new medium for questionnaire delivery, the internet has the potential to revolutionise the survey process. Online (web-based) questionnaires provide several advantages over traditional survey methods in terms of cost, speed, appearance, flexibility, functionality, and usability [1, 2]. For instance, delivery is faster, responses are received more quickly, and data collection can be automated or accelerated [1- 3]. Online-questionnaires can also provide many capabilities not found in traditional paper-based questionnaires: they can include pop-up instructions and error messages; they can incorporate links; and it is possible to encode difficult skip patterns making such patterns virtually invisible to respondents. Like many new technologies, however, online-questionnaires face criticism despite their advantages. Typically, such criticisms focus on the vulnerability of online-questionnaires to the four standard survey error types: namely, coverage, non-response, sampling, and measurement errors. Although, like all survey errors, coverage error (“the result of not allowing all members of the survey population to have an equal or nonzero chance of being sampled for participation in a survey” [2, pg. 9]) also affects traditional survey methods, it is currently exacerbated in online-questionnaires as a result of the digital divide. That said, many developed countries have reported substantial increases in computer and internet access and/or are targeting this as part of their immediate infrastructural development [4, 5]. Indicating that familiarity with information technologies is increasing, these trends suggest that coverage error will rapidly diminish to an acceptable level (for the developed world at least) in the near future, and in so doing, positively reinforce the advantages of online-questionnaire delivery. The second error type – the non-response error – occurs when individuals fail to respond to the invitation to participate in a survey or abandon a questionnaire before it is completed. Given today’s societal trend towards self-administration [2] the former is inevitable, irrespective of delivery mechanism. Conversely, non-response as a consequence of questionnaire abandonment can be relatively easily addressed. Unlike traditional questionnaires, the delivery mechanism for online-questionnaires makes estimation of questionnaire length and time required for completion difficult1, thus increasing the likelihood of abandonment. By incorporating a range of features into the design of an online questionnaire, it is possible to facilitate such estimation – and indeed, to provide respondents with context sensitive assistance during the response process – and thereby reduce abandonment while eliciting feelings of accomplishment [6]. For online-questionnaires, sampling error (“the result of attempting to survey only some, and not all, of the units in the survey population” [2, pg. 9]) can arise when all but a small portion of the anticipated respondent set is alienated (and so fails to respond) as a result of, for example, disregard for varying connection speeds, bandwidth limitations, browser configurations, monitors, hardware, and user requirements during the questionnaire design process. Similarly, measurement errors (“the result of poor question wording or questions being presented in such a way that inaccurate or uninterpretable answers are obtained” [2, pg. 11]) will lead to respondents becoming confused and frustrated. Sampling, measurement, and non-response errors are likely to occur when an online-questionnaire is poorly designed. Individuals will answer questions incorrectly, abandon questionnaires, and may ultimately refuse to participate in future surveys; thus, the benefit of online questionnaire delivery will not be fully realized. To prevent errors of this kind2, and their consequences, it is extremely important that practical, comprehensive guidelines exist for the design of online questionnaires. Many design guidelines exist for paper-based questionnaire design (e.g. [7-14]); the same is not true for the design of online questionnaires [2, 15, 16]. The research presented in this paper is a first attempt to address this discrepancy. Section 2 describes the derivation of a comprehensive set of guidelines for the design of online-questionnaires and briefly (given space restrictions) outlines the essence of the guidelines themselves. Although online-questionnaires reduce traditional delivery costs (e.g. paper, mail out, and data entry), set up costs can be high given the need to either adopt and acquire training in questionnaire development software or secure the services of a web developer. Neither approach, however, guarantees a good questionnaire (often because the person designing the questionnaire lacks relevant knowledge in questionnaire design). Drawing on existing software evaluation techniques [17, 18], we assessed the extent to which current questionnaire development applications support our guidelines; Section 3 describes the framework used for the evaluation, and Section 4 discusses our findings. Finally, Section 5 concludes with a discussion of further work.

What difference doguidelines make? An observational study of online-questionnaire design guidelines put to practical use

As a new medium for questionnaire delivery, the Internet has the potential to revolutionize the survey process. Online-questionnaires can provide many capabilities not found in traditional paper-based questionnaires. Despite this, and the introduction of a plethora of tools to support online-questionnaire creation, current electronic survey design typically replicates the look-and-feel of paper-based questionnaires, thus failing to harness the full power of the electronic delivery medium. A recent environmental scan of online-questionnaire design tools found that little, if any, support is incorporated within these tools to guide questionnaire designers according to best-practice [Lumsden & Morgan 2005]. This paper briefly introduces a comprehensive set of guidelines for the design of online-questionnaires. Drawn from relevant disparate sources, all the guidelines incorporated within the set are proven in their own right; as an initial assessment of the value of the set of guidelines as a practical reference guide, we undertook an informal study to observe the effect of introducing the guidelines into the design process for a complex online-questionnaire. The paper discusses the qualitative findings — which are encouraging for the role of the guidelines in the ‘bigger picture’ of online survey delivery across many domains such as e-government, e-business, and e-health — of this case study.

Online-questionaire design guidelines

As a new medium for questionnaire delivery, the internet has the potential to revolutionize the survey process. Online (web-based) questionnaires provide several advantages over traditional survey methods in terms of cost, speed, appearance, flexibility, functionality, and usability. Designers of online-questionnaires are faced with a plethora of design tools to assist in the development of their electronic questionnaires. Little, if any, support is incorporated, however, within these tools to guide online-questionnaire designers according to best practice. In essence, an online-questionnaire combines questionnaire-based survey functionality with that of a webpage/site. As such, the design of an online-questionnaire should incorporate principles from both contributing fields. Drawing on existing guidelines for paper-based questionnaire design, website design (paying particular attention to issues of accessibility and usability), and existing but scarce guidelines for electronic surveys, we have derived a comprehensive set of guidelines for the design of online-questionnaires. This article introduces this comprehensive set of guidelines – as a practical reference guide – for the design of online-questionnaires.

Guidelines for the design of online-questionnaires

As a new medium for questionnaire delivery, the internet has the potential to revolutionise the survey process. Online (web-based) questionnaires provide several advantages over traditional survey methods in terms of cost, speed, appearance, flexibility, functionality, and usability (Bandilla et al., 2003; Dillman, 2000; Kwak and Radler, 2002). Online-questionnaires can also provide many capabilities not found in traditional paper-based questionnaires: they can include pop-up instructions and error messages; they can incorporate links; and it is possible to encode difficult skip patterns making such patterns virtually invisible to respondents. Despite this, and the introduction of numerous tools to support online-questionnaire creation, current electronic survey design typically replicates that of paper-based questionnaires, failing to harness the full power of the electronic delivery medium. Worse, a recent environmental scan of online-questionnaire design tools found that little, if any, support is incorporated within these tools to guide questionnaire designers according to best-practice (Lumsden and Morgan, 2005). This article introduces a comprehensive set of guidelines - a practical reference guide - for the design of online-questionnaires.

Design of the low vision quality-of-life questionnaire (LVQOL) and measuring the outcome of low-vision rehabilitation

PURPOSE: To design and validate a vision-specific quality-of-life assessment tool to be used in a clinical setting to evaluate low-vision rehabilitation strategy and management. METHODS: Previous vision-related questionnaires were assessed by low-vision rehabilitation professionals and patients for relevance and coverage. The 74 items selected were pretested to ensure correct interpretation. One hundred and fifty patients with low vision completed the chosen questions on four occasions to allow the selection of the most appropriate items. The vision-specific quality of life of patients with low vision was compared with that of 70 age-matched and gender-matched patients with normal vision and before and after low-vision rehabilitation in 278 patients. RESULTS: Items that were unreliable, internally inconsistent, redundant, or not relevant were excluded, resulting in the 25-item Low Vision Quality-of-Life Questionnaire (LVQOL). Completion of the LVQOL results in a summed score between 0 (a low quality of life) and 125 (a high quality of life). The LVQOL has a high internal consistency (α = 0.88) and good reliability (0.72). The average LVQOL score for a population with low vision (60.9 ± 25.1) was significantly lower than the average score of those with normal vision (100.3 ± 20.8). Rehabilitation improved the LVQOL score of those with low vision by an average of 6.8 ± 15.6 (17%). CONCLUSIONS: The LVQOL was shown to be an internally consistent, reliable, and fast method for measuring the vision-specific quality of life of the visually impaired in a clinical setting. It is able to quantify the quality of life of those with low vision and is useful in determining the effects of low-vision rehabilitation. Copyright (C) 2000 Elsevier Science Inc.

Development of a questionnaire to assess the relative subjective benefits of presbyopia correction

Purpose To develop a standardized questionnaire of near visual function and satisfaction to complement visual function evaluations of presbyopic corrections. Setting Eye Clinic, School of Life and Health Sciences, Aston University, Midland Eye Institute and Solihull Hospital, Birmingham, United Kingdom. Design Questionnaire development. Methods A preliminary 26-item questionnaire of previously used near visual function items was completed by patients with monofocal intraocular lenses (IOLs), multifocal IOLs, accommodating IOLs, multifocal contact lenses, or varifocal spectacles. Rasch analysis was used for item reduction, after which internal and test–retest reliabilities were determined. Construct validity was determined by correlating the resulting Near Activity Visual Questionnaire (NAVQ) scores with near visual acuity and critical print size (CPS), which was measured using the Minnesota Low Vision Reading Test chart. Discrimination ability was assessed through receiver-operating characteristic (ROC) curve analysis. Results One hundred fifty patients completed the questionnaire. Item reduction resulted in a 10-item NAVQ with excellent separation (2.92), internal consistency (Cronbach a = 0.95), and test–retest reliability (intraclass correlation coefficient = 0.72). Correlations of questionnaire scores with near visual acuity (r = 0.32) and CPS (r = 0.27) provided evidence of validity, and discrimination ability was excellent (area under ROC curve = 0.91). Conclusion Results show the NAVQ is a reliable, valid instrument that can be incorporated into the evaluation of presbyopic corrections.

The effect of ward design on the well-being of post-operative patients

Changes in the design of hospital wards have usually been determined by architects and members of the nursing and medical professions; the views and preferences of patients have seldom been sought directly. The Hospital Anxiety and Depression scale and the Disturbance Due to Hospital Noise questionnaire were administered to 64 female patients on bay and Nightingale wards together with a questionnaire designed for this study. Perceptions of social and physical factors of ward design were examined, and their relationship to psychological well-being and sleep patterns. The results show that the bay ward seemed to offer a more favourable environment for patients but some of the disadvantages of bay wards are balanced by better staffing levels and better and more modern facilities. Visibility to nurses was lower on the bay ward. The Nightingale ward was perceived as significantly noisier than the bay ward and noise levels were significantly correlated to anxiety scores. Paradoxically the increase in noise levels appeared to improve the perceived level of privacy on the Nightingale ward. Seventy-five per cent of patients were found to prefer the bay ward design, and since neither design appears to have major disadvantages their continued introduction should be encouraged. However, recommendations are made concerning the optimizing of patients' well-being within the bay ward setting.

Teaching our children when to eat:how parental feeding practices inform the development of emotional eating-a longitudinal experimental design

Background: Emotional eating in children has been related to the consumption of energy-dense foods and obesity, but the development of emotional eating in young children is poorly understood. Objectives: We evaluated whether emotional eating can be induced in 5-7-y-old children in the laboratory and assessed whether parental use of overly controlling feeding practices at 3-5 y of age predicts a greater subsequent tendency for children to eat under conditions of mild stress at ages 5-7 y. Design: Forty-one parent-child dyads were recruited to participate in this longitudinal study, which involved parents and children being observed consuming a standard lunch, completing questionnaire measures of parental feeding practices, participating in a research procedure to induce child emotion (or a control procedure), and observing children's consumption of snack foods. Results: Children at ages 5-7 y who were exposed to a mild emotional stressor consumed significantly more calories from snack foods in the absence of hunger than did children in a control group. Parents who reported the use of more food as a reward and restriction of food for health reasons with their children at ages 3-5 y were more likely to have children who ate more under conditions of negative emotion at ages 5-7 y. Conclusions: Parents who overly control children's food intake may unintentionally teach children to rely on palatable foods to cope with negative emotions. Additional research is needed to evaluate the implications of these findings for children's food intake and weight outside of the laboratory setting. This trial was registered at clinicaltrials.gov as NCT01122290.

Collaborative design of assessment criteria to improve undergraduate student engagement and performance

Student engagement is vital in enhancing the student experience and encouraging deeper learning. Involving students in the design of assessment criteria is one way in which to increase student engagement. In 2011, a marking matrix was used at Aston University (UK) for logbook assessment (Group One) in a project-based learning module. The next cohort of students in 2012 (Group Two) were asked to collaboratively redesign the matrix and were given a questionnaire about the exercise. Group Two initially scored a lower average logbook mark than Group One. However, Group Two showed the greatest improvement between assessments, and the quality of, and commitment to, logbooks was noticeably improved. Student input resulted in a more defined, tougher mark scheme. However, this provided an improved feedback system that gave more scope for self-improvement. The majority of students found the exercise incorporated their ideas, enhanced their understanding, and was useful in itself.

Implementation of supply chain management theory in practice:a questionnaire survey in the United Kingdom

Purpose: The purpose of the research described in this paper is to disentangle the rhetoric from the reality in relation to supply chain management (SCM) adoption in practice. There is significant evidence of a divergence between theory and practice in the field of SCM. Research Approach: The authors’ review of the extant SCM literature highlighted a lack of replication studies in SCM, leading to the concept of refined replication being developed. The authors conducted a refined replication of the work of Sweeney et al. (2015) where a new SCM definitional construct – the Four Fundamentals – was proposed. The work presented in this article refines the previous study but adopts the same three-phase approach: focussed interviews, a questionnaire survey, and focus groups. This article covers the second phase of the refined replication study and describes an integrated research design of a questionnaire research to be undertaken in Britain. Findings and Originality: The article presents an integrated research design of a questionnaire research with emphases on the refined replication of previous work of Sweeney et al. (2015) carried out in Ireland and adapting it to the British context. Research Impact: The authors introduce the concept of refined replication in SCM research. This allows previous research to be built upon in order to test understanding of SCM theory and its practical implementation - based on the Four Fundamentals construct - among SCM professionals in Britain. Practical Impact: The article presents the integrated research design of a questionnaire research that may be used in similar studies.