Online-questionaire design guidelines

As a new medium for questionnaire delivery, the internet has the potential to revolutionize the survey process. Online (web-based) questionnaires provide several advantages over traditional survey methods in terms of cost, speed, appearance, flexibility, functionality, and usability. Designers of online-questionnaires are faced with a plethora of design tools to assist in the development of their electronic questionnaires. Little, if any, support is incorporated, however, within these tools to guide online-questionnaire designers according to best practice. In essence, an online-questionnaire combines questionnaire-based survey functionality with that of a webpage/site. As such, the design of an online-questionnaire should incorporate principles from both contributing fields. Drawing on existing guidelines for paper-based questionnaire design, website design (paying particular attention to issues of accessibility and usability), and existing but scarce guidelines for electronic surveys, we have derived a comprehensive set of guidelines for the design of online-questionnaires. This article introduces this comprehensive set of guidelines – as a practical reference guide – for the design of online-questionnaires.

Online-questionnaire design:establishing guidelines and evaluating existing eupport

As a new medium for questionnaire delivery, the internet has the potential to revolutionise the survey process. Online (web-based) questionnaires provide several advantages over traditional survey methods in terms of cost, speed, appearance, flexibility, functionality, and usability [1, 2]. For instance, delivery is faster, responses are received more quickly, and data collection can be automated or accelerated [1- 3]. Online-questionnaires can also provide many capabilities not found in traditional paper-based questionnaires: they can include pop-up instructions and error messages; they can incorporate links; and it is possible to encode difficult skip patterns making such patterns virtually invisible to respondents. Like many new technologies, however, online-questionnaires face criticism despite their advantages. Typically, such criticisms focus on the vulnerability of online-questionnaires to the four standard survey error types: namely, coverage, non-response, sampling, and measurement errors. Although, like all survey errors, coverage error (“the result of not allowing all members of the survey population to have an equal or nonzero chance of being sampled for participation in a survey” [2, pg. 9]) also affects traditional survey methods, it is currently exacerbated in online-questionnaires as a result of the digital divide. That said, many developed countries have reported substantial increases in computer and internet access and/or are targeting this as part of their immediate infrastructural development [4, 5]. Indicating that familiarity with information technologies is increasing, these trends suggest that coverage error will rapidly diminish to an acceptable level (for the developed world at least) in the near future, and in so doing, positively reinforce the advantages of online-questionnaire delivery. The second error type – the non-response error – occurs when individuals fail to respond to the invitation to participate in a survey or abandon a questionnaire before it is completed. Given today’s societal trend towards self-administration [2] the former is inevitable, irrespective of delivery mechanism. Conversely, non-response as a consequence of questionnaire abandonment can be relatively easily addressed. Unlike traditional questionnaires, the delivery mechanism for online-questionnaires makes estimation of questionnaire length and time required for completion difficult1, thus increasing the likelihood of abandonment. By incorporating a range of features into the design of an online questionnaire, it is possible to facilitate such estimation – and indeed, to provide respondents with context sensitive assistance during the response process – and thereby reduce abandonment while eliciting feelings of accomplishment [6]. For online-questionnaires, sampling error (“the result of attempting to survey only some, and not all, of the units in the survey population” [2, pg. 9]) can arise when all but a small portion of the anticipated respondent set is alienated (and so fails to respond) as a result of, for example, disregard for varying connection speeds, bandwidth limitations, browser configurations, monitors, hardware, and user requirements during the questionnaire design process. Similarly, measurement errors (“the result of poor question wording or questions being presented in such a way that inaccurate or uninterpretable answers are obtained” [2, pg. 11]) will lead to respondents becoming confused and frustrated. Sampling, measurement, and non-response errors are likely to occur when an online-questionnaire is poorly designed. Individuals will answer questions incorrectly, abandon questionnaires, and may ultimately refuse to participate in future surveys; thus, the benefit of online questionnaire delivery will not be fully realized. To prevent errors of this kind2, and their consequences, it is extremely important that practical, comprehensive guidelines exist for the design of online questionnaires. Many design guidelines exist for paper-based questionnaire design (e.g. [7-14]); the same is not true for the design of online questionnaires [2, 15, 16]. The research presented in this paper is a first attempt to address this discrepancy. Section 2 describes the derivation of a comprehensive set of guidelines for the design of online-questionnaires and briefly (given space restrictions) outlines the essence of the guidelines themselves. Although online-questionnaires reduce traditional delivery costs (e.g. paper, mail out, and data entry), set up costs can be high given the need to either adopt and acquire training in questionnaire development software or secure the services of a web developer. Neither approach, however, guarantees a good questionnaire (often because the person designing the questionnaire lacks relevant knowledge in questionnaire design). Drawing on existing software evaluation techniques [17, 18], we assessed the extent to which current questionnaire development applications support our guidelines; Section 3 describes the framework used for the evaluation, and Section 4 discusses our findings. Finally, Section 5 concludes with a discussion of further work.

What difference doguidelines make? An observational study of online-questionnaire design guidelines put to practical use

As a new medium for questionnaire delivery, the Internet has the potential to revolutionize the survey process. Online-questionnaires can provide many capabilities not found in traditional paper-based questionnaires. Despite this, and the introduction of a plethora of tools to support online-questionnaire creation, current electronic survey design typically replicates the look-and-feel of paper-based questionnaires, thus failing to harness the full power of the electronic delivery medium. A recent environmental scan of online-questionnaire design tools found that little, if any, support is incorporated within these tools to guide questionnaire designers according to best-practice [Lumsden & Morgan 2005]. This paper briefly introduces a comprehensive set of guidelines for the design of online-questionnaires. Drawn from relevant disparate sources, all the guidelines incorporated within the set are proven in their own right; as an initial assessment of the value of the set of guidelines as a practical reference guide, we undertook an informal study to observe the effect of introducing the guidelines into the design process for a complex online-questionnaire. The paper discusses the qualitative findings — which are encouraging for the role of the guidelines in the ‘bigger picture’ of online survey delivery across many domains such as e-government, e-business, and e-health — of this case study.

Guidelines for the design of online-questionnaires

As a new medium for questionnaire delivery, the internet has the potential to revolutionise the survey process. Online (web-based) questionnaires provide several advantages over traditional survey methods in terms of cost, speed, appearance, flexibility, functionality, and usability (Bandilla et al., 2003; Dillman, 2000; Kwak and Radler, 2002). Online-questionnaires can also provide many capabilities not found in traditional paper-based questionnaires: they can include pop-up instructions and error messages; they can incorporate links; and it is possible to encode difficult skip patterns making such patterns virtually invisible to respondents. Despite this, and the introduction of numerous tools to support online-questionnaire creation, current electronic survey design typically replicates that of paper-based questionnaires, failing to harness the full power of the electronic delivery medium. Worse, a recent environmental scan of online-questionnaire design tools found that little, if any, support is incorporated within these tools to guide questionnaire designers according to best-practice (Lumsden and Morgan, 2005). This article introduces a comprehensive set of guidelines - a practical reference guide - for the design of online-questionnaires.

Designing and implementing a blended learning approach to executive SCM education:a case study from Ireland

The National Institute for Transport and Logistics (NITL) is Ireland’s centre of excellence for supply chain management (SCM). As part of its mission to promote the development of supply chain expertise in Irish business, it designs and delivers executive modular learning programmes. In 2004, as part of a drive to create more flexible learning opportunities for course participants, NITL designed and implemented an eLearning programme, which involved converting traditionally tutored modules to online modules. This paper describes the rationale behind this initiative and the significance of technology as an enabling tool for executive education, as well as detailing the design and implementation processes for the pilot module. The paper concludes with a critique of the expected and actual benefits realised, as well as future development considerations.

Online Education: What Works in Mathematics

Florida State University and University of Helsinki Information technology has the potential to deliver education to everybody by high quality online courses and associated services, and to enhance traditional face-to-face instruction by, e.g., web services offering virtually unlimited practice and step-bystep solutions to practice problems. Regardless of this, tools of information technology have not yet penetrated mathematics education in any meaningful way. This is mostly due to the inertia of academia: instructors are slow to change their working habits. This paper reports on an experiment where all the instructors (seven instructors and six teaching assistants) of a large calculus course were required to base their instruction on online content. The paper will analyze the effectiveness of various solutions used, and finishes with recommendations regarding best practices.

Essential features of serious games design in higher education:linking learning attributes to game mechanics

This paper consolidates evidence and material from a range of specialist and disciplinary fields to provide an evidence-based review and synthesis on the design and use of serious games in higher education. Search terms identified 165 papers reporting conceptual and empirical evidence on how learning attributes and game mechanics may be planned, designed and implemented by university teachers interested in using games, which are integrated into lesson plans and orchestrated as part of a learning sequence at any scale. The findings outline the potential of classifying the links between learning attributes and game mechanics as a means to scaffold teachers’ understanding of how to perpetuate learning in optimal ways while enhancing the in-game learning experience. The findings of this paper provide a foundation for describing methods, frames and discourse around experiences of design and use of serious games, linked to methodological limitations and recommendations for further research in this area.

Universal Design for Learning and Differentiated Instruction: Resolving Competing Mandates of the Individuals with Disabilities Education Act and No Child Left Behind

Recent federal mandates require accountability for providing students with disabilities access to the general education curriculum. In this paper, the authors recommend that principles of Universal Design for Learning and Differentiated Instruction can help school personnel tailor their teaching to meet the various strengths and needs of individual students.

Neuroangradogy in Online Education

The purpose of this perspective in practice essay is to address the effectiveness of online learning based on adult brain-based education. This essay attempts to answer the question: Do what we know about the brain (neurology) and adult teaching methods (andragogy) play a key role in learning in online courses?

Users Acceptance of Online Enrollment Processes in a Higher Education Institution

Little is known about students’ perceptions of online enrollment processes. Student satisfaction is part of the assessment required for accreditation, but evidence suggests that college administrators are oriented to retention and graduation rates rather than to consumer perception. The purpose of this descriptive quantitative study was to develop and validate a model that enables the measurement of online enrollment processes by the analysis of the students’ perceptions. The theoretical framework used to support this study was the process virtualization theory while the conceptual framework was based on Technology Acceptance Model (TAM). TAM is the most valid framework for studying user acceptance of technology and virtual processes. The model was modified, adding a new variable to fit this study. Research questions were used to determine if an institution knows how its students perceive online enrollment processes and how they can become more efficient and effective, improving usage and satisfaction. Descriptive data were collected and analyzed in phases: the pilot study phase, data collection phase, and analysis phases. Inferential statistics were used to draw information from sampled observations of the population; a Cronbach Alpha was conducted to determine the reliability and validity of the model. The study demonstrated that the modified TAM is valid, reliable, and fit to assess the perceptions of the users of online enrollment processes. This study will effect positive social change by providing enrollment managers and administrators information on how to analyze the acceptance their online enrollment processes from the perspective of their students as customers of an institution of higher learning.

A Basic Interpretive Study of the Experiences of University Students Who Have Dropped or Failed an Online Course

Online courses have increased in enrollments over the past few decades. As the number of students taking online courses have increased, so has the number of students who have dropped or failed an online course. According to the literature, online courses may have higher drop rates than traditional, face-to-face courses. The number of students who fail an online course is, also, of concern. As online courses may continue to grow over the next few decades, studies on persistence in online courses may benefit students, administrators, instructional designers, educators, and researchers. Although previous research studies have addressed persistence in online courses, very few examine it from the perspectives of students who were unsuccessful in their courses. These students may have unique insights about the online experience that may have related to their lack of success. The purpose of this study was to understand the experiences of university students who have failed or dropped an online course through the lenses of transactional distance theory and Kember’s model of dropout in distance education. Transactional distance theory discusses the dialog, structure, and learner autonomy involved in an online course, while, Kember’s model presents categories that may relate to dropping an online course. Together, the theory and model may help in understanding the experiences of students who have dropped or failed an online course. In this study, semi-structured interviews were conducted with 20 participants from a large Southeastern university in the United States. Based on the participants’ responses, the data was sorted and ranked according to the amount of transactional distance in their courses, as well as the categories of Kember’s model. Many of the participants who experienced low or high transactional distance have, also, expressed an issue with the goal commitment category of Kember’s model. Additionally, there were important differences in the student characteristics of those who dropped or failed an online course. Furthermore, suggestions for improving online courses were given by the participants. Some of these suggestions included more student-instructor interactions, the use of more technology tools in their online course, and for orientations to the online environment to be offered.

National, inter-institutional, graduate modules – blended learning approaches for PhD education in Ireland

This paper is a case study that describes the design and delivery of national PhD lectures with 40 PhD candidates in Digital Arts and Humanities in Ireland simultaneously to four remote locations, in Trinity College Dublin, in University College Cork, in NUI Maynooth and NUI Galway. Blended learning approaches were utilized to augment traditional teaching practices combining: face-to-face engagement, video-conferencing to multiple sites, social media lecture delivery support – a live blog and micro blogging, shared, open student web presence online. Techniques for creating an effective, active learning environment were discerned via a range of learning options offered to students through student surveys after semester one. Students rejected the traditional lecture format, even through the novel delivery method via video link to a number of national academic institutions was employed. Students also rejected the use of a moderated forum as a means of creating engagement across the various institutions involved. Students preferred a mix of approaches for this online national engagement. The paper discusses successful methods used to promote interactive teaching and learning. These included Peer to peer learning, Workshop style delivery, Social media. The lecture became a national, synchronous workshop. The paper describes how allowing students to have a voice in the virtual classroom they become animated and engaged in an open culture of shared experience and scholarship, create networks beyond their institutions, and across disciplinary boundaries. We offer an analysis of our experiences to assist other educators in their course design, with a particular emphasis on social media engagement.

Toward an Understanding of Faculty Perceptions about Factors that Influence Student Success in Online Education

This paper explores factors that faculty perceive as significant in influencing student success in online education. Technological support, degree of comfort with technology and a student’s personal characteristics were deemed by faculty as critical to success. Understanding these key factors could help to improve student success in online education.

Accountability, Quality, and Student Success in Online Education: A Literature Review of Empirical Studies

This paper examines empirical studies on accountability, quality, and student success in online education. It advances that accountability and quality are critical components for student success in online education. It concludes that there is a lack of empirical studies that examine the effects of these measures on student success.