Online versus postal data collection methods: an examination of issues and a comparison of results

New communications technologies often allow new ways of conducting market research. Determining the advantages of a new data collection method over established alternatives is difficult without thorough comparative testing. Computer-mediated marketing research is one such example of a new technology that has been enthusiastically embraced by marketing organisations and those servicing them. While researchers using the Internet (Net) and World Wide Web (Web) in its early years reported benefits such as high response levels, there is little in the way of comparative evidence to support any claimed advantages. This paper reports on the outcomes of three separate studies in which members (subscribers) of various organisations have been surveyed using both postal and online (email invitation and HTML Web form) data collection methods. The conclusion here is that it would be unwise to assume that one method can be directly substituted for another and obtain the same response. Differences in both the response pattern and demographic profile of respondents between the groups are consistently noticed, such as to warrant further examination of the methods used in online marketing research, and to suggest the need for further study.

Online and postal data collection methods: a comparative study

Computer-mediated marketing research has been enthusiastically embraced by marketing organisations and those servicing them, for many reasons. While researchers using the Internet (Net) and World Wide Web (Web) in its early years reported benefits such as high response levels, there are now issues in this regard. This paper reports on the outcomes of a probabilistic study involving football club members (subscribers) involving both postal and online (e-mail invitation and HTML Web form) data collection methods. The paper reports differences in both the response pattern and demographic profile of respondents between the groups such as to warrant further examination of the methods used in online marketing research, and to suggest the need for further study.

Small business perceptions of postal and online survey research

Concerns over the overall cost of marketing research and the cost per usable response have in large measure caused marketing practitioners to turn to online marketing research techniques, either as a solus technique, or in a mixed mode application. However, the use of e-mail and mixed mode surveys such as postal invitations to complete online questionnaires present both familiar and new issues, as the extant literature illustrates. This paper examines an earlier study before reporting findings from the present study, which employs a method that ascertains the probability of commissioning four survey research methods, described in scenarios and delivered using e-mail and the World Wide Web (Web). It is evident that while perceptions of e-mail, the Internet, and privacy have changed since early use of the Internet and more particularly the World Wide Web, and there is acknowledgement in the literature concerning the lower costs and faster response speeds of online marketing research, small businesses do appear to discriminate in favour of targeted online survey methods over postal surveys, portrayed as scenarios in this study, but that this perception does not apply to all hybrid survey modes.

Online versus postal data collection : a re-examination of the response differences using demographically matched data

Past research has identified differences between online and mail collected responses to the same survey, but differences in the demographics of respondents had also been noted making the cause of the variation unclear. In the research reported here, responses to the same questionnaire, delivered via mail and internet surveys, were demographically matched across a range of variables. This removed the impact of response differences caused by age, gender, type of product consumed and length of customer relationship. Across all the different question types and response scales, significant differences were still found between mail and online respondents, even when data were ipsatised. Notably, online respondents were far less likely to use the end-points of the scale, perhaps indicating issues with the online collection methodology. The conclusion is that the two methods of data collection can not be assumed to be directly inter-changeable, and that the method used can lead to different results if not managed carefully.

Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey : randomised controlled trial

Background: Minimising participant non-response in postal surveys helps to maximise the generalisability of the inferences made from the data collected. The aim of this study was to examine the effect of questionnaire length, personalisation and reminder type on postal survey response rate and quality and to compare the costeffectiveness of the alternative survey strategies.
Methods: In a pilot study for a population study of travel behaviour, physical activity and the environment, 1000 participants sampled from the UK edited electoral register were randomly allocated using a 2 × 2 factorial design to receive one of four survey packs: a personally addressed long (24 page) questionnaire pack, a personally addressed short (15 page) questionnaire pack, a non-personally addressed long questionnaire pack or a nonpersonally addressed short questionnaire pack. Those who did not return a questionnaire were stratified by initial randomisation group and further randomised to receive either a full reminder pack or a reminder postcard. The effects of the survey design factors on response were examined using multivariate logistic regression.
Results: An overall response rate of 17% was achieved. Participants who received the short version of the questionnaire were more likely to respond (OR = 1.48, 95% CI 1.06 to 2.07). In those participants who received a reminder, personalisation of the survey pack and reminder also increased the odds of response (OR = 1.44, 95% CI 1.01 to 1.95). Item non-response was relatively low, but was significantly higher in the long questionnaire than the short (9.8% vs 5.8%; p = .04). The cost per additional usable questionnaire returned of issuing the reminder packs was £23.1 compared with £11.3 for the reminder postcards.
Conclusions: In contrast to some previous studies of shorter questionnaires, this trial found that shortening a relatively lengthy questionnaire significantly increased the response. Researchers should consider the trade off between the value of additional questions and a larger sample. If low response rates are expected, personalisation may be an important strategy to apply. Sending a full reminder pack to non-respondents appears a worthwhile, albeit more costly, strategy.