Online versus postal data collection methods: an examination of issues and a comparison of results

New communications technologies often allow new ways of conducting market research. Determining the advantages of a new data collection method over established alternatives is difficult without thorough comparative testing. Computer-mediated marketing research is one such example of a new technology that has been enthusiastically embraced by marketing organisations and those servicing them. While researchers using the Internet (Net) and World Wide Web (Web) in its early years reported benefits such as high response levels, there is little in the way of comparative evidence to support any claimed advantages. This paper reports on the outcomes of three separate studies in which members (subscribers) of various organisations have been surveyed using both postal and online (email invitation and HTML Web form) data collection methods. The conclusion here is that it would be unwise to assume that one method can be directly substituted for another and obtain the same response. Differences in both the response pattern and demographic profile of respondents between the groups are consistently noticed, such as to warrant further examination of the methods used in online marketing research, and to suggest the need for further study.

Online and postal data collection methods: a comparative study

Computer-mediated marketing research has been enthusiastically embraced by marketing organisations and those servicing them, for many reasons. While researchers using the Internet (Net) and World Wide Web (Web) in its early years reported benefits such as high response levels, there are now issues in this regard. This paper reports on the outcomes of a probabilistic study involving football club members (subscribers) involving both postal and online (e-mail invitation and HTML Web form) data collection methods. The paper reports differences in both the response pattern and demographic profile of respondents between the groups such as to warrant further examination of the methods used in online marketing research, and to suggest the need for further study.

Online versus postal data collection : a re-examination of the response differences using demographically matched data

Past research has identified differences between online and mail collected responses to the same survey, but differences in the demographics of respondents had also been noted making the cause of the variation unclear. In the research reported here, responses to the same questionnaire, delivered via mail and internet surveys, were demographically matched across a range of variables. This removed the impact of response differences caused by age, gender, type of product consumed and length of customer relationship. Across all the different question types and response scales, significant differences were still found between mail and online respondents, even when data were ipsatised. Notably, online respondents were far less likely to use the end-points of the scale, perhaps indicating issues with the online collection methodology. The conclusion is that the two methods of data collection can not be assumed to be directly inter-changeable, and that the method used can lead to different results if not managed carefully.

What have sampling and data collection got to do with good qualitative research?

Objective: To highlight the importance of sampling and data collection  processes in qualitative interview studies, and to discuss the contribution of  these processes to determining the strength of the evidence generated and  thereby to decisions for public health practice and policy.

Approach: This discussion is informed by a hierarchy-of-evidence-for-practice  model. The paper provides succinct guidelines for key sampling and data  collection considerations in qualitative research involving interview studies. The  importance of allowing time for immersion in a given community to become  familiar with the context and population is discussed, as well as the practical  constraints that sometimes operate against this stage. The role of theory in  guiding sample selection is discussed both in terms of identifying likely sources  of rich data and in understanding the issues emerging from the data. It is noted  that sampling further assists in confirming the developing evidence and also  illuminates data that does not seem to fit. The importance of reporting sampling  and data collection processes is highlighted clearly to enable others to assess  both the strength of the evidence and the broader applications of the findings.

Conclusion: Sampling and data collection processes are critical to determining  the quality of a study and the generalisability of the findings. We argue that  these processes should operate within the parameters of the research goal, be  guided by emerging theoretical considerations, cover a range of relevant   participant perspectives, and be clearly outlined in research reports with an  explanation of any research limitations.

Qualitative methods in early-phase drug trials : data and methods from a trial of N-acetylcysteine in schizophrenia

Objective: The pharmacokinetic profile of a drug often gives little indication of its potential therapeutic application, with many therapeutic uses of drugs being discovered serendipitously while being studied for different indications. As hypothesis-driven, quantitative research methodology is exclusively used in early-phase trials, unexpected but important phenomena may escape detection. In this context, this study aimed to examine the potential for integrating qualitative research methods with quantitative methods in early-phase drug trials. To our knowledge, this mixed methodology has not previously been applied to blinded psychopharmacologic trials.

Method: We undertook qualitative data analysis of clinical observations on the dataset of a randomized, double-blind, placebo-controlled trial of N-acetylcysteine (NAC) in patients with DSM-IV-TR–diagnosed schizophrenia (N = 140). Textual data on all participants, deliberately collected for this purpose, were coded using NVivo 2, and emergent themes were analyzed in a blinded manner in the NAC and placebo groups. The trial was conducted from November 2002 to July 2005.

Results: The principal findings of the published trial could be replicated using a qualitative methodology. In addition, significant differences between NAC- and placebo-treated participants emerged for positive and affective symptoms, which had not been captured by the rating scales utilized in the quantitative trial. Qualitative data in this study subsequently led to a positive trial of NAC in bipolar disorder.

Conclusions: The use of qualitative methods may yield broader data and has the potential to complement traditional quantitative methods and detect unexpected efficacy and safety signals, thereby maximizing the findings of early-phase clinical trial research.

A pilot, multisite, randomized controlled trial of a self-directed coping skills training intervention for couples facing prostate cancer : accrual, retention, and data collection issues

PURPOSE: To examine the acceptability of the methods used to evaluate Coping-Together, one of the first self-directed coping skill intervention for couples facing cancer, and to collect preliminary efficacy data. METHODS: Forty-two couples, randomized to a minimal ethical care (MEC) condition or to Coping-Together, completed a survey at baseline and 2 months after, a cost diary, and a process evaluation phone interview. RESULTS: One hundred seventy patients were referred to the study. However, 57 couples did not meet all eligibility criteria, and 51 refused study participation. On average, two to three couples were randomized per month, and on average it took 26 days to enrol a couple in the study. Two couples withdrew from MEC, none from Coping-Together. Only 44 % of the cost diaries were completed, and 55 % of patients and 60 % of partners found the surveys too long, and this despite the follow-up survey being five pages shorter than the baseline one. Trends in favor of Coping-Together were noted for both patients and their partners. CONCLUSIONS: This study identified the challenges of conducting dyadic research, and a number of suggestions were put forward for future studies, including to question whether distress screening was necessary and what kind of control group might be more appropriate in future studies.

Monitoring and evaluating surgical care: defining perioperative mortality rate and standardising data collection

BACKGROUND: Case volume per 100 000 population and perioperative mortality rate (POMR) are key indicators to monitor and strengthen surgical services. However, comparisons of POMR have been restricted by absence of standardised approaches to when it is measured, the ideal denominator, need for risk adjustment, and whether data are available. We aimed to address these issues and recommend a minimum dataset by analysing four large mixed surgical datasets, two from well-resourced settings with sophisticated electronic patient information systems and two from resource-limited settings where clinicians maintain locally developed databases. METHODS: We obtained data from the New Zealand (NZ) National Minimum Dataset, the Geelong Hospital patient management system in Australia, and purpose-built surgical databases in Pietermaritzburg, South Africa (PMZ) and Port Moresby, Papua New Guinea (PNG). Information was sought on inclusion and exclusion criteria, coding criteria, and completeness of patient identifiers, admission, procedure, discharge and death dates, operation details, urgency of admission, and American Society of Anesthesiologists (ASA) score. Date-related errors were defined as missing dates and impossible discrepancies. For every site, we then calculated the POMR, the effect of admission episodes or procedures as denominator, and the difference between in-hospital POMR and 30-day POMR. To determine the need for risk adjustment, we used univariate and multivariate logistic regression to assess the effect on relative POMR for each site of age, admission urgency, ASA score, and procedure type. FINDINGS: 1 365 773 patient admissions involving 1 514 242 procedures were included, among which 8655 deaths were recorded within 30 days. Database inclusion and exclusion criteria differed substantially. NZ and Geelong records had less than 0·1% date-related errors and greater than 99·9% completeness. PMZ databases had 99·9% or greater completeness of all data except date-related items (94·0%). PNG had 99·9% or greater completeness for date of birth or age and admission date and operative procedure, but 80-83% completeness of patient identifiers and date related items. Coding of procedures was not standardised, and only NZ recorded ASA status and complete post-discharge mortality. In-hospital POMR range was 0·38% in NZ to 3·44% in PMZ, and in NZ it underestimated 30-day POMR by roughly a third. The difference in POMR by procedures instead of admission episodes as denominator ranged from 10% to 70%. Age older than 65 years and emergency admission had large independent effects on POMR, but relatively little effect in multivariate analysis on the relative odds of in-hospital death at each site. INTERPRETATION: Hospitals can collect and provide data for case volume and POMR without sophisticated electronic information systems. POMR should initially be defined by in-hospital mortality because post-discharge deaths are not usually recorded, and with procedures as denominator because details allowing linkage of several operations within one patient's admission are not always present. Although age and admission urgency are independently associated with POMR, and ASA and case mix were not included, risk adjustment might not be essential because the relative odds between sites persisted. Standardisation of inclusion criteria and definitions is needed, as is attention to accuracy and completeness of dates of procedures, discharge and death. A one-page, paper-based form, or alternatively a simple electronic data collection form, containing a minimum dataset commenced in the operating theatre could facilitate this process. FUNDING: None.

Screening for post 32-week preterm birth risk: how helpful is routine perinatal data collection?

BACKGROUND: Preterm birth is a clinical event significant but difficult to predict. Biomarkers such as fetal fibronectin and cervical length are effective, but the often are used only for women with clinically suspected preterm risk. It is unknown whether routinely collected data can be used in early pregnancy to stratify preterm birth risk by identifying asymptomatic women. This paper tries to determine the value of the Victorian Perinatal Data Collection (VPDC) dataset in predicting preterm birth and screening for invasive tests.

METHODS: De-identified VPDC report data from 2009 to 2013 were extracted for patients from Barwon Health in Victoria. Logistic regression models with elastic-net regularization were fitted to predict 37-week preterm, with the VPDC antenatal variables as predictors. The models were also extended with two additional variables not routinely noted in the VPDC: previous preterm birth and partner smoking status, testing the hypothesis that these two factors add prediction accuracy. Prediction performance was evaluated using a number of metrics, including Brier scores, Nagelkerke's R(2), c statistic.

RESULTS: Although the predictive model utilising VPDC data had a low overall prediction performance, it had a reasonable discrimination (c statistic 0.646 [95% CI: 0.596-0.697] for 37-week preterm) and good calibration (goodness-of-fit p = 0.61). On a decision threshold of 0.2, a Positive Predictive Value (PPV) of 0.333 and a negative predictive value (NPV) of 0.941 were achieved. Data on previous preterm and partner smoking did not significantly improve prediction.

CONCLUSIONS: For multiparous women, the routine data contains information comparable to some purposely-collected data for predicting preterm risk. But for nulliparous women, the routine data contains insufficient data related to antenatal complications.

Clinical research 5: quantitative data collection and analysis

This six-part research series is aimed at clinicians who wish to develop research skills, or who have a particular clinical problem that they think could be addressed through research. The series aims to provide insight into the decisions that researchers make in the course of their work, and to also provide a foundation for decisions that nurses may make in applying the findings of a study to practice in their own Unit or Department. The series emphasises the practical issues encountered when undertaking research in critical care settings; readers are encouraged to source research methodology textbooks for more detailed guidance on specific aspects of the research process.

A couple of points:

1. It is artificial to describe research as qualitative or quantitative. Studies often include both dimensions. However, for the purposes of this paper/series, this distinction is drawn for clarity of writing.

2. It is common practice for quantitative studies to refer to study ‘subjects’ and qualitative studies to refer to study ‘participants’. For ease of reading, the latter term will be used throughout this series.

Research methods in sales ethics: a review of the empirical literature

From a review of the empirical sales ethics literature, this paper reports findings about some of the research methods used to investigate the decision-making of sales practitioners under ethical conditions. The review identifies that several of the methodological deficiencies raised by previous reviewers of the literature have not been adequately addressed by subsequent researchers. The paper primarily reviews quantitative research studies because of their prevalence in the empirical sales ethics literature, and because studies similar to these have contributed much to marketing ethics theory. This discussion also focuses on sampling and data collection methods, the treatment of respondent and non-response bias, the use of instruments and scales, and the application of the scenario technique. Some suggestions are made that would improve the research methods in each of these areas.

Using autonomous mobile agents for efficient data collection in sensor networks

Sensor networks are emerging as the new frontier in sensing technology, however there are still issues that need to be addressed. Two such issues are data collection and energy conservation. We consider a mobile robot, or a mobile agent, traveling the network collecting information from the sensors themselves before their onboard memory storage buffers are full. A novel algorithm is presented that is an adaptation of a local search algorithm for a special case of the Asymmetric Traveling Salesman Problem with Time-windows (ATSPTW) for solving the dynamic scheduling problem of what nodes are to be visited so that the information collected is not lost. Our algorithms are given and compared to other work.

Data collection for global scheduling in the GENESIS system

In order to schedule parallel processes on available computers there is a need to collect information about different cluster resources. Currently, global scheduling of parallel processes of applications takes into consideration mainly processor load and only in a small number of projects memory utilization. Communication costs, which are high in clusters, are practically neglected. The aim of this paper is to report on our study into cluster parameter measurements, in particular those, which characterize communication costs, the development of the resource discovery server and its initial testing through the study of the influence of communication costs on parallel application performance.