Survey design: Insights from a public sector ERP success story

Research on the impact of Information Systems (IS) reported in both academic literature and popular press has reported confounding results. Some studies have reported encouraging results of IS, while others have reported nil or detrimental results. The contradictory results of these research studies can be partially attributed to the weaknesses in survey instruments. In an attempt to increase the validity of conclusions of IS assessment studies, survey instrument design should follow a rigorous and scientific procedure. This paper illustrates key validity and reliability issues in measuring Information Systems performance, using examples from a study designed to assess Enterprise Resource Planning systems success. The article emphasizes on the importance of the survey method and the theoretical considerations of item derivation, scale development and item evaluation. Examples are provided from the ERP assessment study to supplement the readers understanding of the theoretical concepts of survey design.