Observing the relevance of an information systems success model

This paper contributes to the rigor vs. relevance debate in the Information Systems (IS) discipline. Using the Action Research methodology, this study evaluates the relevance of a rigorously validated IS evaluation model in practice. The study captures observations of operational end-users employing a market leading Enterprise System application for procurement and order fulfillment in an organization. The analysis of the observations demonstrates the broad relevance of the measurement instrument. More importantly, the study identifies several improvements and possible confusions in applying the instrument in the practice.

Linking Theory and Practice: Performing a Reality Check on a Model of IS Success

The issue of ‘rigour vs. relevance’ in IS research has generated an intense, heated debate for over a decade. It is possible to identify, however, only a limited number of contributions on how to increase the relevance of IS research without compromising its rigour. Based on a lifecycle view of IS research, we propose the notion of ‘reality checks’ in order to review IS research outcomes in the light of actual industry demands. We assume that five barriers impact the efficient transfer of IS research outcomes; they are lack of awareness, lack of understandability, lack of relevance, lack of timeliness, and lack of applicability. In seeking to understand the effect of these barriers on the transfer of mature IS research into practice, we used focus groups. We chose DeLone and McLean’s IS success model as our stimulus because it is one of the more widely researched areas of IS.

Understanding the phenomenon of IS success in China through the lens of the IS-Impact model

This study is conducted within the IS-Impact Research Track at Queensland University of Technology (QUT). The goal of the IS-Impact Track is, “to develop the most widely employed model for benchmarking information systems in organizations for the joint benefit of both research and practice” (Gable et al, 2006). IS-Impact is defined as “a measure at a point in time, of the stream of net benefits from the IS, to date and anticipated, as perceived by all key-user-groups” (Gable Sedera and Chan, 2008). Track efforts have yielded the bicameral IS-Impact measurement model; the “impact” half includes Organizational-Impact and Individual-Impact dimensions; the “quality” half includes System-Quality and Information-Quality dimensions. The IS-Impact model, by design, is intended to be robust, simple and generalizable, to yield results that are comparable across time, stakeholders, different systems and system contexts. The model and measurement approach employ perceptual measures and an instrument that is relevant to key stakeholder groups, thereby enabling the combination or comparison of stakeholder perspectives. Such a validated and widely accepted IS-Impact measurement model has both academic and practical value. It facilitates systematic operationalization of a main dependent variable in research (IS-Impact), which can also serve as an important independent variable. For IS management practice it provides a means to benchmark and track the performance of information systems in use. The objective of this study is to develop a Mandarin version IS-Impact model, encompassing a list of China-specific IS-Impact measures, aiding in a better understanding of the IS-Impact phenomenon in a Chinese organizational context. The IS-Impact model provides a much needed theoretical guidance for this investigation of ES and ES impacts in a Chinese context. The appropriateness and soundness of employing the IS-Impact model as a theoretical foundation are evident: the model originated from a sound theory of IS Success (1992), developed through rigorous validation, and also derived in the context of Enterprise Systems. Based on the IS-Impact model, this study investigates a number of research questions (RQs). Firstly, the research investigated what essential impacts have been derived from ES by Chinese users and organizations [RQ1]. Secondly, we investigate which salient quality features of ES are perceived by Chinese users [RQ2]. Thirdly, we seek to answer whether the quality and impacts measures are sufficient to assess ES-success in general [RQ3]. Lastly, the study attempts to address whether the IS-Impact measurement model is appropriate for Chinese organizations in terms of evaluating their ES [RQ4]. An open-ended, qualitative identification survey was employed in the study. A large body of short text data was gathered from 144 Chinese users and 633 valid IS-Impact statements were generated from the data set. A generally inductive approach was applied in the qualitative data analysis. Rigorous qualitative data coding resulted in 50 first-order categories with 6 second-order categories that were grounded from the context of Chinese organization. The six second-order categories are: 1) System Quality; 2) Information Quality; 3) Individual Impacts;4) Organizational Impacts; 5) User Quality and 6) IS Support Quality. The final research finding of the study is the contextualized Mandarin version IS-Impact measurement model that includes 38 measures organized into 4 dimensions: System Quality, information Quality, Individual Impacts and Organizational Impacts. The study also proposed two conceptual models to harmonize the IS-Impact model and the two emergent constructs – User Quality and IS Support Quality by drawing on previous IS effectiveness literatures and the Work System theory proposed by Alter (1999) respectively. The study is significant as it is the first effort that empirically and comprehensively investigates IS-Impact in China. Specifically, the research contributions can be classified into theoretical contributions and practical contributions. From the theoretical perspective, through qualitative evidence, the study test and consolidate IS-Impact measurement model in terms of the quality of robustness, completeness and generalizability. The unconventional research design exhibits creativity of the study. The theoretical model does not work as a top-down a priori seeking for evidence demonstrating its credibility; rather, the study allows a competitive model to emerge from the bottom-up and open-coding analysis. Besides, the study is an example extending and localizing pre-existing theory developed in Western context when the theory is introduced to a different context. On the other hand, from the practical perspective, It is first time to introduce prominent research findings in field of IS Success to Chinese academia and practitioner. This study provides a guideline for Chinese organizations to assess their Enterprise System, and leveraging IT investment in the future. As a research effort in ITPS track, this study contributes the research team with an alternative operationalization of the dependent variable. The future research can take on the contextualized Mandarin version IS-Impact framework as a theoretical a priori model, further quantitative and empirical testing its validity.

Validating the IS-Impact model : two exploratory case studies in China and Malaysia

For over a decade, IT expenditure in China and Malaysia has shown a significant increase, as organisations in these countries are increasingly dependent on information systems (IS) for achieving strategic advantages and business benefits. However, there have been numerous reports of dissatisfaction with IS, and in some cases the effectiveness of the information systems have yet to be reviewed. Two exploratory case studies reported in this paper are the first phase of an overall research in validating the IS-Impact model introduced by Gable, Sedera and Chan in two countries: China and Malaysia. This validation research aims to produce a standard measuring model across different contexts. The purpose of this paper is to present preliminary findings from two exploratory case studies, attempt to test the feasibility of the research design and to investigate applicability of the IS-Impact model in Chinese and Malaysian organisations. Twenty-nine respondents from a Chinese private company and seventeen respondents from a state government in Malaysia were involved in these studies. Findings indicated that most of existing IS-Impact measures are applicable in the study contexts, however, there are some new measures informed by the respondents. Feedback from the case studies also suggested necessary modifications to the Mandarin instrument.

A comprehensive EA benefit realization model -- An exploratory study

Enterprise architecture (EA) management has be-come an intensively discussed approach to manage enterprise transformations. While there is a strong interest in EA frameworks and EA modeling, a lack of knowledge remains about the theoretical foundation of EA benefits. In this paper, we identify EA success factors and EA benefits through a literature review, and integrate these findings with the DeLone & McLean IS success model to propose a theoretical model explaining the realization of EA benefits. In addition, we con-ducted semi-structured interviews with EA experts for a preliminary validation and further exploration of the model. We see this model as a first step to gain insights in and start a discussion on the theory of EA benefit realization. In future research, we plan to empirically validate the proposed model.

Realizing benefits from enterprise architecture : a measurement model

Enterprise architecture (EA) management has become an intensively discussed approach to manage enterprise transformations. Despite the popularity and potential of EA, both researchers and practitioners lament a lack of knowledge about the realization of benefits from EA. To determine the benefits from EA, we explore the various dimensions of EA benefit realization and report on the development of a validated and robust measurement instrument. In this paper, we test the reliability and construct validity of the EA benefit realization model (EABRM), which we have designed based on the DeLone & McLean IS success model and findings from exploratory interviews. A confirmatory factor analysis confirms the existence of an impact of five distinct and individually important dimensions on the benefits derived from EA: EA artefact quality, EA infrastructure quality, EA service quality, EA culture, and EA use. The analysis presented in this paper shows that the EA benefit realization model is an instrument that demonstrates strong reliability and validity.

Success factors for ICT shared services in the higher education sector

Shared services is a prominent organizational arrangement for organizations, in particular for support functions. The success (or failure) of shared services is a critical concern as the move to shared services can entail large scale investment and involve fundamental organizational change. The Higher Education (HE) sector is particularly well poised to benefit from shared services as there is a need to improve organizational performance and strong potential from sharing. Through a multiple case study of shared services experiences in HE, this study identifies ten important antecedents of shared services success: (1) Understanding of shared services; (2) Organizational environment; (3) Top management support; (4) IT environment; (5) Governance; (6) Process centric view; (7) Implementation strategy; (8) Project management; (9) Change management; and (10) Communication. The study then develops a preliminary model of shared services success that addresses the interdependencies between the success factors. As the first empirical success model for shared services, it provides valuable guidance to practice and future research.

Critical success elements for the design and implementation of organisational e-learning

Organisations are engaging in e-learning as a mechanism for delivering flexible learning to meet the needs of individuals and organisations. In light of the increasing use and organisational investment in e-learning, the need for methods to evaluate the success of its design and implementation seems more important than ever. To date, developing a standard for the evaluation of e-learning appears to have eluded both academics and practitioners. The currently accepted evaluation methods for e-learning are traditional learning and development models, such as Kirkpatrick’s model (1976). Due to the technical nature of e-learning it is important to broaden the scope and consider other evaluation models or techniques, such as the DeLone and McLean Information Success Model, that may be applicable to the e-learning domain. Research into the use of e-learning courses has largely avoided considering the applicability of information systems research. Given this observation, it is reasonable to conclude that e-learning implementation decisions and practice could be overlooking useful or additional viewpoints. This research investigated how existing evaluation models apply in the context of organisational e-learning, and resulted in an Organisational E-learning success Framework, which identifies the critical elements for success in an e-learning environment. In particular this thesis highlights the critical importance of three e-learning system creation elements; system quality, information quality, and support quality. These elements were explored in depth and the nature of each element is described in detail. In addition, two further elements were identified as factors integral to the success of an e-learning system; learner preferences and change management. Overall, this research has demonstrated the need for a holistic approach to e-learning evaluation. Furthermore, it has shown that the application of both traditional training evaluation approaches and the D&M IS Success Model are appropriate to the organisational e-learning context, and when combined can provide this holistic approach. Practically, this thesis has reported the need for organisations to consider evaluation at all stages of e-learning from design through to implementation.

Information systems success research : the “20-year update?” panel report from PACIS, 2011

This article summarizes a panel held at the 15th Pacific Asia Conference on Information Systems (PACIS) in Brisbane, Austrailia, in 2011. The panelists proposed a new research agenda for information systems success research. The DeLone and McLean IS Success Model has been one of the most influential models in Information Systems research. However, the nature of information systems continues to change. Information systems are increasingly implemented across layers of infrastructure and application architecture. The diffusion of information systems into many spheres of life means that information systems success needs to be considered in multiple contexts. Services play a much more prominent role in the economies of countries, making the “service” context of information systems increasingly important. Further, improved understandings of theory and measurement offer new opportunities for novel approaches and new research questions about information systems success.

Critical success factors in e‐learning ecosystems: a qualitative study

Abstract
Purpose – The purpose of this paper is to evaluate the critical success factors for sustainable e-learning in an e-learning ecosystem framework. Three critical components of the e-learning ecosystem including principles and methods, processes and systems, and substance and content are considered based on a comprehensive review of the relevant literature in e-learning.
Design/methodology/approach – Systematic interviews are conducted with experts in e-learning for identifying the critical success factors to sustainable e-learning within an e-learning ecosystem framework. This leads to the development of an e-learning success model that describes the underlying relationship between and among the identified critical success factors.
Findings – A comprehensive analysis of the interview results shows that there are several barriers to the effective adoption of the proposed e-learning success model for improving the effectiveness of e- learning. These barriers include a lack of understanding of the technologies behind various pedagogies, insufficiencies of the popular learning management systems, and the sustainability of the learning objects repositories.
Research limitations/implications – The paper highlights the criticality of synergizing the three components of e-learning ecosystems namely pedagogies, technologies and management of learning resources for achieving a sustainable e-learning success.
Practical implications – A better understanding of these barriers would help e-learning stakeholders develop appropriate strategies and policies for the implementation of the proposed e-learning success model towards creating a sustainable e-learning environment.
Originality/value – Specific contributions of this research to the entire e-learning community are discussed with recommendations for concerted policy measures to eliminate the identified barriers in the process of adopting the developed e-learning success model.

Measuring the realization of benefits from enterprise architecture management

Enterprise architecture management (EAM) has become an intensively discussed approach to manage enterprise transformations. While many organizations employ EAM, a notable insecurity about the value of EAM remains. In this paper, we propose a model to measure the realization of benefits from EAM. We identify EAM success factors and EAM benefits through a comprehensive literature review and eleven explorative expert interviews. Based on our findings, we integrate the EAM success factors and benefits with the established DeLone & McLean IS success model resulting in a model that explains the realization of EAM benefits. This model aids organizations as a benchmark and framework for identifying and assessing the setup of their EAM initiatives and whether and how EAM benefits are materialized. We see our model also as a first step to gain insights in and start a discussion on the theory of EAM benefit realization.

Leveraging of software as a service by small and medium enterprises: Information systems capabilities and organisational complementarities

This research explored how small and medium enterprises can achieve success with software as a service (SaaS) applications from cloud. Based upon an empirical investigation of six growth oriented and early technology adopting small and medium enterprises, this study proposes a SaaS for small and medium enterprise success model with two approaches: one for basic and one for advanced benefits. The basic model explains the effective use of SaaS for achieving informational and transactional benefits. The advanced model explains the enhanced use of software as a service for achieving strategic and transformational benefits. Both models explicate the information systems capabilities and organizational complementarities needed for achieving success with SaaS.

Assessing Cybersecurity service quality in corporate environments

This study assess the quality of Cybersecurity as a service provided by IT department in corporate network and provides analysis about the service quality impact on the user, seen as a consumer of the service, and on the organization as well. In order to evaluate the quality of this service, multi-item instrument “SERVQUAL” was used for measuring consumer perceptions of service quality. To provide insights about Cybersecurity service quality impact, DeLone and McLean information systems success model was used. To test this approach, data was collected from over one hundred users from different industries and partial least square (PLS) was used to estimate the research model. This study found that SERVQUAL is adequate to assess Cybersecurity service quality and also found that Cybersecurity service quality positively influences the Cybersecurity use and individual impact in Cybersecurity.