Validating the is-impact model in the Malaysian public sector

This research is one of several ongoing studies conducted within the IT Professional Services (ITPS) research programme at Queensland University of Technology (QUT). In 2003, ITPS introduced the IS-Impact model, a measurement model for measuring information systems success from the viewpoint of multiple stakeholders. The model, along with its instrument, is robust, simple, yet generalisable, and yields results that are comparable across time, stakeholders, different systems and system contexts. The IS-Impact model is defined as “a measure at a point in time, of the stream of net benefits from the Information System (IS), to date and anticipated, as perceived by all key-user-groups”. The model represents four dimensions, which are ‘Individual Impact’, ‘Organizational Impact’, ‘Information Quality’ and ‘System Quality’. The two Impact dimensions measure the up-to-date impact of the evaluated system, while the remaining two Quality dimensions act as proxies for probable future impacts (Gable, Sedera & Chan, 2008). To fulfil the goal of ITPS, “to develop the most widely employed model” this research re-validates and extends the IS-Impact model in a new context. This method/context-extension research aims to test the generalisability of the model by addressing known limitations of the model. One of the limitations of the model relates to the extent of external validity of the model. In order to gain wide acceptance, a model should be consistent and work well in different contexts. The IS-Impact model, however, was only validated in the Australian context, and packaged software was chosen as the IS understudy. Thus, this study is concerned with whether the model can be applied in another different context. Aiming for a robust and standardised measurement model that can be used across different contexts, this research re-validates and extends the IS-Impact model and its instrument to public sector organisations in Malaysia. The overarching research question (managerial question) of this research is “How can public sector organisations in Malaysia measure the impact of information systems systematically and effectively?” With two main objectives, the managerial question is broken down into two specific research questions. The first research question addresses the applicability (relevance) of the dimensions and measures of the IS-Impact model in the Malaysian context. Moreover, this research question addresses the completeness of the model in the new context. Initially, this research assumes that the dimensions and measures of the IS-Impact model are sufficient for the new context. However, some IS researchers suggest that the selection of measures needs to be done purposely for different contextual settings (DeLone & McLean, 1992, Rai, Lang & Welker, 2002). Thus, the first research question is as follows, “Is the IS-Impact model complete for measuring the impact of IS in Malaysian public sector organisations?” [RQ1]. The IS-Impact model is a multidimensional model that consists of four dimensions or constructs. Each dimension is represented by formative measures or indicators. Formative measures are known as composite variables because these measures make up or form the construct, or, in this case, the dimension in the IS-Impact model. These formative measures define different aspects of the dimension, thus, a measurement model of this kind needs to be tested not just on the structural relationship between the constructs but also the validity of each measure. In a previous study, the IS-Impact model was validated using formative validation techniques, as proposed in the literature (i.e., Diamantopoulos and Winklhofer, 2001, Diamantopoulos and Siguaw, 2006, Petter, Straub and Rai, 2007). However, there is potential for improving the validation testing of the model by adding more criterion or dependent variables. This includes identifying a consequence of the IS-Impact construct for the purpose of validation. Moreover, a different approach is employed in this research, whereby the validity of the model is tested using the Partial Least Squares (PLS) method, a component-based structural equation modelling (SEM) technique. Thus, the second research question addresses the construct validation of the IS-Impact model; “Is the IS-Impact model valid as a multidimensional formative construct?” [RQ2]. This study employs two rounds of surveys, each having a different and specific aim. The first is qualitative and exploratory, aiming to investigate the applicability and sufficiency of the IS-Impact dimensions and measures in the new context. This survey was conducted in a state government in Malaysia. A total of 77 valid responses were received, yielding 278 impact statements. The results from the qualitative analysis demonstrate the applicability of most of the IS-Impact measures. The analysis also shows a significant new measure having emerged from the context. This new measure was added as one of the System Quality measures. The second survey is a quantitative survey that aims to operationalise the measures identified from the qualitative analysis and rigorously validate the model. This survey was conducted in four state governments (including the state government that was involved in the first survey). A total of 254 valid responses were used in the data analysis. Data was analysed using structural equation modelling techniques, following the guidelines for formative construct validation, to test the validity and reliability of the constructs in the model. This study is the first research that extends the complete IS-Impact model in a new context that is different in terms of nationality, language and the type of information system (IS). The main contribution of this research is to present a comprehensive, up-to-date IS-Impact model, which has been validated in the new context. The study has accomplished its purpose of testing the generalisability of the IS-Impact model and continuing the IS evaluation research by extending it in the Malaysian context. A further contribution is a validated Malaysian language IS-Impact measurement instrument. It is hoped that the validated Malaysian IS-Impact instrument will encourage related IS research in Malaysia, and that the demonstrated model validity and generalisability will encourage a cumulative tradition of research previously not possible. The study entailed several methodological improvements on prior work, including: (1) new criterion measures for the overall IS-Impact construct employed in ‘identification through measurement relations’; (2) a stronger, multi-item ‘Satisfaction’ construct, employed in ‘identification through structural relations’; (3) an alternative version of the main survey instrument in which items are randomized (rather than blocked) for comparison with the main survey data, in attention to possible common method variance (no significant differences between these two survey instruments were observed); (4) demonstrates a validation process of formative indexes of a multidimensional, second-order construct (existing examples mostly involved unidimensional constructs); (5) testing the presence of suppressor effects that influence the significance of some measures and dimensions in the model; and (6) demonstrates the effect of an imbalanced number of measures within a construct to the contribution power of each dimension in a multidimensional model.

Evaluating operational cloud enterprise systems success : an organizational perspective

This paper presents a formative measurement index to assess cloud enterprise systems success. The scale development procedure is based on Moore and Benbasat (1991), including newer scale development elements which focus on the creation and assessment of formative constructs. The data is analysed using SmartPLS with a sample of 103 IT decision makers. The results show that the perception of net benefits is shaped not only by enterprise-system-specific factors like productivity improvements and higher quality of business processes, but also by factors which are specifically attributed to cloud systems, such as higher strategic flexibility. Reliability, user requirements and customization contribute most to the overall perception of system quality. Information quality shows no cloud-specific facets and is robust in the context of cloud enterprise systems.

Using performance measurement models for benefit realization with enterprise systems: The Queensland Government approach (Case Study)

A range of influences, technical and organizational, has encouraged the wide spread adaption of Enterprise Systems (ES). Nevertheless, there is a growing consensus that Enterprise Systems have in the many cases failed to provide the expected benefits to organizations. This paper presents ongoing research, which analyzes the benefits realization approach of the Queensland Government. This approach applies a modified Balance Scorecard. First, history and background of Queensland Government’s Enterprise Systems initiative is introduced. Second, the most common reasons for ES under performance are related. Third, relevant performance measurement models and the Balanced Scorecard in particular are discussed. Finally, the Queensland Government initiative is evaluated in light of this overview of current work in the area. In the current and future work, the authors aim to use their active involvement in Queensland Government’s benefits realization initiative for an Action Research based project investigating the appropriateness of the Balanced Scorecard for the purposes of Enterprise Systems benefits realization.

Critical success factors of process modeling for enterprise systems

The integrated and process oriented nature of Enterprise Systems (ES) has led organizations to use process modeling as an aid in managing these systems. Enterprise Systems success factor studies explicitly and implicitly state the importance of process modeling and its contribution to overall Enterprise System success. However, no empirical evidence exists on how to conduct process modeling successfully and possibly differentially in the main phases of the ES life-cycle. This paper reports on an empirical investigation of the factors that influence process modeling success. An a-priori model with 8 candidate success factors has been developed to this stage. This paper introduces the research context and objectives, describes the research design and the derived model, and concludes by looking ahead to the next phases of the research design.

Enterprise systems in universities : a teaching case

This teaching case aims to contribute to understanding the phenomenon of Enterprise Systems (ES) implementations in universities. Through this case, students will gain understanding of the importance of ‘contextual elements’ for large scale information systems (IS) implementations, in particular ES. This teaching case illustrates how these contextual factors contribute to the success or failure of such implementations, and how they can influence the decisions that dictate the lifecycle of such systems. The case describes ES implementations at a leading Australian university, and presents a rich account of the institutional, national and industry-sector contexts that have influenced the directions and decisions taken. The journey encountered with the main Enterprise Systems that support Financials, Human Resources and Facilities are described suggesting the lifecycle phases, critical success factors and lessons learnt.

Developing an IS-Impact decision tool : a literature based design science roadmap

This paper derives from research-in-progress intending both Design Research (DR) and Design Science (DS) outputs; the former a management decision tool based in IS-Impact (Gable et al. 2008) kernel theory; the latter being methodological learnings deriving from synthesis of the literature and reflection on the DR ‘case study’ experience. The paper introduces a generic, detailed and pragmatic DS ‘Research Roadmap’ or methodology, deriving at this stage primarily from synthesis and harmonization of relevant concepts identified through systematic archival analysis of related literature. The scope of the Roadmap too has been influenced by the parallel study aim to undertake DR applying and further evolving the Roadmap. The Roadmap is presented in attention to the dearth of detailed guidance available to novice Researchers in Design Science Research (DSR), and though preliminary, is expected to evolve and gradually be substantiated through experience of its application. A key distinction of the Roadmap from other DSR methods is its breadth of coverage of published DSR concepts and activities; its detail and scope. It represents a useful synthesis and integration of otherwise highly disparate DSR-related concepts.

Introducing a small and medium enterprises syllabi design: an enterprise systems hands-on approach

There is increasing attention to the importance of Enterprise Systems (ES) and Information Systems (IS) for Small and Medium Enterprises (SMEs). The same attention must be addressed in IS graduate curriculum. Studies reveal that despite healthy demand from the industry for IS management expertise, most IS graduates are ill-equipped to meet the challenges of modern organizations. The majority of contemporary firms, represented by SMEs, seek employees with a balance of business process knowledge and ES software skills. This article describes a curriculum that teaches Information Technology (IT) and IS managementconcepts in a SMEs context. The curriculum conceptualises a ‘learn-by-doing’ approach, to provide business process and ES software specific knowledge for its students. The approach recommends coverage of traditional content related to SMEs’’ operations, strategies, IT investment and management issues while providing an increased focus on strategic use of enterprise IT. The study addresses to an extent, the perennial challenge of updating IS curriculum, given the rapid pace of technological change.

Ontology-based knowledge management for enterprise systems

Companies face the challenges of expanding their markets, improving products, services and processes, and exploiting intellectual capital in a dynamic network. Therefore, more companies are turning to an Enterprise System (ES). Knowledge management (KM) has also received considerable attention and is continuously gaining the interest of industry, enterprises, and academia. For ES, KM can provide support across the entire lifecycle, from selection and implementation to use. In addition, it is also recognised that an ontology is an appropriate methodology to accomplish a common consensus of communication, as well as to support a diversity of KM activities, such as knowledge repository, retrieval, sharing, and dissemination. This paper examines the role of ontology-based KM for ES (OKES) and investigates the possible integration of ontology-based KM and ES. The authors develop a taxonomy as a framework for understanding OKES research. In order to achieve the objective of this study, a systematic review of existing research was conducted. Based on a theoretical framework of the ES lifecycle, KM, KM for ES, ontology, and ontology-based KM, guided by the framework of study, a taxonomy for OKES is established.

Enterprise systems curricula for Asia-Pacific region : a vision for 2020

Enterprise Systems (ES) have emerged as possibly the most important and challenging development in the corporate use of information technology in the last decade. Organizations have invested heavily in these large, integrated application software suites expecting improvments in; business processes, management of expenditure, customer service, and more generally, competitiveness, improved access to better information/knowledge (i.e., business intelligence and analytics). Forrester survey data consistently shows that investment in ES and enterprise applications in general remains the top IT spending priority, with the ES market estimated at $38 billion and predicted to grow at a steady rate of 6.9%, reaching $50 billion by 2012 (Wang & Hamerman, 2008). Yet, organizations have failed to realize all the anticipated benefits. One of the key reasons is the inability of employees to properly utilize the capabilities of the enterprise systems to complete the work and extract information critical to decision making. In response, universities (tertiary institutes) have developed academic programs aimed at addressing the skill gaps. In parallel with the proliferation of ES, there has been growing recognition of the importance of Teaching Enterprise Systems at tertiary education institutes. Many academic papers have discused the important role of Enterprise System curricula at tertiary education institutes (Ask, 2008; Hawking, 2004; Stewart, 2001), where the teaching philosophises, teaching approaches and challenges in Enterprise Systems education were discussed. Following the global trends, tertiary institutes in the Pacific-Asian region commenced introducing Enterprise System curricula in late 1990s with a range of subjects (a subject represents a single unit, rather than a collection of units; which we refer to as a course) in faculties / schools / departments of Information Technology, Business and in some cases in Engineering. Many tertiary educations commenced their initial subject offers around four salient concepts of Enterprise Systems: (1) Enterprise Systems implementations, (2) Introductions to core modules of Enterprise Systems, (3) Application customization using a programming language (e.g. ABAP) and (4) Systems Administration. While universities have come a long way in developing curricula in the enterprise system area, many obstacles remain: high cost of technology, qualified faculty to teach, lack of teaching materials, etc.

Integrated configuration of enterprise systems for interoperability : towards process model and business document specification alignment

Enterprise Systems (ES) can be understood as the de facto standard for holistic operational and managerial support within an organization. Most commonly ES are offered as commercial off-the-shelf packages, requiring customization in the user organization. This process is a complex and resource-intensive task, which often prevents small and midsize enterprises (SME) from undertaking configuration projects. Especially in the SME market independent software vendors provide pre-configured ES for a small customer base. The problem of ES configuration is shifted from the customer to the vendor, but remains critical. We argue that the yet unexplored link between process configuration and business document configuration must be closer examined as both types of configuration are closely tied to one another.

Exploring subscription renewal intention of operational cloud enterprise systems - a stakeholder perspective

Retaining customers is a relevant topic throughout all service industries. However, only limited attention has been directed towards studying the antecedents of subscription renewal in the context of operational cloud enterprise systems. Cloud services have historically been offered as subscription-based services with the (theoretical) possibility of seamless service cancellation, in contrast to classical IT-Outsourcing contracts or license-based software installations of on-premise enterprise systems. In this work, we investigate the central concept of subscription renewal by focusing on different facets of IS success and their relevance for distinct employee cohorts. Analyzing inter-cohort differences has strong practical implications, as it helps IT vendors to focus on specific IT-related factors when trying to retain customers. Therefore an empirical study was undertaken. The hypotheses were developed on an individual level and tested using survey responses of IT decision makers within companies which adopted cloud enterprise systems. Gathered data was then analyzed using PLS. The results show that subscription renewal intention of the strategic cohort is mainly based on perceived system quality, whereas information quality explains most of the variance of subscription renewal in the management cohort. Beneath the cloud enterprise systems specific contributions, the work adds to the theoretical body of research related to IS success and IS continuation, as well as stakeholder perspectives.

Blast response and safety evaluation of a composite column for use as key element in structural systems

This paper presents the blast response, damage mechanism and evaluation of residual load capacity of a concrete–steel composite (CSC) column using dynamic computer simulation techniques. This study is an integral part of a comprehensive research program which investigated the vulnerability of structural framing systems to catastrophic and progressive collapse under blast loading and is intended to provide design information on blast mitigation and safety evaluation of load bearing vulnerable columns that are key elements in a building. The performance of the CSC column is compared with that of a reinforced concrete (RC) column with the same dimensions and steel ratio. Results demonstrate the superior performance of the CSC column, compared to the RC column in terms of residual load carrying capacity, and its potential for use as a key element in structural systems. The procedure and results presented herein can be used in the design and safety evaluation of key elements of multi-storey buildings for mitigating the impact of blast loads.

Information systems success research : the “20-year update?” panel report from PACIS, 2011

This article summarizes a panel held at the 15th Pacific Asia Conference on Information Systems (PACIS) in Brisbane, Austrailia, in 2011. The panelists proposed a new research agenda for information systems success research. The DeLone and McLean IS Success Model has been one of the most influential models in Information Systems research. However, the nature of information systems continues to change. Information systems are increasingly implemented across layers of infrastructure and application architecture. The diffusion of information systems into many spheres of life means that information systems success needs to be considered in multiple contexts. Services play a much more prominent role in the economies of countries, making the “service” context of information systems increasingly important. Further, improved understandings of theory and measurement offer new opportunities for novel approaches and new research questions about information systems success.

Exploring subscription renewal intention of operational cloud enterprise systems-a socio-technical approach

Despite the fact that customer retention is crucial for providers of cloud enterprise systems, only little attention has been directed towards investigating the antecedents of subscription renewal in an organizational context. This is even more surprising, as cloud services are usually offered as subscription-based pricing models with the (theoretical) possibility of immediate service cancellation, strongly opposing classical long-term IT-Outsourcing contracts or license-based payment plans of on premise enterprise systems. To close this research gap an empirical study was undertaken. Firstly, a conceptual model was drawn from theories of social psychology, organizational system continuance and IS success. The model was subsequently tested using survey responses of senior management within companies which adopted cloud enterprise systems. Gathered data was then analysed using PLS. The results indicate that subscription renewal intention is influenced by both – social-related and technology-specific factors – which are able to explain 50.4% of the variance in the dependent variable. Beneath the cloud enterprise systems specific contributions, the work advances knowledge in the area of organizational system continuance, as well as IS success.

Influence of organic amendments on greenhouse gas emissions and N use efficiency in sub-tropical cropping systems

This thesis investigated the impact of organic sources of nutrients on greenhouse gas emissions (carbon dioxide, nitrous oxide and methane), nitrogen use efficiency and biomass production in subtropical cropping soils. The study was conducted in two main soil types in subtropical ecosystems, sandy loam soil and clay soil, with a variety of organic materials from agro-industrial residues and crop residues. It is important for recycling of agro-industrial residues and agricultural residues and the mitigation of greenhouse gas emissions and nitrogen use efficiency.