Evaluating information systems

The book within which this chapter appears is published as a research reference book (not a coursework textbook) on Management Information Systems (MIS) for seniors or graduate students in Chinese universities. It is hoped that this chapter, along with the others, will be helpful to MIS scholars and PhD/Masters research students in China who seek understanding of several central Information Systems (IS) research topics and related issues. The subject of this chapter - ‘Evaluating Information Systems’ - is broad, and cannot be addressed in its entirety in any depth within a single book chapter. The chapter proceeds from the truism that organizations have limited resources and those resources need to be invested in a way that provides greatest benefit to the organization. IT expenditure represents a substantial portion of any organization’s investment budget and IT related innovations have broad organizational impacts. Evaluation of the impact of this major investment is essential to justify this expenditure both pre- and post-investment. Evaluation is also important to prioritize possible improvements. The chapter (and most of the literature reviewed herein) admittedly assumes a blackbox view of IS/IT1, emphasizing measures of its consequences (e.g. for organizational performance or the economy) or perceptions of its quality from a user perspective. This reflects the MIS emphasis – a ‘management’ emphasis rather than a software engineering emphasis2, where a software engineering emphasis might be on the technical characteristics and technical performance. Though a black-box approach limits diagnostic specificity of findings from a technical perspective, it offers many benefits. In addition to superior management information, these benefits may include economy of measurement and comparability of findings (e.g. see Part 4 on Benchmarking IS). The chapter does not purport to be a comprehensive treatment of the relevant literature. It does, however, reflect many of the more influential works, and a representative range of important writings in the area. The author has been somewhat opportunistic in Part 2, employing a single journal – The Journal of Strategic Information Systems – to derive a classification of literature in the broader domain. Nonetheless, the arguments for this approach are believed to be sound, and the value from this exercise real. The chapter drills down from the general to the specific. It commences with a highlevel overview of the general topic area. This is achieved in 2 parts: - Part 1 addressing existing research in the more comprehensive IS research outlets (e.g. MISQ, JAIS, ISR, JMIS, ICIS), and Part 2 addressing existing research in a key specialist outlet (i.e. Journal of Strategic Information Systems). Subsequently, in Part 3, the chapter narrows to focus on the sub-topic ‘Information Systems Success Measurement’; then drilling deeper to become even more focused in Part 4 on ‘Benchmarking Information Systems’. In other words, the chapter drills down from Parts 1&2 Value of IS, to Part 3 Measuring Information Systems Success, to Part 4 Benchmarking IS. While the commencing Parts (1&2) are by definition broadly relevant to the chapter topic, the subsequent, more focused Parts (3 and 4) admittedly reflect the author’s more specific interests. Thus, the three chapter foci – value of IS, measuring IS success, and benchmarking IS - are not mutually exclusive, but, rather, each subsequent focus is in most respects a sub-set of the former. Parts 1&2, ‘the Value of IS’, take a broad view, with much emphasis on ‘the business Value of IS’, or the relationship between information technology and organizational performance. Part 3, ‘Information System Success Measurement’, focuses more specifically on measures and constructs employed in empirical research into the drivers of IS success (ISS). (DeLone and McLean 1992) inventoried and rationalized disparate prior measures of ISS into 6 constructs – System Quality, Information Quality, Individual Impact, Organizational Impact, Satisfaction and Use (later suggesting a 7th construct – Service Quality (DeLone and McLean 2003)). These 6 constructs have been used extensively, individually or in some combination, as the dependent variable in research seeking to better understand the important antecedents or drivers of IS Success. Part 3 reviews this body of work. Part 4, ‘Benchmarking Information Systems’, drills deeper again, focusing more specifically on a measure of the IS that can be used as a ‘benchmark’3. This section consolidates and extends the work of the author and his colleagues4 to derive a robust, validated IS-Impact measurement model for benchmarking contemporary Information Systems (IS). Though IS-Impact, like ISS, has potential value in empirical, causal research, its design and validation has emphasized its role and value as a comparator; a measure that is simple, robust and generalizable and which yields results that are as far as possible comparable across time, across stakeholders, and across differing systems and systems contexts.

Measuring the impact of information systems in Malaysia

Information Systems researchers have employed a diversity of sometimes inconsistent measures of IS success, seldom explicating the rationale, thereby complicating the choice for future researchers. In response to these and other issues, Gable, Sedera and Chan introduced the IS-Impact measurement model. This model represents “the stream of net benefits from the Information System (IS), to date and anticipated, as perceived by all key-user-groups”. Although the IS-Impact model was rigorously validated in previous research, there is a need to further generalise and validate it in different context. This paper reported the findings of the IS-Impact model revalidation study at four state governments in Malaysia with 232 users of a financial system that is currently being used at eleven state governments in Malaysia. Data was analysed following the guidelines for formative measurement validation using SmartPLS. Based on the PLS results, data supported the IS-Impact dimensions and measures thus confirming the validity of the IS-Impact model in Malaysia. This indicates that the IS-Impact model is robust and can be used across different context.

Maturity models in the hospital information systems management initial research

Os modelos de maturidade são instrumentos facilitadores da gestão das organizações, incluindo a gestão da sua função sistemas de informação, não sendo exceção as organizações hospitalares. Neste artigo apresenta-se uma investigação inicial que visa o desenvolvimento de um abrangente modelo de maturidade para a gestão dos sistemas de informação hospitalares. O desenvolvimento deste modelo justifica-se porque os modelos de maturidade atuais no domínio da gestão dos sistemas informação hospitalares ainda se encontram numa fase embrionária de desenvolvimento, sobretudo porque são pouco detalhados, não disponibilizam ferramentas para determinação da maturidade e não apresentam as características dos estágios de maturidade estruturadas por diferentes fatores de influência.

Benchmarking for information systems management using issues framework studies:cContent and methodology

As a means of benchmarking their position and assisting with anticipating an uncertain future, the identification of critical information systems (IS) management issues frameworks is becoming an increasingly important research task for both academics and industrialists. This paper provides a description and summary of previous work on identifying IS issues frameworks by reviewing 20 research investigations in terms of what they studied and how they were conducted. It also suggests some possible directions and methodologies for future research. The summary and suggestions for further work are applicable for issues framework research in the IS management field as well as in other business and management areas.

Information systems management positions - a market perspective

Information systems (IS) managers have become key senior executives for organising the IT resources for delivering support to businesses. Understanding characteristics of IS managers’ employment positions is hence an increasingly important topic in computer personnel research. An investigation in Singapore that included a job advertisement analysis, surveys and case studies was thus conducted to investigate such aspects. This article presents the findings of the job advertisement analysis concerning what kinds of IS managers the market is seeking and what are the basic conditions for such management positions. The literature in this area asserts that job advertisements represent firms’ wishes and the nature of the conditions required of different IS personnel. The results of this analysis therefore reflect a collective market perspective about the changing IS managerial workplace. The results of the analysis benefit both firms and IS employees in formulating personnel development plans and actions, and raise issues for further research.

10th European Conference on Information Systems Management

The 10th European Conference on Information Systems Management is being held at The University of Evora, Portugal on the 8 /9 September 2016. The Conference Chair is Paulo Silva and the Programme Chairs are Prof. Rui Quaresma and Prof. António Guerreiro. ECISM provides an opportunity for individuals researching and working in the broad field of information systems management, including IT evaluation to come together to exchange ideas and discuss current research in the field. This has developed into a particularly important forum for the present era, where the modern challenges of managing information and evaluating the effectiveness of related technologies are constantly evolving in the world of Big Data and Cloud Computing. We hope that this year’s conference will provide you with plenty of opportunities to share your expertise with colleagues from around the world. The keynote speakers for the Conference are Carlos Zorrinho from the Portuguese Delegation and Isabel Ramos from University of Minho, Portugal. ECISM 2016 received an initial submission of 84 abstracts. After the double blind peer review process 25 aca demic papers, 7 PhD research papers, 3 Masters research paper and 5 work in progress papers have been ac cepted for publication in these Conference Proceedings. These papers represent research from around the world, including Belgium, Brazil, China, Czech Republic, Kazakhstan, Malaysia, New Zealand, Norway, Oman, Poland, Portugal, South Africa, Sweden, The Netherlands, UK and Vietnam.

Identifying the knowledge management structures in enterprise systems projects

Enterprise System (ES) implementation and management are knowledge intensive tasks that inevitably draw upon the experience of a wide range of people with diverse knowledge capabilities. Knowledge Management (KM) has been identified as a critical success factor in ES projects. Despite the recognized importance of managing knowledge for ES benefits realization, systematic attempts to conceptualize KM-structures have been few. Where the adequacy of KM-structures is assessed, the process and measures are typically idiosyncratic and lack credibility. Using the ‘KM-process’, itself based in sociology of knowledge, this paper conceptualizes four main constructs to measure the adequacy of KM-structures. The SEM model is tested using 310 responses gathered from 27 ES installations that had implemented SAP R/3. The findings reveal six constructs for KM-structure. Furthermore, the paper demonstrates the application of KM-structures in the context of ES using the Adaptive Structuration Theory. The results demonstrate that having adequate KM-structures in place, while necessary, is not sufficient. These rules and resources must be appropriated to have greater positive influence on the Enterprise System. Furthermore, the study provides empirical support for knowledge-based theory by illustrating the importance of knowledge use/re-use (vs. knowledge creation) as the most important driver in the process of KM.

Key organizational elements for effective information and knowledge management

Effective information and knowledge management (IKM) is critical to corporate success; yet, its actual establishment and management is not yet fully understood. We identify ten organizational elements that need to be addressed to ensure the effective implementation and maintenance of information and knowledge management within organizations. We define these elements and provide key characterizations. We then discuss a case study that describes the implementation of an information system (designed to support IKM) in a medical supplies organization. We apply the framework of organizational elements in our analysis to uncover the enablers and barriers in this systems implementation project. Our analysis suggests that taking the ten organizational elements into consideration when implementing information systems will assist practitioners in managing information and knowledge processes more effectively and efficiently. We discuss implications for future research.

Modelling complex resource requirements in Business Process Management Systems

Real-world business processes are resource-intensive. In work environments human resources usually multitask, both human and non-human resources are typically shared between tasks, and multiple resources are sometimes necessary to undertake a single task. However, current Business Process Management Systems focus on task-resource allocation in terms of individual human resources only and lack support for a full spectrum of resource classes (e.g., human or non-human, application or non-application, individual or teamwork, schedulable or unschedulable) that could contribute to tasks within a business process. In this paper we develop a conceptual data model of resources that takes into account the various resource classes and their interactions. The resulting conceptual resource model is validated using a real-life healthcare scenario.

Using management objectives to specify management information systems : a contribution to MIS success

Data warehouse projects, today, are in an ambivalent situation. On the one hand, data warehouses are critical for a company’s success and various methodological and technological tools are sophisticatedly developed to implement them. On the other hand, a significant amount of data warehouse projects fails due to non-technical reasons such as insufficient management support or in-corporative employees. But management support and user participation can be increased dramatically with specification methods that are understandable to these user groups. This paper aims at overcoming possible non-technical failure reasons by introducing a user-adequate specification approach within the field of management information systems.

Business information systems:technology, development and management in the E-business

This major text assumes no prior knowledge of IS or IT and builds both business and Information systems knowledge to enable the reader to choose the right systems, to develop them and to manage them effectively. The three-part structure to the book covers: Introduction to business information systems Business information systems development Business information systems management Suitable for any IS, BIS or MIS course from UG to MBA level within a Business or Computer Science Department.