Evaluating information systems

The book within which this chapter appears is published as a research reference book (not a coursework textbook) on Management Information Systems (MIS) for seniors or graduate students in Chinese universities. It is hoped that this chapter, along with the others, will be helpful to MIS scholars and PhD/Masters research students in China who seek understanding of several central Information Systems (IS) research topics and related issues. The subject of this chapter - ‘Evaluating Information Systems’ - is broad, and cannot be addressed in its entirety in any depth within a single book chapter. The chapter proceeds from the truism that organizations have limited resources and those resources need to be invested in a way that provides greatest benefit to the organization. IT expenditure represents a substantial portion of any organization’s investment budget and IT related innovations have broad organizational impacts. Evaluation of the impact of this major investment is essential to justify this expenditure both pre- and post-investment. Evaluation is also important to prioritize possible improvements. The chapter (and most of the literature reviewed herein) admittedly assumes a blackbox view of IS/IT1, emphasizing measures of its consequences (e.g. for organizational performance or the economy) or perceptions of its quality from a user perspective. This reflects the MIS emphasis – a ‘management’ emphasis rather than a software engineering emphasis2, where a software engineering emphasis might be on the technical characteristics and technical performance. Though a black-box approach limits diagnostic specificity of findings from a technical perspective, it offers many benefits. In addition to superior management information, these benefits may include economy of measurement and comparability of findings (e.g. see Part 4 on Benchmarking IS). The chapter does not purport to be a comprehensive treatment of the relevant literature. It does, however, reflect many of the more influential works, and a representative range of important writings in the area. The author has been somewhat opportunistic in Part 2, employing a single journal – The Journal of Strategic Information Systems – to derive a classification of literature in the broader domain. Nonetheless, the arguments for this approach are believed to be sound, and the value from this exercise real. The chapter drills down from the general to the specific. It commences with a highlevel overview of the general topic area. This is achieved in 2 parts: - Part 1 addressing existing research in the more comprehensive IS research outlets (e.g. MISQ, JAIS, ISR, JMIS, ICIS), and Part 2 addressing existing research in a key specialist outlet (i.e. Journal of Strategic Information Systems). Subsequently, in Part 3, the chapter narrows to focus on the sub-topic ‘Information Systems Success Measurement’; then drilling deeper to become even more focused in Part 4 on ‘Benchmarking Information Systems’. In other words, the chapter drills down from Parts 1&2 Value of IS, to Part 3 Measuring Information Systems Success, to Part 4 Benchmarking IS. While the commencing Parts (1&2) are by definition broadly relevant to the chapter topic, the subsequent, more focused Parts (3 and 4) admittedly reflect the author’s more specific interests. Thus, the three chapter foci – value of IS, measuring IS success, and benchmarking IS - are not mutually exclusive, but, rather, each subsequent focus is in most respects a sub-set of the former. Parts 1&2, ‘the Value of IS’, take a broad view, with much emphasis on ‘the business Value of IS’, or the relationship between information technology and organizational performance. Part 3, ‘Information System Success Measurement’, focuses more specifically on measures and constructs employed in empirical research into the drivers of IS success (ISS). (DeLone and McLean 1992) inventoried and rationalized disparate prior measures of ISS into 6 constructs – System Quality, Information Quality, Individual Impact, Organizational Impact, Satisfaction and Use (later suggesting a 7th construct – Service Quality (DeLone and McLean 2003)). These 6 constructs have been used extensively, individually or in some combination, as the dependent variable in research seeking to better understand the important antecedents or drivers of IS Success. Part 3 reviews this body of work. Part 4, ‘Benchmarking Information Systems’, drills deeper again, focusing more specifically on a measure of the IS that can be used as a ‘benchmark’3. This section consolidates and extends the work of the author and his colleagues4 to derive a robust, validated IS-Impact measurement model for benchmarking contemporary Information Systems (IS). Though IS-Impact, like ISS, has potential value in empirical, causal research, its design and validation has emphasized its role and value as a comparator; a measure that is simple, robust and generalizable and which yields results that are as far as possible comparable across time, across stakeholders, and across differing systems and systems contexts.

Object-centric process models and the design of flexible processes

Mainstream business process modelling techniques promote a design paradigm wherein the activities to be performed within a case, together with their usual execution order, form the backbone of a process model, on top of which other aspects are anchored. This paradigm, while eective in standardised and production-oriented domains, shows some limitations when confronted with processes where case-by-case variations and exceptions are the norm. In this thesis we develop the idea that the eective design of exible process models calls for an alternative modelling paradigm, one in which process models are modularised along key business objects, rather than along activity decompositions. The research follows a design science method, starting from the formulation of a research problem expressed in terms of requirements, and culminating in a set of artifacts that have been devised to satisfy these requirements. The main contributions of the thesis are: (i) a meta-model for object-centric process modelling incorporating constructs for capturing exible processes; (ii) a transformation from this meta-model to an existing activity-centric process modelling language, namely YAWL, showing the relation between object-centric and activity-centric process modelling approaches; and (iii) a Coloured Petri Net that captures the semantics of the proposed meta-model. The meta-model has been evaluated using a framework consisting of a set of work ow patterns. Moreover, the meta-model has been embodied in a modelling tool that has been used to capture two industrial scenarios.

Strategic information systems research : an archival analysis

The Journal of Strategic Information Systems (JSIS) has been an international outlet for Information Systems research that focuses on strategic issues since 1991. This paper reports on an analysis of the research published in JSIS to date. The paper presents a preliminary classification system for research topics related to Strategic Information Systems into which all 316 JSIS research papers as at end 2009 are classified. Discussion on changing emphases in topics over time is provided, in the context of the editorial philosophy of the journal. The paper seeks to stimulate discussion on future directions for research in Strategic Information Systems.

Business service management : service and service quality

Managing service quality is of primary importance for organizations that are increasingly service oriented, and offering a growing range of services to external and internal customers. Managing service quality requires the capacity to measure service quality, concomitantly requiring explicit conceptions of ‘service’ and ‘service quality’. This white-paper explores three keys areas of service and service marketing literature: service definition and conceptualisation, service classifications, and service quality models, and make the following observations and proposals.

Re-conceptualising IS function's support performance : a preliminary model

While IS function has gained widespread attention for over two decades, there is little consensus among information systems (IS) researchers and practitioners on how best to evaluate IS function's support performance. This paper reports on preliminary findings of a larger research effort proceeds from a central interest in the importance of evaluating IS function's support in organisations. This study is the first that attempts to re-conceptualise and conceive evaluate IS function's support as a multi- dimensional formative construct. We argue that a holistic measure for evaluating evaluate IS function's support should consist of dimensions that together assess the variety of the support functions and the quality of the support services provided to end-users. Thus, the proposed model consists of two halves, "Variety" and "Quality" within which resides seven dimensions. The Variety half includes five dimensions: Training; Documentation; Data- related Support, Software-related Support; and Hardware-related Support. The Quality half includes two dimensions: IS Support Staff and Support Services Performance. The proposed model is derived using a directed content analysis of 83 studies; from top IS outlets, employing the characteristics of the analytic theory and consistent with formative construct development procedures.

Enterprise systems in universities : a teaching case

This teaching case aims to contribute to understanding the phenomenon of Enterprise Systems (ES) implementations in universities. Through this case, students will gain understanding of the importance of ‘contextual elements’ for large scale information systems (IS) implementations, in particular ES. This teaching case illustrates how these contextual factors contribute to the success or failure of such implementations, and how they can influence the decisions that dictate the lifecycle of such systems. The case describes ES implementations at a leading Australian university, and presents a rich account of the institutional, national and industry-sector contexts that have influenced the directions and decisions taken. The journey encountered with the main Enterprise Systems that support Financials, Human Resources and Facilities are described suggesting the lifecycle phases, critical success factors and lessons learnt.

Understanding shared services : an exploration of the IS literature

In a competitive environment, companies continuously innovate to offer superior services at lower costs. ‘Shared Services’ have been extensively adopted in practice as a means for improving organizational performance. Shared Services are considered most appropriate for support functions and are widely adopted in human resource management, finance and accounting, and more recently employed as an information systems (IS) function. As computer-based corporate information systems have become de facto and the backbone of administrative systems, the technical impediments to sharing have come down dramatically. As this trend continues, CIOs and IT professionals need a deeper understanding of the Shared Services phenomenon. Yet, analysis of IS academic literature reveals that Shared Services, though mentioned in more than 100 articles, has received little in depth attention. This paper investigates the current status of Shared Services in IS literature. The authors present a detailed review of literature from main IS journals and conferences. The paper concludes with a tentative operational definition, a list of perceived main objectives of Shared Services, and an agenda for related future research.

Knowledge management competence for Enterprise System success

This study conceptualizes, operationalises and validates the concept of Knowledge Management Competence as a four-phase multidimensional formative index. Employing survey data from 310 respondents representing 27 organizations using the SAP Enterprise System Financial module, the study results demonstrate a large, significant, positive relationship between Knowledge Management Competence and Enterprise Systems Success (ES-success, as conceived by Gable Sedera and Chan (2008)); suggesting important implications for practice. Strong evidence of the validity of Knowledge Management Competence as conceived and operationalised, too suggests potential from future research evaluating its relationships with possible antecedents and consequences.

News produsage in a pro-am mediasphere : why citizen journalism matters

The rise to prominence of citizen journalism is usually described as a paradigm shift in the relationship between news organizations and their audiences – ‘citizen’ or ‘amateur’ journalists are positioned as inherently different from, and possibly in competition with, ‘professional’ journalists. Some journalists in the industry have taken up this theme, and are at pains to distinguish their professional, supposedly objective and accountable practices from what they describe as the opinionated and partisan ‘armchair journalism’ of amateurs, while some citizen journalists, in turn, give as good at they get and describe the professionals as lackeys of their corporate masters who (willingly or unwittingly) fall prey to political and commercial spin. In fact, the very terminology we use to describe both sides creates the impression that professionals are not also citizens, and that citizen journalists are incapable of having professional skills and knowledges; in reality, of course, the lines between them are much less clear.

Measurement of affect during simulation-based training

Training designed to support and strengthen higher-order mental abilities now often involves immersion in Virtual Reality where dangerous real world scenarios can be safely replicated. However despite the growing popularity of advanced training simulations, methods for evaluating their use rely heavily on subjective measures or analysis of final outcomes. Without dynamic, objective performance measures the outcome of training in terms of impact on cognitive skills and ability to transfer newly acquired skills to the real world is unknown. The relationship between affective intensity and cognitive learning provides a potential new approach to ensure the processing of cognitions which occur prior to final outcomes, such as problem-solving and decision-making, are adequately evaluated. This paper describes the technical aspects of pilot work recently undertaken to develop a new measurement tool designed to objectively track individual affect levels during simulation-based training.

Contemporary take on deconstructed Medea : a review

THEATRE: The New Dead: Medea Material. By Heiner Muller. Stella Electrika in association with La Boite Theatre Company, Brisbane, November 19. THERE has been a lot of intensity in independent theatre in Brisbane during the past year, as companies, production houses and producers have begun building new programs and platforms to support an expansion of pathways within the local theatre ecology. Audiences have been exposed to works signalling the diversity of what Brisbane theatre makers want to see on stage, from productions of new local and international pieces to new devised works, and the results of residencies and development programs. La Boite Theatre Company closes its inaugural indie season with a work that places it at the contemporary, experimental end of the spectrum. The New Dead: Medea Material is emerging director Kat Henry's interpretation of Heiner Muller's 1981 text Despoiled Shore Medea Material Landscape with Argonauts. Start of sidebar. Skip to end of sidebar. End of sidebar. Return to start of sidebar. Muller is known for his radical adaptations of historical dramas, from the Greeks to Shakespeare, and for deconstructed texts in which the characters - in this case, Medea - violently reject the familial, cultural and political roles society has laid out for them. Muller's combination of deconstructed characters, disconnected poetic language and constant references to aspects of popular culture and the Cold War politics he sought to abjure make his texts challenging to realise. The poetry entices but the density, together with the increasing distance of the Cold War politics in the texts, leaves contemporary directors with clear decisions to make about how to adapt these open texts. In The New Dead: Medea Material, Henry works with some interesting imagery and conceptual territory. Lucinda Shaw as Medea, Guy Webster as Jason and Kimie Tsukakoshi as King Creon's daughter Glauce, the woman for whom Jason forsakes his wife Medea, each reference different aspects of contemporary culture. Medea is a bitter, drunken, satin-gowned diva with bite; Jason - first seen lounging in front of the television with a beer in an image reminiscent of Sarah Kane's in-yer-face characterisation of Hippolytus in Phaedra's Love - has something of the rock star about him; and Glauce is a roller-skating, karaoke-singing, pole-dancing young temptress. The production is given a contemporary tone, dominated by Medea's twisted love and loss, rather than by any commentary on her circumstances. Its strength is the aesthetic Henry creates, supported by live electro-pop music, a band stage that stands as a metaphor for Jason's sea voyage, and multimedia that inserts images of the story unfolding beyond these characters' speeches as sorts of subconscious flashes. While Tsukakoshi is engaging throughout, there are moments when Shaw and Webster's performances - particularly in the songs - are diminished by a lack of clarity. The result is a piece that, while slightly lacking in its realisation at times, undoubtedly flags Henry's facility as an emerging director and what she wants to bring to the Brisbane theatre scene.

What is ERP?

Though enterprise resource planning (ERP) has gained some prominence in the information systems (IS) literature over the past few years and is a significant phenomenon in practice, through (a) a historical analysis, (b) meta-analysis of representative IS literature, and (c) a survey of academic experts, we reveal dissenting views on the phenomenon. Given this diversity of perspectives, it is unlikely that at this stage a broadly agreed definition of ERP can be achieved. We thus seek to increase awareness of the issues and stimulate further discussion, with the ultimate aim being to: (a) aid communication amongst researchers and between researchers and practitioners; (2) inform development of teaching materials on ERP and related concepts in university curricula and in commercial education and training; and (3) aid communication amongst clients, consultants and vendors. Increase transparence of the ERP concept within IS may also benefit other aligned fields of knowledge.