Evaluating information systems

The book within which this chapter appears is published as a research reference book (not a coursework textbook) on Management Information Systems (MIS) for seniors or graduate students in Chinese universities. It is hoped that this chapter, along with the others, will be helpful to MIS scholars and PhD/Masters research students in China who seek understanding of several central Information Systems (IS) research topics and related issues. The subject of this chapter - ‘Evaluating Information Systems’ - is broad, and cannot be addressed in its entirety in any depth within a single book chapter. The chapter proceeds from the truism that organizations have limited resources and those resources need to be invested in a way that provides greatest benefit to the organization. IT expenditure represents a substantial portion of any organization’s investment budget and IT related innovations have broad organizational impacts. Evaluation of the impact of this major investment is essential to justify this expenditure both pre- and post-investment. Evaluation is also important to prioritize possible improvements. The chapter (and most of the literature reviewed herein) admittedly assumes a blackbox view of IS/IT1, emphasizing measures of its consequences (e.g. for organizational performance or the economy) or perceptions of its quality from a user perspective. This reflects the MIS emphasis – a ‘management’ emphasis rather than a software engineering emphasis2, where a software engineering emphasis might be on the technical characteristics and technical performance. Though a black-box approach limits diagnostic specificity of findings from a technical perspective, it offers many benefits. In addition to superior management information, these benefits may include economy of measurement and comparability of findings (e.g. see Part 4 on Benchmarking IS). The chapter does not purport to be a comprehensive treatment of the relevant literature. It does, however, reflect many of the more influential works, and a representative range of important writings in the area. The author has been somewhat opportunistic in Part 2, employing a single journal – The Journal of Strategic Information Systems – to derive a classification of literature in the broader domain. Nonetheless, the arguments for this approach are believed to be sound, and the value from this exercise real. The chapter drills down from the general to the specific. It commences with a highlevel overview of the general topic area. This is achieved in 2 parts: - Part 1 addressing existing research in the more comprehensive IS research outlets (e.g. MISQ, JAIS, ISR, JMIS, ICIS), and Part 2 addressing existing research in a key specialist outlet (i.e. Journal of Strategic Information Systems). Subsequently, in Part 3, the chapter narrows to focus on the sub-topic ‘Information Systems Success Measurement’; then drilling deeper to become even more focused in Part 4 on ‘Benchmarking Information Systems’. In other words, the chapter drills down from Parts 1&2 Value of IS, to Part 3 Measuring Information Systems Success, to Part 4 Benchmarking IS. While the commencing Parts (1&2) are by definition broadly relevant to the chapter topic, the subsequent, more focused Parts (3 and 4) admittedly reflect the author’s more specific interests. Thus, the three chapter foci – value of IS, measuring IS success, and benchmarking IS - are not mutually exclusive, but, rather, each subsequent focus is in most respects a sub-set of the former. Parts 1&2, ‘the Value of IS’, take a broad view, with much emphasis on ‘the business Value of IS’, or the relationship between information technology and organizational performance. Part 3, ‘Information System Success Measurement’, focuses more specifically on measures and constructs employed in empirical research into the drivers of IS success (ISS). (DeLone and McLean 1992) inventoried and rationalized disparate prior measures of ISS into 6 constructs – System Quality, Information Quality, Individual Impact, Organizational Impact, Satisfaction and Use (later suggesting a 7th construct – Service Quality (DeLone and McLean 2003)). These 6 constructs have been used extensively, individually or in some combination, as the dependent variable in research seeking to better understand the important antecedents or drivers of IS Success. Part 3 reviews this body of work. Part 4, ‘Benchmarking Information Systems’, drills deeper again, focusing more specifically on a measure of the IS that can be used as a ‘benchmark’3. This section consolidates and extends the work of the author and his colleagues4 to derive a robust, validated IS-Impact measurement model for benchmarking contemporary Information Systems (IS). Though IS-Impact, like ISS, has potential value in empirical, causal research, its design and validation has emphasized its role and value as a comparator; a measure that is simple, robust and generalizable and which yields results that are as far as possible comparable across time, across stakeholders, and across differing systems and systems contexts.

Remote interaction and user research : anaesthetic preadmission activity

In the absence of telehealth technology, rural patients must travel to a regional or metropolitan hospital for a preadmission consultation one week before their surgery. Currently, examination of the patient’s chest using a stethoscope (auscultation) is not possible over a telehealth network as existing digital stethoscopes have been designed for in-person auscultation. We report on the initial phase of research which ultimately aims to design a digital stethoscope for use in the telehealth context. This initial research phase describes the complexity of the activity of preadmission clinics and the implications for the design of the stethoscope. The research is conducted through field studies of existing face-to-face and remote consultations.

User-centered video quality assessment for scalable video coding of H.264/AVC standard

Scalable video coding of H.264/AVC standard enables adaptive and flexible delivery for multiple devices and various network conditions. Only a few works have addressed the influence of different scalability parameters (frame rate, spatial resolution, and SNR) on the user perceived quality within a limited scope. In this paper, we have conducted an experiment of subjective quality assessment for video sequences encoded with H.264/SVC to gain a better understanding of the correlation between video content and UPQ at all scalable layers and the impact of rate-distortion method and different scalabilities on bitrate and UPQ. Findings from this experiment will contribute to a user-centered design of adaptive delivery of scalable video stream.

User-generated content and the future of public broadcasting : a case study of the special broadcasting service

This thesis presents a case study of the Special Broadcasting Service documenting the broadcasting challenges posed by user-generated content initiatives and the work-place approach to strategies for participation. Using the action research method, the project findings reveal that limitations to resources and funding determined the scope for innovation and that the practice of executive editorial control over content was considered fundamental to fulfilling the responsibilities of the public service mandate. Media workers were overwhelmingly positive about the enhanced productive capabilities of the audience and willing to facilitate moderated interactions, however the effectiveness of these initiatives differed according to the level of skills required. This thesis demonstrates how participatory initiatives can enhance aspects of the public service remit relating to cultural diversity, the servicing of niche interests, and broader social representation, and help reinvigorate the relevance of public service broadcasting in the digitalised media sphere.

User-Created Content and Online Social Networks

Traditional media are under assault from digital technologies. Online advertising is eroding the financial basis of newspapers and television, demarcations between different forms of media are fading, and audiences are fragmenting. We can podcast our favourite radio show, data accompanies television programs, and we catch up with newspaper stories on our laptops. Yet mainstream media remain enormously powerful. The Media and Communications in Australia offers a systematic introduction to this dynamic field. Fully updated and revised to take account of recent developments, this third edition outlines the key media industries and explains how communications technologies are impacting on them. It provides a thorough overview of the main approaches taken in studying the media, and includes new chapters on social media, gaming, telecommunications, sport and cultural diversity. With contributions from some of Australia's best researchers and teachers in the field, The Media and Communications in Australia is the most comprehensive and reliable introduction to media and communications available. It is an ideal student text, and a reference for teachers of media and anyone interested in this influential industry.

Charities, civil society and the charity law reviews on the island of Ireland

This article considers the opportunity, presented by the coincidence of simultaneous charity law reviews in the two jurisdictions on the island of Ireland, for an adjustment of charity law frameworks to maximise appropriate and effective charitable activity within each jurisdiction,while also facilitating the coordination of some such activity between both. It examines the nature of civil society and charity law, and the relationship between them. The article argues that a creative legislative response to this opportunity could address themes of social inclusion common to both jurisdictions and thereby contribute to the consolidation of civil society on this island.

Modelling urban public transit users’ route choice behaviour : a review and outlook

Public transport is one of the key promoters of sustainable urban transport. To encourage and increase public transport patronage it is important to investigate the route choice behaviours of urban public transit users. This chapter reviews the main developments of modelling urban public transit users’ route choice behaviours in a historical perspective, from the 1960s to the present time. The approaches re- viewed for this study include the early heuristic studies on finding the least-cost transit route and all-or- nothing transit assignment, the bus common lines problem, the disaggregate discrete choice models, the deterministic and stochastic user equilibrium transit assignment models, and the recent dynamic transit assignment models. This chapter also provides an outlook for the future directions of modelling transit users’ route choice behaviours. Through the comparison with the development of models for motorists’ route choice and traffic assignment problems, this chapter advocates that transit route choice research should draw inspiration from the research outcomes from the road area, and that the modelling practice of transit users’ route choice should further explore the behavioural complexities.

Analysis of primary user duty cycle impact on spectrum sensing performance

Spectrum sensing is considered to be one of the most important tasks in cognitive radio. Many sensing detectors have been proposed in the literature, with the common assumption that the primary user is either fully present or completely absent within the window of observation. In reality, there are scenarios where the primary user signal only occupies a fraction of the observed window. This paper aims to analyse the effect of the primary user duty cycle on spectrum sensing performance through the analysis of a few common detectors. Simulations show that the probability of detection degrades severely with reduced duty cycle regardless of the detection method. Furthermore we show that reducing the duty cycle has a greater degradation on performance than lowering the signal strength.

New Media and Public Communication: Mapping Australian User-Created Content in Online Social Networks

Presentation describling a project in data intensive research in the humanities. Measuring activity of publically available data in social networks such as Blogosphere, Twitter, Flickr, YouTube

An architecture for user configurable mobile collaborative geographic applications

Geographic information is increasingly being touted for use in research and industrial projects. While the technology is now available and affordable, there is a lack of easy to use software that takes advantage of geographic information. This is an important problem because users are often researchers or scientists who have insufficient software skills, and by providing applications that are easier to use, time and financial resources can be taken from training and be better applied to the actual research and development work. A solution for this problem must cater for the user and research needs. In particular it must allow for mobile operation for fieldwork, flexibility or customisability of data input, sharing of data with other tools and collaborative capabilities for the usual teamwork environment. This thesis has developed a new architecture and data model to achieve the solution. The result is the Mobile Collaborative Annotation framework providing an implementation of the new architecture and data model. Mobile Collaborative Mapping implements the framework as a Web 2.0 mashup rich internet application and has proven to be an effective solution through its positive application to a case study with fieldwork scientists. This thesis has contributed to research into mobile computing, collaborative computing and geospatial systems by creating a simpler entry point to mobile geospatial applications, enabling simplified collaboration and providing tangible time savings.

Transit users' route-choice modelling in transit assignment : a review

This paper reviews the main studies on transit users’ route choice in thecontext of transit assignment. The studies are categorized into three groups: static transit assignment, within-day dynamic transit assignment, and emerging approaches. The motivations and behavioural assumptions of these approaches are re-examined. The first group includes shortest-path heuristics in all-or-nothing assignment, random utility maximization route-choice models in stochastic assignment, and user equilibrium based assignment. The second group covers within-day dynamics in transit users’ route choice, transit network formulations, and dynamic transit assignment. The third group introduces the emerging studies on behavioural complexities, day-to-day dynamics, and real-time dynamics in transit users’ route choice. Future research directions are also discussed.