Evaluating information systems

The book within which this chapter appears is published as a research reference book (not a coursework textbook) on Management Information Systems (MIS) for seniors or graduate students in Chinese universities. It is hoped that this chapter, along with the others, will be helpful to MIS scholars and PhD/Masters research students in China who seek understanding of several central Information Systems (IS) research topics and related issues. The subject of this chapter - ‘Evaluating Information Systems’ - is broad, and cannot be addressed in its entirety in any depth within a single book chapter. The chapter proceeds from the truism that organizations have limited resources and those resources need to be invested in a way that provides greatest benefit to the organization. IT expenditure represents a substantial portion of any organization’s investment budget and IT related innovations have broad organizational impacts. Evaluation of the impact of this major investment is essential to justify this expenditure both pre- and post-investment. Evaluation is also important to prioritize possible improvements. The chapter (and most of the literature reviewed herein) admittedly assumes a blackbox view of IS/IT1, emphasizing measures of its consequences (e.g. for organizational performance or the economy) or perceptions of its quality from a user perspective. This reflects the MIS emphasis – a ‘management’ emphasis rather than a software engineering emphasis2, where a software engineering emphasis might be on the technical characteristics and technical performance. Though a black-box approach limits diagnostic specificity of findings from a technical perspective, it offers many benefits. In addition to superior management information, these benefits may include economy of measurement and comparability of findings (e.g. see Part 4 on Benchmarking IS). The chapter does not purport to be a comprehensive treatment of the relevant literature. It does, however, reflect many of the more influential works, and a representative range of important writings in the area. The author has been somewhat opportunistic in Part 2, employing a single journal – The Journal of Strategic Information Systems – to derive a classification of literature in the broader domain. Nonetheless, the arguments for this approach are believed to be sound, and the value from this exercise real. The chapter drills down from the general to the specific. It commences with a highlevel overview of the general topic area. This is achieved in 2 parts: - Part 1 addressing existing research in the more comprehensive IS research outlets (e.g. MISQ, JAIS, ISR, JMIS, ICIS), and Part 2 addressing existing research in a key specialist outlet (i.e. Journal of Strategic Information Systems). Subsequently, in Part 3, the chapter narrows to focus on the sub-topic ‘Information Systems Success Measurement’; then drilling deeper to become even more focused in Part 4 on ‘Benchmarking Information Systems’. In other words, the chapter drills down from Parts 1&2 Value of IS, to Part 3 Measuring Information Systems Success, to Part 4 Benchmarking IS. While the commencing Parts (1&2) are by definition broadly relevant to the chapter topic, the subsequent, more focused Parts (3 and 4) admittedly reflect the author’s more specific interests. Thus, the three chapter foci – value of IS, measuring IS success, and benchmarking IS - are not mutually exclusive, but, rather, each subsequent focus is in most respects a sub-set of the former. Parts 1&2, ‘the Value of IS’, take a broad view, with much emphasis on ‘the business Value of IS’, or the relationship between information technology and organizational performance. Part 3, ‘Information System Success Measurement’, focuses more specifically on measures and constructs employed in empirical research into the drivers of IS success (ISS). (DeLone and McLean 1992) inventoried and rationalized disparate prior measures of ISS into 6 constructs – System Quality, Information Quality, Individual Impact, Organizational Impact, Satisfaction and Use (later suggesting a 7th construct – Service Quality (DeLone and McLean 2003)). These 6 constructs have been used extensively, individually or in some combination, as the dependent variable in research seeking to better understand the important antecedents or drivers of IS Success. Part 3 reviews this body of work. Part 4, ‘Benchmarking Information Systems’, drills deeper again, focusing more specifically on a measure of the IS that can be used as a ‘benchmark’3. This section consolidates and extends the work of the author and his colleagues4 to derive a robust, validated IS-Impact measurement model for benchmarking contemporary Information Systems (IS). Though IS-Impact, like ISS, has potential value in empirical, causal research, its design and validation has emphasized its role and value as a comparator; a measure that is simple, robust and generalizable and which yields results that are as far as possible comparable across time, across stakeholders, and across differing systems and systems contexts.

Effectiveness of constructability concept in the provision of infrastructure assets

The concept of constructability is to use construction knowledge and experience during all phases of a project, particularly in the earliest phases of planning and design. It facilitates project objectives before delivery stage, and decreases unnecessary costs during construction phase. Despite the extensive use, constructability concept fails to address many issues related to Operation and Maintenance (O&M) of construction projects. Extending constructability concept, to include the O&M issues, could lead to the projects that are not fitted for construction purposes only, but also fitted for use. This study reviews the literature of constructability implementation, its benefits and shortcomings during the infrastructure life cycle, as well as the delivery success factors of infrastructure projects. This contributes to the propose need of a model to improve the effectiveness and efficiency of infrastructure project by extending the concept of constructability to include O&M. Development of such a model can facilitate post-occupancy stakeholders’ participation in a constructability program. It will help infrastructure owners eliminate project reworks, and improve O&M effectiveness and efficiency.

Extending constructability concept to include operation and maintenance issues

The concept of constructability uses integration art of individual functions through a valuable and timely construction inputs into planning and design development stages. It results in significant savings in cost and time needed to finalize infrastructure projects. However, available constructability principles, developed by CII Australia (1993), do not cover Operation and Maintenance (O&M) phases of projects, whilst major cost and time in multifaceted infrastructure projects are spent in post-occupancy stages. This paper discusses the need to extend the constructability concept by examining current O&M issues in the provision of multifaceted building projects. It highlights available O&M problems and shortcomings of building projects, as well as their causes and reasons in different categories. This initial categorization is an efficient start point for testing probable present O&M issues in various cases of complex infrastructure building projects. This preliminary categorization serve as a benchmark to develop an extended constructability model that considers the whole project life cycle phases rather than a specific phase. It anticipates that the development of an extended constructability model can reduce significant number of reworks, mistakes, extra costs and time wasted during delivery stages of multifaceted building projects.

A survey to evaluate the implementation of a national clinical assessment form

Purpose: The Australian Universities Radiation Therapy Student Clinical Assessment Form (AURTSCAF) was designed to assess the clinical skills of radiation therapy (RT) students from the six universities that offer entry level RT programs. Given the AURTSCAF has now been in use for over two years, the Radiation Therapy Program Coordinators (RTPC) group initiated a post implementation evaluation survey. This formed the final phase of the AURTSCAF project and was funded by the Radiation Oncology Division of the Department of Health and Ageing. Methods: A cross-sectional designed survey using purposive sampling was distributed via email to all RT clinical sites. The survey asked questions about the requirements of a pass grade for students at different stages of their program, and the addition of a new category of assessment related to fitness to practise. Response types included both forced choice closed ended responses and open ended responses. There was also a section for open comments about the AURTSCAF. Results: There were 100 responses (55%) from clinicians who had utilised the assessment form over the previous 12 month period. Responses highlighted several positives with regard to the utility and implementation of the form. Comments regarding areas for improvement with the standardisation of the grading of students and consensus for the addition of a new domain in fitness for practise have informed the recommended changes proposed for 2012. Conclusion: This evaluation has provided a representative sample of the views of clinicians involved in assessing students on clinical placement. Recommendations include the addition of the sixth domain of assessment: Fitness for practise, the addition of descriptors and prompts for this domain in the user guide, the addition of a consensus statement about the use of the rating scale and dissemination of the proposed changes nationally.

A multidimensional model of client success when engaging external consultants

Too often the relationship between client and external consultants is perceived as one of protagonist versus antogonist. Stories on dramatic, failed consultancies abound, as do related anecdotal quips. A contributing factor to many "apparently" failed consultancies is a poor appreciation by both the client and consultant of the client's true goals for the project and how to assess progress toward these goals. This paper presents and analyses a measurement model for assessing client success when engaging an external consultant. Three main areas of assessment are identified: (1) the consultant;s recommendations, (2) client learning, and (3) consultant performance. Engagement success is emperically measured along these dimensions through a series of case studies and a subsequent survey of clients and consultants involved in 85 computer-based information system selection projects. Validation fo the model constructs suggests the existence of six distinct and individually important dimensions of engagement success. both clients and consultants are encouraged to attend to these dimensions in pre-engagement proposal and selection processes, and post-engagement evaluation of outcomes.

Extension of constructability to include operation and maintenance for infrastructure projects

This project was a step forward in developing an extension of the concept of constructability to include the post-occupancy stages of operation and maintenance. This was through an in-depth study of Australian health projects and interviews with professionals in the field. The thesis investigated how the operation and maintenance stakeholders can enter the initial planning, design and construction phases resulting in more efficient and effective delivery of infrastructure projects.

Review of approaches to evaluate the effectiveness of weed biological control agents

We review key issues, available approaches and analyses to encourage and assist practitioners to develop sound plans to evaluate the effectiveness of weed biological control agents at various phases throughout a program. Assessing the effectiveness of prospective agents before release assists the selection process, while post-release evaluation aims to determine the extent that agents are alleviating the ecological, social and economic impacts of the weeds. Information gathered on weed impacts prior to the initiation of a biological control program is necessary to provide baseline data and devise performance targets against which the program can subsequently be evaluated. Detailed data on weed populations, associated plant communities and, in some instances ecosystem processes collected at representative sites in the introduced range several years before the release of agents can be compared with similar data collected later to assess agent effectiveness. Laboratory, glasshouse and field studies are typically used to assess agent effectiveness. While some approaches used for field studies may be influenced by confounding factors, manipulative experiments where agents are excluded (or included) using chemicals or cages are more robust but time-consuming and expensive to implement. Demographic modeling and benefit–cost analyses are increasingly being used to complement other studies. There is an obvious need for more investment in long-term post-release evaluation of agent effectiveness to rigorously document outcomes of biological control programs.

An outpatient respiratory rehabilitation program:Findings from a community hospital

Objectives: To determine patient satisfaction with a community hospital's respiratory rehabilitation program and to assess changes in patient physical and emotional function and quality of life. Design: Pre- and post-program measures were made on a variety of physiological and psychosocial factors. A modified version of the Chronic Respiratory Disease Questionnaire was administered before and after the 8-week multidisciplinary and comprehensive respiratory rehabilitation program. The post-program questionnaire also included a number of service delivery and patient satisfaction and quality-of-life questions. Setting: Respiratory Rehabilitation Program at St. Joseph's Hospital, a community hospital in Brantford, Ont., in active partnership with the Brant County Lung Association. Brant County is located in Central West Ontario, and has both urban and rural areas and a population of approximately 125 000 people. Participants: Twenty-nine patients, with a diagnosis of moderate to severe chronic obstructive pulmonary disease (COPD) who were referred to the Fall 1997 and Spring 1998 programs, were enrolled in the study. Outcome measures: Changes in physical and emotional function, health knowledge, skills mastery, quality of life and satisfaction with the program. Results: Twenty-one of 29 patients completed the program. Statistically significant and clinically important improvements were found between all pre- and post-program evaluation scores (distance walked, fatigue, dyspnea, emotional function, skills mastery and health knowledge). Participants were very satisfied with the program and felt it improved their quality of life. Conclusion: The positive outcomes reported rom randomized controlled trials of respiratory rehabilitation programs can be achieved in a community hospital setting.

Characterisation of Bioresorbable Composite Performance for Validation of New On-Line Process Monitoring Technology

There is an increasing interest in the biomedical field to create implantable medical devices to provide a temporary mechanical function for use inside the human body. In many of these applications bioresorbable polymer composites using PLLA with β-TCP , are increasingly being used due to their biocompatability, biodegradability and mechanical strength.1,3 These medical devices can be manufactured using conventional plastics processing methods such as injection moulding and extrusion, however there is great need to understand and control the process due to a lack of knowledge on the influence of processing on material properties. With the addition of biocompatible additives there is also a requirement to be able to predict the quality and level of dispersion within the polymer matrix. On-line UV-Vis spectroscopy has been shown to monitor the quality of fillers in polymers. This can eliminate time consuming and costly post-process evaluation of additive dispersion. The aim of this work was to identify process and performance relationships of PLLA/β-TCP composites with respect to melt-extrusion conditions. This is part of a wider study into on-line process monitoring of bioresorbable polymers as used in the medical industry.
These results show that final properties of the PLLA/ β-TCP composite are highly influenced by the particle size and loading. UV-Vis spectroscopy can be used on-line to monitor the final product and this can be utilised as a valuable tool for quality control in an application where consistent performance is of paramount importance.

Adaptação na transição do pré-escolar para o 1º ano de escolaridade : delineamento e avaliação da eficácia de um programa de intervenção

Tese de doutoramento, Psicologia da Educação, Universidade de Lisboa, Instituto de Educação, 2015

Comparison of a leukocyte esterase test with endometrial cytology for the diagnosis of subclinical endometritis and correlation with first service pregnancy rate in postpartum Holstein cows

L’objectif de la présente étude était d’évaluer un test d’estérase leucocytaire (LE) pour le diagnostic de l’endométrite subclinique chez les vaches Holstein en période postpartum. Les tests effectués à partir d’échantillons provenant soit de l’endomètre (UtLE) ou du col utérin (CxLE) ont été comparés à la cytologie endométriale (CE). Par ailleurs, deux méthodes d’évaluation des lames ont été comparées. Deux cent quatre vingt-cinq vaches Holstein de 5 troupeaux laitiers commerciaux ont été évaluées entre 21 et 47 jours en lait (JEL). Soixante sept vaches ont été diagnostiquées avec une endométrite clinique suite à un examen transrectal et vaginoscopique et ont été exclues de l’étude. Deux cent dix-huit vaches ont eu des prélèvements pour la CE et le test LE. La fonction ovarienne a été déterminée à la palpation transrectale. La banque de données utilisée pour chacune des vaches a été effectuée à partir du logiciel DSA (Dossier de Santé Animale) laitier. Le pourcentage de neutrophiles était significativement corrélé avec les scores de LE utérin et cervical. L’activité de CxLE et UtLE diminuait significativement avec les JEL, mais n’était pas associée au risque de gestation à 90 JEL (n= 186). Le pourcentage de neutrophiles mesuré à la CE entre 32 et 47 JEL était associé significativement au risque de gestation à 90 JEL (n=94, P=0.04). Pour la même période, selon une analyse de survie, les vaches avec >2,6% de neutrophiles à la CE étaient définies comme étant atteintes d’une endométrite subclinique avec une prévalence de 56%. Les résultats indiquent que le test d’estérase utérin ou cervical a une bonne concordance avec le pourcentage de neutrophiles à la CE. Une endométrite subclinique diagnostiquée par cytologie endometriale entre 32 et 47 JEL est associée à une réduction du risque de gestation au premier service.

L’utilisation de la réadaptation par la réalité virtuelle pour traiter l’incontinence urinaire mixte de la femme âgée : une étude de faisabilité

But : Cette étude a comme objectif principal d’évaluer la faisabilité d’utiliser une combinaison de renforcement des muscles du plancher pelvien (MPP) et de réadaptation par la réalité virtuelle (RRV) (programme d’entraînement MPP/RRV) pour traiter l’incontinence urinaire mixte (IUM) de la femme âgée. La faisabilité est évaluée par le taux de participation et d’achèvement du programme d’entraînement MPP/RRV et du programme d’exercices à domicile. Les objectifs secondaires de cette étude sont 1) d’évaluer l’effet du programme d’entraînement MPP/RRV sur la sévérité des symptômes urinaires, sur les dimensions de la qualité de vie, sur la fonction musculaire des MPP et sur les fonctions cognitives, plus précisément le processus exécutif de coordination de deux tâches, 2) d’évaluer, suite à l’intervention, la satisfaction des participantes. Le volet qualitatif a, quant à lui, pour objectif d’identifier les facteurs ayant influencé la participation au programme d’entraînement MPP/RRV ainsi que les avantages et les inconvénients associés à l’ajout de la RRV au traitement conventionnel de renforcement des MPP. Méthode : Les participantes ont effectué deux évaluations pré-traitement (pré-1 et pré-2), ont assisté à 12 classes hebdomadaires d’entraînement MPP/RRV et ont effectué une évaluation post-traitement. Elles ont aussi complété un programme d’exercices des MPP à domicile. Finalement, suite à la dernière classe, elles ont participé à un groupe de discussion. Les évaluations pré-1 et pré-2 ont été effectuées à deux semaines d’intervalle et étaient nécessaires pour s’assurer de la stabilité des mesures en l’absence d’une intervention. Résultats : 24 femmes ont participé à l’étude. Les taux de participation aux classes d’exercices hebdomadaires et au programme d’exercices à domicile étaient de 91 % et de 92 % respectivement. Le taux d’achèvement au programme d’entraînement MPP/RRV était de 96 %. Le programme d’entraînement MPP/RRV s’est avéré efficace dans l’amélioration des symptômes urinaires, de la qualité de vie ainsi que de la capacité à coordonner deux tâches simultanées chez les femmes âgées avec de l’IUM; il tend également à améliorer la fonction musculaire du plancher pelvien de ces femmes. L’appréciation de la composante RRV, par les participantes, était de 9.8/10 et ces dernières l’ont identifiée comme un facilitateur à leur participation hebdomadaire au traitement. Conclusion : Ce projet de maîtrise a permis de démontrer qu’une combinaison de renforcement des MPP et de RRV est une approche de traitement acceptable pour les femmes âgées souffrant d’IUM puisqu’elles sont capables de suivre les exigences de ce programme d’entraînement.

Integração do elemento vegetal no edificado

Esta dissertação é muito mais do que um trabalho sobre ajardinamento de edifícios, pois pretende-se que seja uma ferramenta de ajuda em benefício da transformação de milhares de metros quadrados, das coberturas e das fachadas, muitas vezes negligenciados, em espaços verdes públicos beneficiando, não somente, os promotores imobiliários mas, acima de tudo os seus ocupantes e usufrutuários, que desta forma podem tirar partido de novos espaços de estar, ao ar livre. Enquanto a cidade, ao nível térreo, apresenta um aumento de tráfego rodoviário, com inegável poluição e confusão, é possível, desta forma, usufruir de um espaço aberto. Esta dissertação procura estudar os edifícios que já integram o elemento vegetal no seu conceito e que sejam um sucesso na forma como usam o elemento vegetal, de modo a divulgar as técnicas de construção, entender as decisões dos projectistas e o impacto nos utilizadores que habitam e usufruem dos espaços – Análise-Pós-Ocupação. Esta dissertação pretende demonstrar o quanto é importante que as cidades apresentem espaços verdes para uso da população. As zonas verdes existentes nas cidades são inequivocamente um importante indicador da qualidade ambiental existente nessas mesmas cidades. As coberturas e fachadas verdes arrefecem os edifícios, capturam e filtram as águas da chuva, proporcionam habitat à vida selvagem, reduzem o efeito de estufa das cidades, proporcionam uma preciosidade estética, uma experiência recreativa e por vezes comida, para os habitantes das cidades. Pretende-se assim, focar os benefícios humanos, sociais e naturais que se obtêm ao introduzir vegetação nas paredes, terraços, pátios e coberturas dos edifícios. Assim foi considerado útil usar uma metodologia para também analisar o tipo de espécies usadas nos estudos de caso, e determinar a necessidade de uma ferramenta para que a selecção vegetal respeite critérios de biodiversidade e melhor adaptabilidade aos ecossistemas locais, sabendo de antemão que não podem nem pretendem substituir os habitats naturais, que as cidades também podem e devem apresentar.

Putting the S-word back into Sustainability: Can we be more social?

In an era dominated by climate change debate and environmentalism there is a real danger that the important ‘social’ pillar of sustainability drops out of our vocabulary. This can happen at a variety of scales from business level through to building and neighbourhood level regeneration and development. Social sustainability should be at the heart of all housing and mixed-use development but for a variety of reasons tends to be frequently underplayed. The recent English city riots have brought this point back sharply into focus. The relationships between people, places and the local economy all matter and this is as true today as it was in the late 19th century when Patrick Geddes, the great pioneering town planner and ecologist, wrote of ‘place-work-folk’. This paper, commissioned from Tim Dixon, explains what is meant by social sustainability (and how it is linked to concepts such as social capital and social cohesion); why the debate matters during a period when ‘localism’ is dominating political debate; and what is inhibiting its growth and its measurement. The paper reviews best practice in post-occupancy social sustainability metric systems, based on recent research undertaken by the author on Dockside Green in Vancouver, and identifi es some of the key operational issues in mainstreaming the concept within major mixed-use projects. The paper concludes by offering a framework for the key challenges faced in setting strategic corporate goals and objectives; prioritising and selecting the most appropriate investments; and measuring social sustainability performance by identifying the required data sources