Evaluating information systems

The book within which this chapter appears is published as a research reference book (not a coursework textbook) on Management Information Systems (MIS) for seniors or graduate students in Chinese universities. It is hoped that this chapter, along with the others, will be helpful to MIS scholars and PhD/Masters research students in China who seek understanding of several central Information Systems (IS) research topics and related issues. The subject of this chapter - ‘Evaluating Information Systems’ - is broad, and cannot be addressed in its entirety in any depth within a single book chapter. The chapter proceeds from the truism that organizations have limited resources and those resources need to be invested in a way that provides greatest benefit to the organization. IT expenditure represents a substantial portion of any organization’s investment budget and IT related innovations have broad organizational impacts. Evaluation of the impact of this major investment is essential to justify this expenditure both pre- and post-investment. Evaluation is also important to prioritize possible improvements. The chapter (and most of the literature reviewed herein) admittedly assumes a blackbox view of IS/IT1, emphasizing measures of its consequences (e.g. for organizational performance or the economy) or perceptions of its quality from a user perspective. This reflects the MIS emphasis – a ‘management’ emphasis rather than a software engineering emphasis2, where a software engineering emphasis might be on the technical characteristics and technical performance. Though a black-box approach limits diagnostic specificity of findings from a technical perspective, it offers many benefits. In addition to superior management information, these benefits may include economy of measurement and comparability of findings (e.g. see Part 4 on Benchmarking IS). The chapter does not purport to be a comprehensive treatment of the relevant literature. It does, however, reflect many of the more influential works, and a representative range of important writings in the area. The author has been somewhat opportunistic in Part 2, employing a single journal – The Journal of Strategic Information Systems – to derive a classification of literature in the broader domain. Nonetheless, the arguments for this approach are believed to be sound, and the value from this exercise real. The chapter drills down from the general to the specific. It commences with a highlevel overview of the general topic area. This is achieved in 2 parts: - Part 1 addressing existing research in the more comprehensive IS research outlets (e.g. MISQ, JAIS, ISR, JMIS, ICIS), and Part 2 addressing existing research in a key specialist outlet (i.e. Journal of Strategic Information Systems). Subsequently, in Part 3, the chapter narrows to focus on the sub-topic ‘Information Systems Success Measurement’; then drilling deeper to become even more focused in Part 4 on ‘Benchmarking Information Systems’. In other words, the chapter drills down from Parts 1&2 Value of IS, to Part 3 Measuring Information Systems Success, to Part 4 Benchmarking IS. While the commencing Parts (1&2) are by definition broadly relevant to the chapter topic, the subsequent, more focused Parts (3 and 4) admittedly reflect the author’s more specific interests. Thus, the three chapter foci – value of IS, measuring IS success, and benchmarking IS - are not mutually exclusive, but, rather, each subsequent focus is in most respects a sub-set of the former. Parts 1&2, ‘the Value of IS’, take a broad view, with much emphasis on ‘the business Value of IS’, or the relationship between information technology and organizational performance. Part 3, ‘Information System Success Measurement’, focuses more specifically on measures and constructs employed in empirical research into the drivers of IS success (ISS). (DeLone and McLean 1992) inventoried and rationalized disparate prior measures of ISS into 6 constructs – System Quality, Information Quality, Individual Impact, Organizational Impact, Satisfaction and Use (later suggesting a 7th construct – Service Quality (DeLone and McLean 2003)). These 6 constructs have been used extensively, individually or in some combination, as the dependent variable in research seeking to better understand the important antecedents or drivers of IS Success. Part 3 reviews this body of work. Part 4, ‘Benchmarking Information Systems’, drills deeper again, focusing more specifically on a measure of the IS that can be used as a ‘benchmark’3. This section consolidates and extends the work of the author and his colleagues4 to derive a robust, validated IS-Impact measurement model for benchmarking contemporary Information Systems (IS). Though IS-Impact, like ISS, has potential value in empirical, causal research, its design and validation has emphasized its role and value as a comparator; a measure that is simple, robust and generalizable and which yields results that are as far as possible comparable across time, across stakeholders, and across differing systems and systems contexts.

Part-time work of high school students and impact on educational outcomes

While increasing numbers of young high school students engage in part-time work, there is no consensus about its impact on educational outcomes. Indeed this field has had a dearth of research. The present paper presents a review of recent research, primarily from Australia and the US, although it is acknowledged that there are considerable contextual differences. Suggestions for school counsellors to harness the students’ experiences to assist in educational and career decision-making are presented.

Inequalities in cardiovascular disease mortality : the role of behavioural, physiological and social risk factors

Background: While the relationship between socioeconomic disadvantage and cardiovascular disease (CVD) is well established, the role that traditional cardiovascular risk factors play in this association remains unclear. We examined the association between education attainment and CVD mortality and the extent to which behavioural, social and physiological factors explained this relationship. Methods: Adults (n=38 355) aged 40-69 years living in Melbourne, Australia were recruited in 1990-1994. Subjects with baseline CVD risk factor data ascertained through questionnaire and physical measurement were followed for an average of 9.4 years with CVD deaths verified by review of medical records and autopsy reports. Results: CVD mortality was higher for those with primary education only compared to those who had completed tertiary education, with a hazard ratio (HR) of 1.66 (95% confidence interval [CI] 1.11-2.49) after adjustment for age, country of birth and gender. Those from the lowest educated group had a more adverse cardiovascular risk factor profile compared to the highest educated group, and adjustment for these risk factors reduced the HR to 1.18 (95% CI 0.78-1.77). In analysis of individual risk factors, smoking and waist circumference explained most of the difference in CVD mortality between the highest and lowest education groups. Conclusions: Most of the excess CVD mortality in lower socioeconomic groups can be explained by known risk factors, particularly smoking and overweight. While targeting cardiovascular risk factors should not divert efforts from addressing the underlying determinants of health inequalities, it is essential that known risk factors are addressed effectively among lower socioeconomic groups.

New insights into appetite, inflammation and the use of fish oil in hemodialysis patients

Protein-energy wasting (PEW) is commonly seen in patients with chronic kidney disease (CKD). The condition is characterised by chronic, systemic low-grade inflammation which affects nutritional status by a variety of mechanisms including reducing appetite and food intake and increasing muscle catabolism. PEW is linked with co-morbidities such as cardiovascular disease, and is associated with lower quality of life, increased hospitalisations and a 6-fold increase in risk of death1. Significant gender differences have been found in the severity and effects of several markers of PEW. There have been limited studies testing the ability of anti-inflammatory agents or nutritional interventions to reduce the effects of PEW in dialysis patients. This thesis makes a significant contribution to the understanding of PEW in dialysis patients. It advances understanding of measurement techniques for two of the key components, appetite and inflammation, and explores the effect of fish oil, an anti-inflammatory agent, on markers of PEW in dialysis patients. The first part of the thesis consists of two methodological studies conducted using baseline data. The first study aims to validate retrospective ratings of hunger, desire to eat and fullness on visual analog scales (VAS) (paper and pen and electronic) as a new method of measuring appetite in dialysis patients. The second methodological study aims to assess the ability of a variety of methods available in routine practice to detect the presence of inflammation. The second part of the thesis aims to explore the effect of 12 weeks supplementation with 2g per day of Eicosapentaenoic Acid (EPA), a longchain fatty acid found in fish oil, on markers of PEW. A combination of biomarkers and psychomarkers of appetite and inflammation are the main outcomes being explored, with nutritional status, dietary intake and quality of life included as secondary outcomes. A lead in phase of 3 months prior to baseline was used so that each person acts as their own historical control. The study also examines whether there are gender differences in response to the treatment. Being an exploratory study, an important part of the work is to test the feasibility of the intervention, thus the level of adherence and factors associated with adherence are also presented. The studies were conducted at the hemodialysis unit of the Wesley Hospital. Participants met the following criteria: adult, stage 5 CKD on hemodialysis for at least 3 months, not expected to receive a transplant or switch to another dialysis modality during the study, absence of intellectual impairment or mental illness impairing ability to follow instructions or complete the intervention. A range of intermediate, clinical and patient-centred outcome measures were collected at baseline and 12 weeks. Inflammation was measured using five biomarkers: c-reactive protein (CRP), interleukin-6 (IL6), intercellular adhesion molecule (sICAM-1), vascular cell adhesion molecule (sVCAM-1) and white cell count (WCC). Subjective appetite was measured using the first question from the Appetite and Dietary Assessment (ADAT) tool and VAS for measurements of hunger, desire to eat and fullness. A novel feature of the study was the assessment of the appetite peptides leptin, ghrelin and peptide YY as biomarkers of appetite. Nutritional status/inflammation was assessed using the Malnutrition-Inflammation Score (MIS) and the Patient-Generated Subjective Global Assessment (PG-SGA). Dietary intake was measured using 3-day records. Quality of life was measured using the Kidney Disease Quality of Life Short Form version 1.3 (KDQOL-SF™ v1.3 © RAND University), which combines the Short-Form 36 (SF36) with a kidney-disease specific module2. A smaller range of these variables was available for analysis during the control phase (CRP, ADAT, dietary intake and nutritional status). Statistical analysis was carried out using SPSS version 14 (SPSS Inc, Chicago IL, USA). Analysis of the first part of the thesis involved descriptive and bivariate statistics, as well as Bland-Altman plots to assess agreement between methods, and sensitivity analysis/ROC curves to test the ability of methods to predict the presence of inflammation. The unadjusted (paired ttests) and adjusted (linear mixed model) change over time is presented for the main outcome variables of inflammation and appetite. Results are shown for the whole group followed by analyses according to gender and adherence to treatment. Due to the exploratory nature of the study, trends and clinical significance were considered as important as statistical significance. Twenty-eight patients (mean age 61±17y, 50% male, dialysis vintage 19.5 (4- 101) months) underwent baseline assessment. Seven out of 28 patients (25%) reported sub-optimal appetite (self-reported as fair, poor or very poor) despite all being well nourished (100% SGA A). Using the VAS, ratings of hunger, but not desire to eat or fullness, were significantly (p<0.05) associated with a range of relevant clinical variables including age (r=-0.376), comorbidities (r=-0.380) nutritional status (PG-SGA score, r=-0.451), inflammatory markers (CRP r=-0.383; sICAM-1 r=-0.387) and seven domains of quality of life. Patients expressed a preference for the paper and pen method of administering VAS. None of the tools (appetite, MIS, PG-SGA, albumin or iron) showed an acceptable ability to detect patients who are inflamed. It is recommended that CRP should be tested more frequently as a matter of course rather than seeking alternative methods of measuring inflammation. 27 patients completed the 12 week intervention. 20 patients were considered adherent based on changes in % plasma EPA, which rose from 1.3 (0.94)% to 5.2 (1.1)%, p<0.001, in this group. The major barriers to adherence were forgetting to take the tablets as well as their size. At 12 weeks, inflammatory markers remained steady apart from the white cell count which decreased (7.6(2.5) vs 7.0(2.2) x109/L, p=0.058) and sVCAM-1 which increased (1685(654) vs 2249(925) ng/mL, p=0.001). Subjective appetite using VAS increased (51mm to 57mm, +12%) and there was a trend towards reduction in peptide YY (660(31) vs 600(30) pg/mL, p=0.078). There were some gender differences apparent, with the following adjusted change between baseline and week 12: CRP (males -3% vs females +17%, p=0.19), IL6 (males +17% vs females +48%, p=0.77), sICAM-1 (males -5% vs females +11%, p=0.07), sVCAM-1 (males +54% vs females +19%, p=0.08) and hunger ratings (males 20% vs females -5%, p=0.18). On balance, males experienced a maintainence or reduction in three inflammatory markers and an improvement in hunger ratings, and therefore appeared to have responded better to the intervention. Compared to those who didn’t adhere, adherent patients maintained weight (mean(SE) change: +0.5(1.6) vs - 0.8(1.2) kg, p=0.052) and fat-free mass (-0.1 (1.6) vs -1.8 (1.8) kg, p=0.045). There was no difference in change between the intervention and control phase for CRP, appetite, nutritional status or dietary intake. The thesis makes a significant contribution to the evidence base for understanding of PEW in dialysis patients. It has advanced knowledge of methods of assessing inflammation and appetite. Retrospective ratings of hunger on a VAS appear to be a valid method of assessing appetite although samples which include patients with very poor appetite are required to confirm this. Supplementation with fish oil appeared to improve subjective appetite and dampen the inflammatory response. The effectiveness of the intervention is influenced by gender and adherence. Males appear to be more responsive to the primary outcome variables than females, and the quality of response is improved with better adherence. These results provide evidence to support future interventions aimed at reducing the effects of PEW in dialysis patients.

The relationship between communication and team performance : testing moderators and identifying communication profiles in established work teams

Communication is one team process factor that has received considerable research attention in the team literature. This literature provides equivocal evidence regarding the role of communication in team performance and yet, does not provide any evidence for when communication becomes important for team performance. This research program sought to address this evidence gap by a) testing task complexity and team member diversity (race diversity, gender diversity and work value diversity) as moderators of the team communication — performance relationship; and b) testing a team communication — performance model using established teams across two different task types. The functional perspective was used as the theoretical framework for operationalizing team communication activity. The research program utilised a quasi-experimental research design with participants from a large multi-national information technology company whose Head Office was based in Sydney, Australia. Participants voluntarily completed two team building exercises (a decision making and production task), and completed two online questionnaires. In total, data were collected from 1039 individuals who constituted 203 work teams. Analysis of the data revealed a small number of significant moderation effects, not all in the expected direction. However, an interesting and unexpected finding also emerged from Study One. Large and significant correlations between communication activity ratings were found across tasks, but not within tasks. This finding suggested that teams were displaying very similar profiles of communication on each task, despite the tasks having different communication requirements. Given this finding, Study Two sought to a) determine the relative importance of task versus team effects in explaining variance in team communication measures for established teams; b) determine if established teams had reliable and discernable team communication profiles and if so, c) investigate whether team communication profiles related to task performance. Multi-level modeling and repeated measures analysis of variance (ANOVA) revealed that task type did not have an effect on team communication ratings. However, teams accounted for 24% of the total variance in communication measures. Through cluster analysis, five reliable and distinct team communication profiles were identified. Consistent with the findings of the multi-level analysis and repeated measures ANOVA, teams’ profiles were virtually identical across the decision making and production tasks. A relationship between communication profile and performance was identified for the production task, although not for the decision making task. This research responds to calls in the literature for a better understanding of when communication becomes important for team performance. The moderators tested in this research were not found to have a substantive or reliable effect on the relationship between communication and performance. However, the consistency in team communication activity suggests that established teams can be characterized by their communication profiles and further, that these communication profiles may have implications for team performance. The findings of this research provide theoretical support for the functional perspective in terms of the communication – performance relationship and further support the team development literature as an explanation for the stability in team communication profiles. This research can also assist organizations to better understand the specific types of communication activity and profiles of communication that could offer teams a performance advantage.

Real time detectionof airborne fungal spores and investigations into their dynamics in indoor air

Concern regarding the health effects of indoor air quality has grown in recent years, due to the increased prevalence of many diseases, as well as the fact that many people now spend most of their time indoors. While numerous studies have reported on the dynamics of aerosols indoors, the dynamics of bioaerosols in indoor environments are still poorly understood and very few studies have focused on fungal spore dynamics in indoor environments. Consequently, this work investigated the dynamics of fungal spores in indoor air, including fungal spore release and deposition, as well as investigating the mechanisms involved in the fungal spore fragmentation process. In relation to the investigation of fungal spore dynamics, it was found that the deposition rates of the bioaerosols (fungal propagules) were in the same range as the deposition rates of nonbiological particles and that they were a function of their aerodynamic diameters. It was also found that fungal particle deposition rates increased with increasing ventilation rates. These results (which are reported for the first time) are important for developing an understanding of the dynamics of fungal spores in the air. In relation to the process of fungal spore fragmentation, important information was generated concerning the airborne dynamics of the spores, as well as the part/s of the fungi which undergo fragmentation. The results obtained from these investigations into the dynamics of fungal propagules in indoor air significantly advance knowledge about the fate of fungal propagules in indoor air, as well as their deposition in the respiratory tract. The need to develop an advanced, real-time method for monitoring bioaerosols has become increasingly important in recent years, particularly as a result of the increased threat from biological weapons and bioterrorism. However, to date, the Ultraviolet Aerodynamic Particle Sizer (UVAPS, Model 3312, TSI, St Paul, MN) is the only commercially available instrument capable of monitoring and measuring viable airborne micro-organisms in real-time. Therefore (for the first time), this work also investigated the ability of the UVAPS to measure and characterise fungal spores in indoor air. The UVAPS was found to be sufficiently sensitive for detecting and measuring fungal propagules. Based on fungal spore size distributions, together with fluorescent percentages and intensities, it was also found to be capable of discriminating between two fungal spore species, under controlled laboratory conditions. In the field, however, it would not be possible to use the UVAPS to differentiate between different fungal spore species because the different micro-organisms present in the air may not only vary in age, but may have also been subjected to different environmental conditions. In addition, while the real-time UVAPS was found to be a good tool for the investigation of fungal particles under controlled conditions, it was not found to be selective for bioaerosols only (as per design specifications). In conclusion, the UVAPS is not recommended for use in the direct measurement of airborne viable bioaerosols in the field, including fungal particles, and further investigations into the nature of the micro-organisms, the UVAPS itself and/or its use in conjunction with other conventional biosamplers, are necessary in order to obtain more realistic results. Overall, the results obtained from this work on airborne fungal particle dynamics will contribute towards improving the detection capabilities of the UVAPS, so that it is capable of selectively monitoring and measuring bioaerosols, for which it was originally designed. This work will assist in finding and/or improving other technologies capable of the real-time monitoring of bioaerosols. The knowledge obtained from this work will also be of benefit in various other bioaerosol applications, such as understanding the transport of bioaerosols indoors.

A novel system for the measurement of body shape and construction of seating supports for patients

Patients with severe back deformities can greatly benefit from customized medical seating. Customized medical seating is made by taking measurements of each individual patient and making the seat as per these measurements. The current measuring systems employed by the industry are limited to use in clinics which are generally located only in major population centres. Patients living in remote areas are severely affected by this as the clinics could be far away and inaccessible for these patients. To provide service of customized medical seating requires a new measurement system which is portable so that the system could be transported to the patients in remote areas. The requirements for a new measurement system are analysed to suite the needs of Equipment Technology Services of the Cerebral Palsy League of Queensland. Design for a new measurement system was conceptualised by reviewing systems and technologies in various scientific disciplines. Design for a new system was finalised by optimizing each individual component. The final approach was validated by measuring difficult models and repeating the process to check for process variances. This system has now been adopted for clinical evaluation by ETS Suggestions have been made for further improvements in this new measurement approach.

Assessment of architectural work experience by employers and students

A major element in the architectural work experience program at Queensland University of Technology is the assessment reports provided by students and employers. This paper gives an analysis of assessments submitted during the period 2000 to 2007 as viewed from a practice-base perspective. By comparing the 398 student assessments with 403 employer assessments in five specific categories over an eight year period one is able to obtain a clear understanding of the performance of the program and the relevance of its various sections for its participants that is not always obvious in a yearly analysis. In the major work experience areas there is close agreement between the student and employer assessments. However, the analysis did highlight a misunderstanding of the program’s aims by some participants. Overall the students were very positive about the program and appreciated the opportunity to work on real projects and be given a degree of responsibility for these projects. For Work Integrated Learning (WIL) practitioners this study clearly demonstrates the value of obtaining assessments from students and employers in order to establish the acceptance of a WIL program.

A practical example of a siphon at work

In this paper, some classroom experiments are described for correcting the common misconception that the operation of a siphon depends on atmospheric pressure. One experiment makes use of a chain model of a siphon and another demonstrates that flow rate is dependent on the height difference between the inflow and outflow of a siphon and not atmospheric pressure. A real-life example of the use of a siphon to refill a lake in South Australia is described, demonstrating that the siphon is not only of academic interest but has practical applications.

Ultrasonic measurement of fine head movements in a standard ophthalmic headrest

We aimed to investigate the naturally occurring horizontal plane movements of a head stabilized in a standard ophthalmic headrest and to analyze their magnitude, velocity, spectral characteristics, and correlation to the cardio pulmonary system. Two custom-made air-coupled highly accurate (±2 μm)ultrasound transducers were used to measure the displacements of the head in different horizontal directions with a sampling frequency of 100 Hz. Synchronously to the head movements, an electrocardiogram (ECG) signal was recorded. Three healthy subjects participated in the study. Frequency analysis of the recorded head movements and their velocities was carried out, and functions of coherence between the two displacements and the ECG signal were calculated. Frequency of respiration and the heartbeat were clearly visible in all recorded head movements. The amplitude of head displacements was typically in the range of ±100 μm. The first harmonic of the heartbeat (in the range of 2–3 Hz), rather than its principal frequency, was found to be the dominant frequency of both head movements and their velocities. Coherence analysis showed high interdependence between the considered signals for frequencies of up to 20 Hz. These findings may contribute to the design of better ophthalmic headrests and should help other studies in the decision making of whether to use a heavy headrest or a bite bar.