Formative and reflective measurement and validation mismatch in survey research : an archival analysis of information systems success constructs 1985-2007

The generic IS-success constructs first identified by DeLone and McLean (1992) continue to be widely employed in research. Yet, recent work by Petter et al (2007) has cast doubt on the validity of many mainstream constructs employed in IS research over the past 3 decades; critiquing the almost universal conceptualization and validation of these constructs as reflective when in many studies the measures appear to have been implicitly operationalized as formative. Cited examples of proper specification of the Delone and McLean constructs are few, particularly in light of their extensive employment in IS research. This paper introduces a four-stage formative construct development framework: Conceive > Operationalize > Respond > Validate (CORV). Employing the CORV framework in an archival analysis of research published in top outlets 1985-2007, the paper explores the extent of possible problems with past IS research due to potential misspecification of the four application-related success dimensions: Individual-Impact, Organizational-Impact, System-Quality and Information-Quality. Results suggest major concerns where there is a mismatch of the Respond and Validate stages. A general dearth of attention to the Operationalize and Respond stages in methodological writings is also observed.

The strategic and operational dimensions of staff training and professional development for information professionals : what neXus2 has revealed

The paper seeks to continue the debate about the need for professionals in the library and information services (LIS) sector to continually engage in career-long learning to sustain and develop their knowledge and skills in a dynamic industry. Aims: The neXus2 workforce study has been funded by the ALIA and the consortium of National and State Libraries Australasia (NSLA). It builds on earlier research work (the neXus census) that looked at the demographic, educational and career perspectives of individual library and information professions, to critically examine institutional policies and practices associated with the LIS workforce. The research aims to develop a clearer understanding of the issues impacting on workforce sustainability, workforce capability and workforce optimisation. Methods: The research methodology involved an extensive online survey conducted in March 2008 which collected data on organisational and general staffing; recruitment and retention; staff development and continuing professional education; and succession planning. Encouragement to participate was provided by key industry groups, including academic, public, health, law and government library and information agencies, with the result that around 150 institutions completed the questionnaire. Results: The paper will specifically discuss the research findings relating to training and professional development, to measure the scope and distribution of training activities across the workforce, to consider the interrelationship between the strategic and operational dimensions of staff development in individual institutions and to analyse the common and distinctive factors evident in the different sectors of the profession. Conclusion: The neXus2 project has successfully engaged LIS institutions in the collection of complex industry data that is relevant to the future education and workforce strategies for all areas of the profession. Cross-sector forums such as Information Online 2009 offer the opportunity for stimulating professional dialogue on the key issues.

Views of general practitioners and benzodiazepine users on benzodiazepines : a qualitative analysis

Effectively assisting benzodiazepine users to cease use requires a greater understanding of general practitioners’ (GPs)and benzodiazepine users’ views on using and ceasing benzodiazepines. This paper reports the findings from a qualitative study that examined the views of 28 GPs and 23 benzodiazepine users (BUs) in Cairns, Australia. A semistructured interview was conducted with all participants and the information gained was analysed using the Consensual Qualitative Research Approach, which allowed comparisons to be made between the views of the two groups of interviewees. There was commonality between GPs and BUs on reasons for commencing benzodiazepines, the role of dependence in continued use, and the importance of lifestyle change in its cessation. However, several differences emerged regarding commencement of use and processes of cessation. In particular, users felt there was greater need for GPs to routinely advise patients about non-pharmacological management of their problems and potential adverse consequences of long-term use before commencing benzodiazepines. Cessation could be discussed with all patients who use benzodiazepines for longer than 3 months, strategies offered to assist in management of withdrawal and anxiety, and referral to other health service providers for additional support. Lifestyle change could receive greater focus at all stages of treatment.

Mining Negative Relevance Feedback for Information Filtering

It is a big challenge to clearly identify the boundary between positive and negative streams. Several attempts have used negative feedback to solve this challenge; however, there are two issues for using negative relevance feedback to improve the effectiveness of information filtering. The first one is how to select constructive negative samples in order to reduce the space of negative documents. The second issue is how to decide noisy extracted features that should be updated based on the selected negative samples. This paper proposes a pattern mining based approach to select some offenders from the negative documents, where an offender can be used to reduce the side effects of noisy features. It also classifies extracted features (i.e., terms) into three categories: positive specific terms, general terms, and negative specific terms. In this way, multiple revising strategies can be used to update extracted features. An iterative learning algorithm is also proposed to implement this approach on RCV1, and substantial experiments show that the proposed approach achieves encouraging performance.

An effective model of using negative relevance feedback for information filtering

Over the years, people have often held the hypothesis that negative feedback should be very useful for largely improving the performance of information filtering systems; however, we have not obtained very effective models to support this hypothesis. This paper, proposes an effective model that use negative relevance feedback based on a pattern mining approach to improve extracted features. This study focuses on two main issues of using negative relevance feedback: the selection of constructive negative examples to reduce the space of negative examples; and the revision of existing features based on the selected negative examples. The former selects some offender documents, where offender documents are negative documents that are most likely to be classified in the positive group. The later groups the extracted features into three groups: the positive specific category, general category and negative specific category to easily update the weight. An iterative algorithm is also proposed to implement this approach on RCV1 data collections, and substantial experiments show that the proposed approach achieves encouraging performance.

A 5 year review of information requests to QISU

The Queensland Injury Surveillance Unit (QISU) has been collecting and analysing injury data in Queensland since 1988. QISU data is collected from participating emergency departments (EDs) in urban, rural and remote areas of Queensland. Using this data, QISU produces several injury bulletins per year on selected topics, providing a picture of Queensland injury, and setting this in the context of relevant local, national and international research and policy. These bulletins are used by numerous government and non-government groups to inform injury prevention and practice throughout the state. QISU bulletins are also used by local and state media to inform the general public of injury risk and prevention strategies. In addition to producing the bulletins, QISU regularly responds to requests for information from a variety of sources. These requests often require additional analysis of QISU data to tailor the response to the needs of the end user. This edition of the bulletin reviews 5 years of information requests to QISU.

Evaluating information systems

The book within which this chapter appears is published as a research reference book (not a coursework textbook) on Management Information Systems (MIS) for seniors or graduate students in Chinese universities. It is hoped that this chapter, along with the others, will be helpful to MIS scholars and PhD/Masters research students in China who seek understanding of several central Information Systems (IS) research topics and related issues. The subject of this chapter - ‘Evaluating Information Systems’ - is broad, and cannot be addressed in its entirety in any depth within a single book chapter. The chapter proceeds from the truism that organizations have limited resources and those resources need to be invested in a way that provides greatest benefit to the organization. IT expenditure represents a substantial portion of any organization’s investment budget and IT related innovations have broad organizational impacts. Evaluation of the impact of this major investment is essential to justify this expenditure both pre- and post-investment. Evaluation is also important to prioritize possible improvements. The chapter (and most of the literature reviewed herein) admittedly assumes a blackbox view of IS/IT1, emphasizing measures of its consequences (e.g. for organizational performance or the economy) or perceptions of its quality from a user perspective. This reflects the MIS emphasis – a ‘management’ emphasis rather than a software engineering emphasis2, where a software engineering emphasis might be on the technical characteristics and technical performance. Though a black-box approach limits diagnostic specificity of findings from a technical perspective, it offers many benefits. In addition to superior management information, these benefits may include economy of measurement and comparability of findings (e.g. see Part 4 on Benchmarking IS). The chapter does not purport to be a comprehensive treatment of the relevant literature. It does, however, reflect many of the more influential works, and a representative range of important writings in the area. The author has been somewhat opportunistic in Part 2, employing a single journal – The Journal of Strategic Information Systems – to derive a classification of literature in the broader domain. Nonetheless, the arguments for this approach are believed to be sound, and the value from this exercise real. The chapter drills down from the general to the specific. It commences with a highlevel overview of the general topic area. This is achieved in 2 parts: - Part 1 addressing existing research in the more comprehensive IS research outlets (e.g. MISQ, JAIS, ISR, JMIS, ICIS), and Part 2 addressing existing research in a key specialist outlet (i.e. Journal of Strategic Information Systems). Subsequently, in Part 3, the chapter narrows to focus on the sub-topic ‘Information Systems Success Measurement’; then drilling deeper to become even more focused in Part 4 on ‘Benchmarking Information Systems’. In other words, the chapter drills down from Parts 1&2 Value of IS, to Part 3 Measuring Information Systems Success, to Part 4 Benchmarking IS. While the commencing Parts (1&2) are by definition broadly relevant to the chapter topic, the subsequent, more focused Parts (3 and 4) admittedly reflect the author’s more specific interests. Thus, the three chapter foci – value of IS, measuring IS success, and benchmarking IS - are not mutually exclusive, but, rather, each subsequent focus is in most respects a sub-set of the former. Parts 1&2, ‘the Value of IS’, take a broad view, with much emphasis on ‘the business Value of IS’, or the relationship between information technology and organizational performance. Part 3, ‘Information System Success Measurement’, focuses more specifically on measures and constructs employed in empirical research into the drivers of IS success (ISS). (DeLone and McLean 1992) inventoried and rationalized disparate prior measures of ISS into 6 constructs – System Quality, Information Quality, Individual Impact, Organizational Impact, Satisfaction and Use (later suggesting a 7th construct – Service Quality (DeLone and McLean 2003)). These 6 constructs have been used extensively, individually or in some combination, as the dependent variable in research seeking to better understand the important antecedents or drivers of IS Success. Part 3 reviews this body of work. Part 4, ‘Benchmarking Information Systems’, drills deeper again, focusing more specifically on a measure of the IS that can be used as a ‘benchmark’3. This section consolidates and extends the work of the author and his colleagues4 to derive a robust, validated IS-Impact measurement model for benchmarking contemporary Information Systems (IS). Though IS-Impact, like ISS, has potential value in empirical, causal research, its design and validation has emphasized its role and value as a comparator; a measure that is simple, robust and generalizable and which yields results that are as far as possible comparable across time, across stakeholders, and across differing systems and systems contexts.

A pilot randomised control trial of general practitioner and exercise physiologist interventions to reduce primary cardiovascular risk in overweight patients [Abstract]

Aims & Rationale/Objectives: With the knowledge that overweight is a major public health concern in Australia, that a multidisciplinary team approach to the management of lifestyle-related conditions is supported, and that the Australian Government recently recognised the role of the exercise physiologist (EP) in reducing the health burden of disease by their inclusion for reimbursement under the Medicare Plus scheme, this study sought to undertake a pilot RCT to compare GP and EP interventions to reduce primary cardiovascular risk in the overweight general practice population. Methods and Measures: Overweight patients recruited by a convenience sample of GPs were randomised into one of three arms: the control group, or the GP or EP intervention group (in which patients received either five GP or five EP consultations over 24 weeks). Patients had baseline, 12- and 24-week measures of body composition and cardio-respiratory fitness, and completed baseline and end-of-study surveys, fasting lipids and glucose. GPs and EPs completed an end-of-study survey. Results:Sixty-seven patients attended the baseline assessment. Overall retention rate was 67%. Patients were generally satisfied with the effectiveness of the interventions and their weight reduction. Favourable trends in BMI, weight, glucose and exercise levels for GP and EP intervention groups and in physical activity levels for all groups Conclusions: This study supports the feasibility of a RCT of GP and EP interventions for decreasing primary cardiovascular risk in the overweight general practice population.

Adaptive dual control of topic-based information retrieval

Information Retrieval is an important albeit imperfect component of information technologies. A problem of insufficient diversity of retrieved documents is one of the primary issues studied in this research. This study shows that this problem leads to a decrease of precision and recall, traditional measures of information retrieval effectiveness. This thesis presents an adaptive IR system based on the theory of adaptive dual control. The aim of the approach is the optimization of retrieval precision after all feedback has been issued. This is done by increasing the diversity of retrieved documents. This study shows that the value of recall reflects this diversity. The Probability Ranking Principle is viewed in the literature as the “bedrock” of current probabilistic Information Retrieval theory. Neither the proposed approach nor other methods of diversification of retrieved documents from the literature conform to this principle. This study shows by counterexample that the Probability Ranking Principle does not in general lead to optimal precision in a search session with feedback (for which it may not have been designed but is actively used). Retrieval precision of the search session should be optimized with a multistage stochastic programming model to accomplish the aim. However, such models are computationally intractable. Therefore, approximate linear multistage stochastic programming models are derived in this study, where the multistage improvement of the probability distribution is modelled using the proposed feedback correctness method. The proposed optimization models are based on several assumptions, starting with the assumption that Information Retrieval is conducted in units of topics. The use of clusters is the primary reasons why a new method of probability estimation is proposed. The adaptive dual control of topic-based IR system was evaluated in a series of experiments conducted on the Reuters, Wikipedia and TREC collections of documents. The Wikipedia experiment revealed that the dual control feedback mechanism improves precision and S-recall when all the underlying assumptions are satisfied. In the TREC experiment, this feedback mechanism was compared to a state-of-the-art adaptive IR system based on BM-25 term weighting and the Rocchio relevance feedback algorithm. The baseline system exhibited better effectiveness than the cluster-based optimization model of ADTIR. The main reason for this was insufficient quality of the generated clusters in the TREC collection that violated the underlying assumption.

Family-focused weight management program for five- to nine-year-olds incorporating parenting skills training with healthy lifestyle information to support behaviour modification

his case study aims to describe how general parenting principles can be used as part of parent-led, family-focused child weight management that is in line with current Australian Clinical Practice Guidelines. A parent-led, family-focused child weight management program was designed for use by dietitians with parents of young children (five- to nine-year-olds). The program utilises the cornerstones of overweight treatment: diet, activity, behaviour modification and family support delivered in an age-appropriate, family-focused manner. Parents participate in 16 sessions (4 parenting-focused, 8 lifestyle-focused and 4 individual telephone support calls) conducted weekly, fortnightly then monthly over six months. This case study illustrates how a family used the program, resulting in reduced degree of overweight and stabilised waist circumference in the child over 12 months. In conclusion, linking parenting skills to healthy family lifestyle education provides an innovative approach to family-focused child weight management. It addresses key Australian Clinical Practice Guidelines, works at the family level, and provides a means for dietitians to easily adopt age-appropriate behaviour modification as part of their practice.

Information sharing in the 21st century : progress and challenges

With the increasing threat of cyber and other attacks on critical infrastructure, governments throughout the world have been organizing industry to share information on possible threats. In Australia the Office of the Attorney General has formed Trusted Information Sharing Networks (TISN) for the various critical industries such as banking and electricity. Currently the majority of information for a TISN is shared at physical meetings. To meet cyber threats there are clearly limitations to physical meetings. Many of these limitations can be overcome by the creation of a virtual information sharing network (VISN). However there are many challenges to overcome in the design of a VISN both from a policy and technical viewpoint. We shall discuss some of these challenges in this talk.

Limited information estimation and evaluation of DSGE models

We advance the proposition that dynamic stochastic general equilibrium (DSGE) models should not only be estimated and evaluated with full information methods. These require that the complete system of equations be specified properly. Some limited information analysis, which focuses upon specific equations, is therefore likely to be a useful complement to full system analysis. Two major problems occur when implementing limited information methods. These are the presence of forward-looking expectations in the system as well as unobservable non-stationary variables. We present methods for dealing with both of these difficulties, and illustrate the interaction between full and limited information methods using a well-known model.

A pattern mining approach for information filtering systems

It is a big challenge to clearly identify the boundary between positive and negative streams for information filtering systems. Several attempts have used negative feedback to solve this challenge; however, there are two issues for using negative relevance feedback to improve the effectiveness of information filtering. The first one is how to select constructive negative samples in order to reduce the space of negative documents. The second issue is how to decide noisy extracted features that should be updated based on the selected negative samples. This paper proposes a pattern mining based approach to select some offenders from the negative documents, where an offender can be used to reduce the side effects of noisy features. It also classifies extracted features (i.e., terms) into three categories: positive specific terms, general terms, and negative specific terms. In this way, multiple revising strategies can be used to update extracted features. An iterative learning algorithm is also proposed to implement this approach on the RCV1 data collection, and substantial experiments show that the proposed approach achieves encouraging performance and the performance is also consistent for adaptive filtering as well.

Body-site distribution of skin cancer, pre-malignant and common benign pigmented lesions excised in general practice

Background Concern about skin cancer is a common reason for people from predominantly fair-skinned populations to present to primary care doctors. Objectives To examine the frequency and body-site distribution of malignant, pre-malignant and benign pigmented skin lesions excised in primary care. Methods This prospective study conducted in Queensland, Australia, included 154 primary care doctors. For all excised or biopsied lesions, doctors recorded the patient's age and sex, body site, level of patient pressure to excise, and the clinical diagnosis. Histological confirmation was obtained through pathology laboratories. Results Of 9650 skin lesions, 57·7% were excised in males and 75·0% excised in patients ≥50years. The most common diagnoses were basal cell carcinoma (BCC) (35·1%) and squamous cell carcinoma (SCC) (19·7%). Compared with the whole body, the highest densities for SCC, BCC and actinic keratoses were observed on chronically sun-exposed areas of the body including the face in males and females, the scalp and ears in males, and the hands in females. The density of BCC was also high on intermittently or rarely exposed body sites. Females, younger patients and patients with melanocytic naevi were significantly more likely to exert moderate/high levels of pressure on the doctor to excise. Conclusions More than half the excised lesions were skin cancer, which mostly occurred on the more chronically sun-exposed areas of the body. Information on the type and body-site distribution of skin lesions can aid in the diagnosis and planned management of skin cancer and other skin lesions commonly presented in primary care.

Functional fixation : experimental evidence on the presentation of financial information through different digital formats

This study examines whether, in the presentation of financial information, digital formats address the concern over users’ functional fixation. The accounting literature indicates that the presentation of financial information either within the financial statements or in the notes to the financial statements often creates functional fixation where users of financial statements fail to adjust for differences in accounting policy. This leads users to judge what would otherwise be identical financial situations as being different due to the different accounting policies and methods adopted. It has been suggested that the use of digital formats in presenting financial reports may overcome functional fixation. Using an experimental design involving accountants in public practice, the results indicate that the use of digital formats to present financial reports does not fully overcome the issue of functional fixation in the processing of financial information. Although the participants were able to identify and extract relevant information, irrespective of whether or not the information was presented within the financial statements or in the notes to the accounts, the evidence indicates that functional fixation remained when the participants made final decisions based on available information. This suggests that functional fixation may not be caused by access to or extraction of information but by the level of perceived significance based on where the information is reported in the financial statements. In general, the results indicate that current technology may not be able to fully reduce functional fixation in the evaluation of financial information prepared in accordance with different accounting policies and methods.