A critical evaluation and comparison of two formative measures of system quality using criterion variables

Evaluating the validity of formative variables has presented ongoing challenges for researchers. In this paper we use global criterion measures to compare and critically evaluate two alternative formative measures of System Quality. One model is based on the ISO-9126 software quality standard, and the other is based on a leading information systems research model. We find that despite both models having a strong provenance, many of the items appear to be non-significant in our study. We examine the implications of this by evaluating the quality of the criterion variables we used, and the performance of PLS when evaluating formative models with a large number of items. We find that our respondents had difficulty distinguishing between global criterion variables measuring different aspects of overall System Quality. Also, because formative indicators “compete with one another” in PLS, it may be difficult to develop a set of measures which are all significant for a complex formative construct with a broad scope and a large number of items. Overall, we suggest that there is cautious evidence that both sets of measures are valid and largely equivalent, although questions still remain about the measures, the use of criterion variables, and the use of PLS for this type of model evaluation.

A critical evaluation and comparison of two formative measures of system quality using criterion variables

Evaluating the validity of formative variables has presented ongoing challenges for researchers. In this paper we use global criterion measures to compare and critically evaluate two alternative formative measures of System Quality. One model is based on the ISO-9126 software quality standard, and the other is based on a leading information systems research model. We find that despite both models having a strong provenance, many of the items appear to be non-significant in our study. We examine the implications of this by evaluating the quality of the criterion variables we used, and the performance of PLS when evaluating formative models with a large number of items. We find that our respondents had difficulty distinguishing between global criterion variables measuring different aspects of overall System Quality. Also, because formative indicators “compete with one another” in PLS, it may be difficult to develop a set of measures which are all significant for a complex formative construct with a broad scope and a large number of items. Overall, we suggest that there is cautious evidence that both sets of measures are valid and largely equivalent, although questions still remain about the measures, the use of criterion variables, and the use of PLS for this type of model evaluation.

Information sharing with key suppliers : a transaction cost theory perspective

Purpose: The purpose of this paper is to explain variations in discretionary information shared between buyers and key suppliers. The paper also aims to examine how the extent of information shared affects buyers’ performance in terms of resource usage, output, and flexibility. ----- ----- Design/methodology/approach: The data for the paper comprise 221 Finnish and Swedish non-service companies obtained through a mail survey. The hypothesized relationships were tested using partial least squares modelling with reflective and formative constructs.----- ----- Findings: The results of the study suggest that (environmental and demand) uncertainty and interdependency can to some degree explain the extent of information shared between a buyer and key supplier. Furthermore, information sharing improves buyers’ performance with respect to resource usage, output, and flexibility.----- ----- Research limitations/implications: A limitation to the paper relates to the data, which only included buyers.Abetter approach would have been to collect data from both, buyers and key suppliers. Practical implications – Companies face a wide range of supply chain solutions that enable and encourage collaboration across organizations. This paper suggests a more selective and balanced approach toward adopting the solutions offered as the benefits are contingent on a number of factors such as uncertainty. Also, the risks of information sharing are far too high for a one size fits all approach.----- ----- Originality/value: The paper illustrates the applicability of transaction cost theory to the contemporary era of e-commerce. With this finding, transaction cost economics can provide a valuable lens with which to view and interpret interorganizational information sharing, a topic that has received much attention in the recent years.

Realizing benefits from enterprise architecture : a measurement model

Enterprise architecture (EA) management has become an intensively discussed approach to manage enterprise transformations. Despite the popularity and potential of EA, both researchers and practitioners lament a lack of knowledge about the realization of benefits from EA. To determine the benefits from EA, we explore the various dimensions of EA benefit realization and report on the development of a validated and robust measurement instrument. In this paper, we test the reliability and construct validity of the EA benefit realization model (EABRM), which we have designed based on the DeLone & McLean IS success model and findings from exploratory interviews. A confirmatory factor analysis confirms the existence of an impact of five distinct and individually important dimensions on the benefits derived from EA: EA artefact quality, EA infrastructure quality, EA service quality, EA culture, and EA use. The analysis presented in this paper shows that the EA benefit realization model is an instrument that demonstrates strong reliability and validity.

Measuring IS-support : a conceptual model

This paper reports on a conceptual model of a larger research effort proceeding from a central interest in the importance of assessing the IS-Support provided to key-user groups. This study conceptualised a new multidimensional IS-Support construct with four dimensions: training, documentation, assistance and authorisation, which form the overarching construct – IS-Support. We argue that a holistic measure for assessing IS-Support should consist of dimensions, and measures, that together assess the variety of the support provided to IS key-user groups. The proposed IS-Support construct is defined as the support the IS key-user groups receive to increase their capabilities in utilising information systems within the organisation. With two interrelated phases, conceptualisation phase and validation phase, to rigorously hypothesise and validate a measurement model, the IS-Support model, proposed in this study, is intended to include the characteristics of analytic theory.

Evaluating operational cloud enterprise systems success : an organizational perspective

This paper presents a formative measurement index to assess cloud enterprise systems success. The scale development procedure is based on Moore and Benbasat (1991), including newer scale development elements which focus on the creation and assessment of formative constructs. The data is analysed using SmartPLS with a sample of 103 IT decision makers. The results show that the perception of net benefits is shaped not only by enterprise-system-specific factors like productivity improvements and higher quality of business processes, but also by factors which are specifically attributed to cloud systems, such as higher strategic flexibility. Reliability, user requirements and customization contribute most to the overall perception of system quality. Information quality shows no cloud-specific facets and is robust in the context of cloud enterprise systems.

Characterizing intuition in information systems research

The study of intuition is an emerging area of research in psychology, social sciences, and business studies. It is increasingly of interest to the study of management, for example in decision-making as a counterpoint to structured approaches. Recently work has been undertaken to conceptualize a construct for the intuitive nature of technology. However to-date there is no common under-standing of the term intuition in information systems (IS) research. This paper extends the study of intuition in IS research by using exploratory research to cate-gorize the use of the word “intuition” and related terms in papers published in two prominent IS journals over a ten year period. The entire text of MIS Quarterly and Information Systems Research was reviewed for the years 1999 through 2008 using searchable PDF versions of these publications. As far as could be deter-mined, this is the first application of this approach in the analysis of the text of IS academic journals. The use of the word “intuition” and related terms was catego-rized using coding consistent with Grounded Theory. The focus of this research was on the first two stages of Grounded Theory analysis - the development of codes and constructs. Saturation of coding was not reached: an extended review of these publications would be required to enable theory development. Over 400 incidents of the use of “intuition”, and related terms were found in the articles reviewed. The most prominent use of the term of “intuition” was coded as “Intui-tion as Authority” in which intuition was used to validate a research objective or finding; representing approximately 37 per cent of codes assigned. The second most common coding occurred in research articles with mathematical analysis, representing about 19 per cent of the codes assigned, for example where a ma-thematical formulation or result was “intuitive”. The possibly most impactful use of the term “intuition” was “Intuition as Outcome”, representing approximately 7 per cent of all coding, which characterized research results as adding to the intui-tive understanding of a research topic or phenomena. This research contributes to a greater theoretical understanding of intuition enabling insight into the use of intuition, and the eventual development of a theory on the use of intuition in academic IS research publications. It also provides potential benefits to practi-tioners by providing insight into and validation of the use of intuition in IS man-agement. Research directions include the creation of reflective and/or formative constructs for intuition in information systems research.

A gestalt generated from grounded theory concerning intuition in IS research

Intuition is an important and under-researched concept in information systems. Prior exploratory research has shown that that there is potential to characterize the use of intuition in academic information systems research. This paper extends this research to all of the available issues of two leading IS journals with the aim of reaching an approximation of theoretical saturation. Specifically, the entire text of MISQ and ISR was reviewed for the years 1990 through 2009 using searchable PDF versions of these publications. All references to intuition were coded on a basis consistent with Grounded Theory, interpreted as a gestalt and represented as a mind-map. In the period 1990-2009, 681 incidents of the use of "intuition", and related terms were found in the articles reviewed, representing a greater range of codes than prior research. In addition, codes were assigned to all issues of MIS Quarterly from commencement of publication to the end of the 2012 publication year to support the conjecture that coding saturation has been approximated. The most prominent use of the term of "intuition" was coded as "Intuition as Authority" in which intuition was used to validate a statement, research objective or a finding; representing approximately 34 per cent of codes assigned. In research articles where mathematical analysis was presented, researchers not infrequently commented on the degree to which a mathematical formulation was "intuitive"; this was the second most common coding representing approximately 16 per cent of the codes. The possibly most impactful use of the term "intuition" was "Intuition as Outcome", representing approximately 7 per cent of all coding, which characterized research results as adding to the intuitive understanding of a research topic or phenomena.This research aims to contribute to a greater theoretical understanding of the use of intuition in academic IS research publications. It provides potential benefits to practitioners by providing insight into the use of intuition in IS management, for example, emphasizing the emerging importance of "intuitive technology". Research directions include the creation of reflective and/or formative constructs for intuition in information systems research and the expansion of this novel research method to additional IS academic publications and topics.

Formative and reflective measurement and validation mismatch in survey research : an archival analysis of information systems success constructs 1985-2007

The generic IS-success constructs first identified by DeLone and McLean (1992) continue to be widely employed in research. Yet, recent work by Petter et al (2007) has cast doubt on the validity of many mainstream constructs employed in IS research over the past 3 decades; critiquing the almost universal conceptualization and validation of these constructs as reflective when in many studies the measures appear to have been implicitly operationalized as formative. Cited examples of proper specification of the Delone and McLean constructs are few, particularly in light of their extensive employment in IS research. This paper introduces a four-stage formative construct development framework: Conceive > Operationalize > Respond > Validate (CORV). Employing the CORV framework in an archival analysis of research published in top outlets 1985-2007, the paper explores the extent of possible problems with past IS research due to potential misspecification of the four application-related success dimensions: Individual-Impact, Organizational-Impact, System-Quality and Information-Quality. Results suggest major concerns where there is a mismatch of the Respond and Validate stages. A general dearth of attention to the Operationalize and Respond stages in methodological writings is also observed.

The Australian Grey Nomads : are they who we think they are? Enhancing formative research through the quantitative assessment of psychological constructs

Issue addressed: Measures of 'social identity' and 'psychological sense of community' were included within a broader formative research inquiry to gain insight into the identity characteristics and level of connectedness among older recreational road travellers (commonly known as Grey Nomads). The research sought to gain insights on how best to reach or speak to this growing driver cohort. ----- ----- Method: Participants included 631 older recreational road travellers ranging in age from 50 years to over 80 years. Data were obtained through three scales which were incorporated into a larger formative research survey; an identity hierarchy, the Three Factor Model of Social Identity and the Sense of Community Index. ----- ----- Results: Older recreational road travellers see themselves principally as couples, with social group identity being secondary. Although many identified to some degree with the Grey Nomad identity, when asked to self categorise as either members of the Broad Network of Recreational Vehicle Travellers or as Grey Nomads, the majority categorised themselves as the former. Those identifying as Grey Nomads, however, reported significantly higher levels of 'social identification' and 'sense of community'. ----- ----- Conclusion: The Grey Nomad identity may not be the best identity at which to target road safety messages for this cohort. Targeting travelling 'couples' may be more efficacious. Using the 'Grey Nomad' identity is likely to reap at least some success, however, given that many identified to some degree with this group identity. Those identifying as Grey Nomads may be more open to community participation or behaviour change given their significantly higher levels of 'social identity' and 'sense of community'.

Making climate change manageable : how accounting constructs new power-knowledge regimes

This paper examines the modern power of accounting to permeate new spheres and create new objects by examining how climate change becomes a new category for accounting’s attention. It outlines the socio-political problem space where accounting and climate change connect by tracing the emergence of the UK’s Climate Change Act (2008) to a specifically modern calculating attitude described here as ‘managing by the numbers’. It suggests the intersection of accounting and climate change was made possible by accounting’s role in tying disciplinary subjectivities and objectivities together whilst operating simultaneously at the level of individuals, organisations and government. Such that when faced with new unknowns we revert to previous ways of managing we have come to know and experienced throughout our formative years in schools, hospitals, firms and government departments. In this way, accounting’s emergence in the domain of managing climate change implies a transformation that cannot be explained merely as a practical response to a global warming problem, but rather as an example of a new power-knowledge regime that makes possible the management and control of a new organisational phenomenon called climate change.

A formative evaluation of the American Diabetes Association's Choosing Health and Sensible Exercise childhood obesity intervention

Background. The increasing prevalence of overweight among youth in the United States, and the parallel rise in related medical comorbidities has led to a growing need for efficient weight-management interventions. Purpose. The aim of this study was to evaluate the effects of the Choosing Health and Sensible Exercise (C.H.A.S.E.) childhood obesity prevention program on Body Mass Index (BMI), physical activity and dietary behaviors. Methods. This study utilized de-identified data collected during the fall 2006 session of the C.H.A.S.E. program. A total of 65 students at Woodview Elementary School and Deepwater Elementary School participated in this intervention. The C.H.A.S.E. program is a 10-week obesity prevention program that focuses on nutrition and physical activity education. Collection of height and weight data, and a health behavior survey was conducted during the first and last week of the intervention. Paired t-tests were used to determine statistically significant differences between pre- and post-intervention measurements. One-way analysis of variance was used to adjust for potential confounders, such as gender, age, BMI category ("normal weight", "at risk overweight", or "overweight"), and self-reported weight loss goals. Data were analyzed using STATA, v. 9.2. Results. A significant decrease in mean BMI (p< 0.05) was found after the 10-week intervention. While the results were statistically significant for the group as a whole, changes in BMI were not significant when stratified by age, sex, or ethnicity. The mean overall scores for the behavior survey did not change significantly pre- and post-intervention; however, significant differences were found in the dietary intention scale, indicating that students were more likely to intend to make healthier food choices (p<0.05). No statistically significant decreases in BMI were found when stratified for baseline BMI-for-age percentiles or baseline weight loss efforts (p>0.05). Conclusion. The results of this evaluation provide information that will be useful in planning and implementing an effective childhood obesity intervention in the future. Changes in the self-reported dietary intentions and BMI show that the C.H.A.S.E. program is capable of modifying food choice selection and decreasing BMI. Results from the behavior questionnaire indicate that students in the intervention program were making changes in a positive direction. Future implementation of the C.H.A.S.E. program, as well as other childhood obesity interventions, may want to consider incorporating additional strategies to increase knowledge and other behavioral constructs associated with decreased BMI. In addition, obesity prevention programs may want to increase parental involvement and increase the dose or intensity of the intervention. ^

On a distinction between casual and formative variable models:theoretical issues and practical implications for organizational researchers

Formative measurement has seen increasing acceptance in organizational research since the turn of the 21st Century. However, in more recent times, a number of criticisms of the formative approach have appeared. Such work argues that formatively-measured constructs are empirically ambiguous and thus flawed in a theory-testing context. The aim of the present paper is to examine the underpinnings of formative measurement theory in light of theories of causality and ontology in measurement in general. In doing so, a thesis is advanced which draws a distinction between reflective, formative, and causal theories of latent variables. This distinction is shown to be advantageous in that it clarifies the ontological status of each type of latent variable, and thus provides advice on appropriate conceptualization and application. The distinction also reconciles in part both recent supportive and critical perspectives on formative measurement. In light of this, advice is given on how most appropriately to model formative composites in theory-testing applications, placing the onus on the researcher to make clear their conceptualization and operationalisation.

Problems with formative and higher-order reflective variables

Cadogan and Lee (this issue) discuss the problems inherent in modeling formative latent variables as endogenous. In response to the commentaries by Rigdon (this issue) and Finn and Wang (this issue), the present article extends the discussion on formative measures. First, the article shows that regardless of whether statistical identification is achieved, researchers are unable to illuminate the nature of a formative latent variable. Second, the study clarifies issues regarding formative indicator weighting, highlighting that the weightings of formative components should be specified as part of the construct definition. Finally, the study shows that higher-order reflective constructs are invalid, highlights the damage their use can inflict on theory development and knowledge accumulation, and provides recommendations on a number of alternative models which should be used in their place (including the formative model). © 2012 Elsevier Inc.

Formative variables and unreal variables:why the formative MIMIC model is invalid

In this rejoinder, we provide a response to the three commentaries written by Diamantopoulos, Howell, and Rigdon (all this issue) on our paper The MIMIC Model and Formative Variables: Problems and Solutions (also this issue). We contrast the approach taken in the latter paper (where we focus on clarifying the assumptions required to reject the formative MIMIC model) by spending time discussing what assumptions would be necessary to accept the use of the formative MIMIC model as a viable approach. Importantly, we clarify the implications of entity realism and show how it is entirely logical that some theoretical constructs can be considered to have real existence independent of their indicators, and some cannot. We show how the formative model only logically holds when considering these ‘unreal’ entities. In doing so, we provide important counter-arguments for much of the criticisms made in Diamantopoulos’ commentary, and the distinction also helps clarify a number of issues in the commentaries of Howell and Rigdon (both of which in general agree with our original paper). We draw together these various threads to provide a set of conceptual tools researchers can use when thinking about the entities in their theoretical models.