A measurement instrument for process modeling research : development, test and procedural model

Process modeling is an emergent area of Information Systems research that is characterized through an abundance of conceptual work with little empirical research. To fill this gap, this paper reports on the development and validation of an instrument to measure user acceptance of process modeling grammars. We advance an extended model for a multi-stage measurement instrument development procedure, which incorporates feedback from both expert and user panels. We identify two main contributions: First, we provide a validated measurement instrument for the study of user acceptance of process modeling grammars, which can be used to assist in further empirical studies that investigate phenomena associated with the business process modeling domain. Second, in doing so, we describe in detail a procedural model for developing measurement instruments that ensures high levels of reliability and validity, which may assist fellow scholars in executing their empirical research.

Self-neglect: development and evaluation of a self-neglect (SN-37) measurement instrument

Aim: To develop and evaluate the psychometric properties of an instrument for the measurement of self-neglect (SN).Conceptual Framework: An elder self-neglect (ESN) conceptual framework guided the literature review and scale development. The framework has two key dimensions physical/psycho-social and environmental and seven sub dimensions which are representative of the factors that can contribute to intentional and unintentional SN. Methods: A descriptive cross-sectional design was adopted to achieve the research aim. The study was conducted in two phases. Phase 1 involved the development of the questionnaire content and structure. Phase 2 focused on establishing the psychometric properties of the instrument. Content validity was established by a panel of 8 experts and piloted with 9 health and social care professionals. The instrument was subsequently posted with a stamped addressed envelope to 566 health and social care professionals who met specific eligibility criteria across the four HSE areas. A total of 341 questionnaires were returned, a response rate of 60% and 305 (50%) completed responses were included in exploratory factor analysis (EFA). Item and factor analyses were performed to elicit the instruments underlying factor structure and establish preliminary construct validity. Findings: Item and factor analyses resulted in a logically coherent, 37 items, five factor solution, explaining 55.6% of the cumulative variance. The factors were labelled: ‘Environment’, ‘Social Networks’, ‘Emotional and Behavioural Liability’, ‘Health Avoidance’ and ‘Self-Determinism’. The factor loadings were >0.40 for all items on each of the five subscales. Preliminary construct validity was supported by findings. Conclusion: The main outcome of this research is a 37 item Self-Neglect (SN-37) measurement instrument that was developed by EFA and underpinned by an ESN conceptual framework. Preliminary psychometric evaluation of the instrument is promising. Future work should be directed at establishing the construct and criterion related validity of the instrument.

Towards an information society measurement instrument for Latin America and the Caribbean: getting started with census, household and business surveys

Includes bibliography

A balanced scorecard approach to enterprise systems performance measurement

A range of influences, both technical and organisational, has encouraged the wide spread adoption of Enterprise Systems (ES). Nevertheless, there is a growing consensus that Enterprise Systems have in many cases failed to provide expected benefits. The increasing role of, and dependency on ES (and IT in general), and the ‘uncertainty’ of these large investments, have created a strong need to monitor and measure ES performance. This paper reports on a research project aimed at deriving an ‘Enterprise Systems benefits measurement instrument’. The research seeks to identify how Enterprise Systems benefits can be usefully measured, with a ‘balance’ between qualitative and quantitative factors.

Developing sustainable IT-related capabilities : instrument development and test

Research has established that firms' IT-related capabilities at a point in time explain IT-related performance differences across firms. IT resources, however, are dynamic, and evolve at an exponential rate. This means we need to understand how to sustain firms' existing capabilities to leverage opportunities offered by new IT resources. Wet suggests a higher-level resource that can sustain firms' existing IT-related capabilities. Second, we report on the development of a valid and reliable measurement instrument for measuring this higher-level resource in four stages, which includes expert feedback and a field test. The validated instrument would be useful in extending the IT business value studies to investigate how firms can sustain their IT-related capabilities. This effort will provide a deeper understanding of how firms can secure sustainable IT-related business value from their acquired IT resources.

Realizing benefits from enterprise architecture : a measurement model

Enterprise architecture (EA) management has become an intensively discussed approach to manage enterprise transformations. Despite the popularity and potential of EA, both researchers and practitioners lament a lack of knowledge about the realization of benefits from EA. To determine the benefits from EA, we explore the various dimensions of EA benefit realization and report on the development of a validated and robust measurement instrument. In this paper, we test the reliability and construct validity of the EA benefit realization model (EABRM), which we have designed based on the DeLone & McLean IS success model and findings from exploratory interviews. A confirmatory factor analysis confirms the existence of an impact of five distinct and individually important dimensions on the benefits derived from EA: EA artefact quality, EA infrastructure quality, EA service quality, EA culture, and EA use. The analysis presented in this paper shows that the EA benefit realization model is an instrument that demonstrates strong reliability and validity.

Development and validation of an instrument to measure organizational cultures' support of business process management

The purpose of Business Process Management (BPM) is to increase the efficiency and effectiveness of organizational processes through improvement and innovation. Despite a common understanding that culture is an important element in these efforts, there is a dearth of theoretical and empirical research on culture as a facilitator of successful BPM. We develop the BPM culture construct and propose a validated instrument with which to measure organizational cultures’ support of BPM. The operationalization of the BPM culture concept provides a theoretical foundation for future research and a tool to assist organizations in developing a cultural environment that supports successful BPM.

Effect of the interfacial area measurement parameters on the push-out strength between fiber post and dentin

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Measuring the use of information and communication technologies (ICTs) in the classroom

In 2003, the “ICT Curriculum Integration Performance Measurement Instrument” was developed froman extensive review ofthe contemporary international and Australian research pertaining to the definition and measurement of ICT curriculum integration in classrooms (Proctor, Watson, & Finger, 2003). The 45-item instrument that resulted was based on theories and methodologies identified by the literature review. This paper describes psychometric results from a large-scale evaluation of the instrument subsequently conducted, as recommended by Proctor, Watson, and Finger (2003). The resultant 20-item, two-factor instrument, now called “Learning with ICTs: Measuring ICT Use in the Curriculum,” is both statistically and theoretically robust. This paper should be read in association with the original paper published in Computers in the Schools(Proctor, Watson, & Finger, 2003) that described in detail the theoretical framework underpinning the development of the instrument.

Measuring cognitive and dispositional characteristics of creativity in elementary students

This article describes the theoretical underpinning and development of a measurement instrument that provides teachers with a tool to observe the personal creativity characteristics of individual students. The instrument was developed by compiling a list of characteristics derived from the literature to be indicative of the personal characteristics of creative people. The list was then reduced by grouping like characteristics to 9 cognitive and dispositional traits that were considered appropriate for elementary students. The 9-item instrument was then administered in 24 classrooms to 520 Year 6 and Year 7 students. Factor analysis using maximum likelihood extraction with an oblimin rotation revealed a single factor with an eigenvalue greater than 1 and accounting for 63% of the variance. All 9 items on this factor loaded at .72 or greater. The results indicated that the Creativity Checklist has very high internal consistency and is a reliable measurement instrument (a = .93).

Construction performance and organisational culture

The purpose of this paper is to summarize the outcomes of a detailed research study carried out as part of the fulfilment of a doctoral programme which examined the relationships between, and impacts of organisational culture on construction performance within a Hong Kong context. The research used a mixed methodology approach consisting of an organisational culture survey using an adapted validated and reliable measurement instrument (the Denison Organisational Culture Survey), mini-case studies in four Hong Kong construction companies and correlated the derived culture scores against performance scores measured by the Hong Kong Housing Department Performance Assessment Scoring System (PASS). The significance of the research was to advance knowledge of the importance of organisational culture strength as a performance driver in the construction industry and the further proof of the culture performance links using a set of measures of the latter which were not financially-based. The findings of the research make a contribution to theory by further validating the work by Denison (1990) and others, not only in that a successful link between organisational culture and performance was demonstrated, but it also identifies particular cultural factors in organisations that appear to be significantly responsible for achieving successful outcomes and reveals opportunities for further research into the organisational culture of construction companies Keywords: organisational culture, construction performance, business success.

“Entrepreneurial management” as a strategic choice in firm behavior : linking it with performance

Establishing the core principals of “entrepreneurial management” within an organization describes a certain strategic choice that affects a company in six dimensions, according to Stevenson (1983). Our aim is to empirically measure entrepreneurial management (it’s existence and degree) and to link this measured strategic choice (for or against) entrepreneurial management with firm performance. Our argument here is that companies that follow core principals of entrepreneurial management should outperform other more administrative firms in certain measures of strategic performance. This paper builds on an empirical investigation published by Brown, Davidson & Wiklund (2001), who have developed and tested a reliable measurement instrument for Stevenson’s definition of “entrepreneurial management” (Stevenson 1983, Stevenson & Jarillo 1990). In the first part of our paper we aim to replicate and to some extent improve this study. In the second part we link the measured degree of “entrepreneurial management” with firm performance. To our knowledge, even so Stevenson’s definition of entrepreneurial management is commonly acknowledged and Brown et al. (2001) developed a reliable instrument to empirically capture this behavioral approach to management, the construct of entrepreneurial management never before has been linked to firm performance in an empirical study. Since most papers on corporate entrepreneurship and firm performance are based on Covin & Slevin’s (1991) or Miller’s (1983) concept of entrepreneurial orientation, we contribute to the literature on corporate entrepreneurship in a novel way, given the fact that the entrepreneurial management dimensions measured in our study can theoretically and empirically be clearly distinguished from the construct of entrepreneurial orientation as defined by Covin & Selvin (1991).

Observing the relevance of an information systems success model

This paper contributes to the rigor vs. relevance debate in the Information Systems (IS) discipline. Using the Action Research methodology, this study evaluates the relevance of a rigorously validated IS evaluation model in practice. The study captures observations of operational end-users employing a market leading Enterprise System application for procurement and order fulfillment in an organization. The analysis of the observations demonstrates the broad relevance of the measurement instrument. More importantly, the study identifies several improvements and possible confusions in applying the instrument in the practice.

Validating the is-impact model in the Malaysian public sector

This research is one of several ongoing studies conducted within the IT Professional Services (ITPS) research programme at Queensland University of Technology (QUT). In 2003, ITPS introduced the IS-Impact model, a measurement model for measuring information systems success from the viewpoint of multiple stakeholders. The model, along with its instrument, is robust, simple, yet generalisable, and yields results that are comparable across time, stakeholders, different systems and system contexts. The IS-Impact model is defined as “a measure at a point in time, of the stream of net benefits from the Information System (IS), to date and anticipated, as perceived by all key-user-groups”. The model represents four dimensions, which are ‘Individual Impact’, ‘Organizational Impact’, ‘Information Quality’ and ‘System Quality’. The two Impact dimensions measure the up-to-date impact of the evaluated system, while the remaining two Quality dimensions act as proxies for probable future impacts (Gable, Sedera & Chan, 2008). To fulfil the goal of ITPS, “to develop the most widely employed model” this research re-validates and extends the IS-Impact model in a new context. This method/context-extension research aims to test the generalisability of the model by addressing known limitations of the model. One of the limitations of the model relates to the extent of external validity of the model. In order to gain wide acceptance, a model should be consistent and work well in different contexts. The IS-Impact model, however, was only validated in the Australian context, and packaged software was chosen as the IS understudy. Thus, this study is concerned with whether the model can be applied in another different context. Aiming for a robust and standardised measurement model that can be used across different contexts, this research re-validates and extends the IS-Impact model and its instrument to public sector organisations in Malaysia. The overarching research question (managerial question) of this research is “How can public sector organisations in Malaysia measure the impact of information systems systematically and effectively?” With two main objectives, the managerial question is broken down into two specific research questions. The first research question addresses the applicability (relevance) of the dimensions and measures of the IS-Impact model in the Malaysian context. Moreover, this research question addresses the completeness of the model in the new context. Initially, this research assumes that the dimensions and measures of the IS-Impact model are sufficient for the new context. However, some IS researchers suggest that the selection of measures needs to be done purposely for different contextual settings (DeLone & McLean, 1992, Rai, Lang & Welker, 2002). Thus, the first research question is as follows, “Is the IS-Impact model complete for measuring the impact of IS in Malaysian public sector organisations?” [RQ1]. The IS-Impact model is a multidimensional model that consists of four dimensions or constructs. Each dimension is represented by formative measures or indicators. Formative measures are known as composite variables because these measures make up or form the construct, or, in this case, the dimension in the IS-Impact model. These formative measures define different aspects of the dimension, thus, a measurement model of this kind needs to be tested not just on the structural relationship between the constructs but also the validity of each measure. In a previous study, the IS-Impact model was validated using formative validation techniques, as proposed in the literature (i.e., Diamantopoulos and Winklhofer, 2001, Diamantopoulos and Siguaw, 2006, Petter, Straub and Rai, 2007). However, there is potential for improving the validation testing of the model by adding more criterion or dependent variables. This includes identifying a consequence of the IS-Impact construct for the purpose of validation. Moreover, a different approach is employed in this research, whereby the validity of the model is tested using the Partial Least Squares (PLS) method, a component-based structural equation modelling (SEM) technique. Thus, the second research question addresses the construct validation of the IS-Impact model; “Is the IS-Impact model valid as a multidimensional formative construct?” [RQ2]. This study employs two rounds of surveys, each having a different and specific aim. The first is qualitative and exploratory, aiming to investigate the applicability and sufficiency of the IS-Impact dimensions and measures in the new context. This survey was conducted in a state government in Malaysia. A total of 77 valid responses were received, yielding 278 impact statements. The results from the qualitative analysis demonstrate the applicability of most of the IS-Impact measures. The analysis also shows a significant new measure having emerged from the context. This new measure was added as one of the System Quality measures. The second survey is a quantitative survey that aims to operationalise the measures identified from the qualitative analysis and rigorously validate the model. This survey was conducted in four state governments (including the state government that was involved in the first survey). A total of 254 valid responses were used in the data analysis. Data was analysed using structural equation modelling techniques, following the guidelines for formative construct validation, to test the validity and reliability of the constructs in the model. This study is the first research that extends the complete IS-Impact model in a new context that is different in terms of nationality, language and the type of information system (IS). The main contribution of this research is to present a comprehensive, up-to-date IS-Impact model, which has been validated in the new context. The study has accomplished its purpose of testing the generalisability of the IS-Impact model and continuing the IS evaluation research by extending it in the Malaysian context. A further contribution is a validated Malaysian language IS-Impact measurement instrument. It is hoped that the validated Malaysian IS-Impact instrument will encourage related IS research in Malaysia, and that the demonstrated model validity and generalisability will encourage a cumulative tradition of research previously not possible. The study entailed several methodological improvements on prior work, including: (1) new criterion measures for the overall IS-Impact construct employed in ‘identification through measurement relations’; (2) a stronger, multi-item ‘Satisfaction’ construct, employed in ‘identification through structural relations’; (3) an alternative version of the main survey instrument in which items are randomized (rather than blocked) for comparison with the main survey data, in attention to possible common method variance (no significant differences between these two survey instruments were observed); (4) demonstrates a validation process of formative indexes of a multidimensional, second-order construct (existing examples mostly involved unidimensional constructs); (5) testing the presence of suppressor effects that influence the significance of some measures and dimensions in the model; and (6) demonstrates the effect of an imbalanced number of measures within a construct to the contribution power of each dimension in a multidimensional model.