Formative and reflective measurement and validation mismatch in survey research : an archival analysis of information systems success constructs 1985-2007

The generic IS-success constructs first identified by DeLone and McLean (1992) continue to be widely employed in research. Yet, recent work by Petter et al (2007) has cast doubt on the validity of many mainstream constructs employed in IS research over the past 3 decades; critiquing the almost universal conceptualization and validation of these constructs as reflective when in many studies the measures appear to have been implicitly operationalized as formative. Cited examples of proper specification of the Delone and McLean constructs are few, particularly in light of their extensive employment in IS research. This paper introduces a four-stage formative construct development framework: Conceive > Operationalize > Respond > Validate (CORV). Employing the CORV framework in an archival analysis of research published in top outlets 1985-2007, the paper explores the extent of possible problems with past IS research due to potential misspecification of the four application-related success dimensions: Individual-Impact, Organizational-Impact, System-Quality and Information-Quality. Results suggest major concerns where there is a mismatch of the Respond and Validate stages. A general dearth of attention to the Operationalize and Respond stages in methodological writings is also observed.

The measurement of broadband ultrasonic attenuation in cancellous bone-a review of the science and technology

The measurement of broadband ultrasonic attenuation (BUA) in cancellous bone at the calcaneus was first described in 1984. The assessment of osteoporosis by BUA has recently been recognized by Universities UK, within its EurekaUK book, as being one of the “100 discoveries and developments in UK Universities that have changed the world” over the past 50 years, covering the whole academic spectrum from the arts and humanities to science and technology. Indeed, BUA technique has been clinically validated and is utilized worldwide, with at least seven commercial systems providing calcaneal BUA measurement. However, a fundamental understanding of the dependence of BUA upon the material and structural properties of cancellous bone is still lacking. This review aims to provide a science- and technology-orientated perspective on the application of BUA to the medical disease of osteoporosis.

An exploratory study of innovation effectiveness measurement between Australian and Thai SMEs

The advantages of using a balanced approach to measurement of overall organisational performance are well-known. We examined the effects of a balanced approach in the more specific domain of measuring innovation effectiveness in 144 small to medium sized companies in Australia and Thailand. We found that there were no differences in the metrics used by Australian and Thai companies. In line with our hypotheses, we found that those SMEs that took a balanced approach were more likely to perceive benefits of implemented innovations than those that used only a financial approach to measurement. The perception of benefits then had a subsequent effect on overall attitudes towards innovation. The study shows the importance of measuring both financial and non-financial indicators of innovation effectiveness within SMEs and discusses ways in which these can be conducted with limited resources.

An empirical study : a role of financial and non-financial performance measurement and perceived innovation effectiveness

Practitioners and academics often assume that investments in innovation will lead to organizational improvements. However, previous research has often shown that implemented innovations fail to realise these potential improvements. On the other hand, organisation, perhaps, has been growing and productive because of the innovation, but traditional measurements have failed to capture that growth. In order to help organizations capture their innovation performance effectively, this study examined the organizations which employ different types of performance measurement and their perception of innovation effectiveness.

Revisiting the conceptualization and measurement of service quality

Since the 1980s, industries and researchers have sought to better understand the quality of services due to the rise in their importance (Brogowicz, Delene and Lyth 1990). More recent developments with online services, coupled with growing recognition of service quality (SQ) as a key contributor to national economies and as an increasingly important competitive differentiator, amplify the need to revisit our understanding of SQ and its measurement. Although ‘SQ’ can be broadly defined as “a global overarching judgment or attitude relating to the overall excellence or superiority of a service” (Parasuraman, Berry and Zeithaml 1988), the term has many interpretations. There has been considerable progress on how to measure SQ perceptions, but little consensus has been achieved on what should be measured. There is agreement that SQ is multi-dimensional, but little agreement as to the nature or content of these dimensions (Brady and Cronin 2001). For example, within the banking sector, there exist multiple SQ models, each consisting of varying dimensions. The existence of multiple conceptions and the lack of a unifying theory bring the credibility of existing conceptions into question, and beg the question of whether it is possible at some higher level to define SQ broadly such that it spans all service types and industries. This research aims to explore the viability of a universal conception of SQ, primarily through a careful re-visitation of the services and SQ literature. The study analyses the strengths and weaknesses of the highly regarded and widely used global SQ model (SERVQUAL) which reflects a single-level approach to SQ measurement. The SERVQUAL model states that customers evaluate SQ (of each service encounter) based on five dimensions namely reliability, assurance, tangibles, empathy and responsibility. SERVQUAL, however, failed to address what needs to be reliable, assured, tangible, empathetic and responsible. This research also addresses a more recent global SQ model from Brady and Cronin (2001); the B&C (2001) model, that has potential to be the successor of SERVQUAL in that it encompasses other global SQ models and addresses the ‘what’ questions that SERVQUAL didn’t. The B&C (2001) model conceives SQ as being multidimensional and multi-level; this hierarchical approach to SQ measurement better reflecting human perceptions. In-line with the initial intention of SERVQUAL, which was developed to be generalizable across industries and service types, this research aims to develop a conceptual understanding of SQ, via literature and reflection, that encompasses the content/nature of factors related to SQ; and addresses the benefits and weaknesses of various SQ measurement approaches (i.e. disconfirmation versus perceptions-only). Such understanding of SQ seeks to transcend industries and service types with the intention of extending our knowledge of SQ and assisting practitioners in understanding and evaluating SQ. The candidate’s research has been conducted within, and seeks to contribute to, the ‘IS-Impact’ research track of the IT Professional Services (ITPS) Research Program at QUT. The vision of the track is “to develop the most widely employed model for benchmarking Information Systems in organizations for the joint benefit of research and practice.” The ‘IS-Impact’ research track has developed an Information Systems (IS) success measurement model, the IS-Impact Model (Gable, Sedera and Chan 2008), which seeks to fulfill the track’s vision. Results of this study will help future researchers in the ‘IS-Impact’ research track address questions such as: • Is SQ an antecedent or consequence of the IS-Impact model or both? • Has SQ already been addressed by existing measures of the IS-Impact model? • Is SQ a separate, new dimension of the IS-Impact model? • Is SQ an alternative conception of the IS? Results from the candidate’s research suggest that SQ dimensions can be classified at a higher level which is encompassed by the B&C (2001) model’s 3 primary dimensions (interaction, physical environment and outcome). The candidate also notes that it might be viable to re-word the ‘physical environment quality’ primary dimension to ‘environment quality’ so as to better encompass both physical and virtual scenarios (E.g: web sites). The candidate does not rule out the global feasibility of the B&C (2001) model’s nine sub-dimensions, however, acknowledges that more work has to be done to better define the sub-dimensions. The candidate observes that the ‘expertise’, ‘design’ and ‘valence’ sub-dimensions are supportive representations of the ‘interaction’, physical environment’ and ‘outcome’ primary dimensions respectively. The latter statement suggests that customers evaluate each primary dimension (or each higher level of SQ classification) namely ‘interaction’, physical environment’ and ‘outcome’ based on the ‘expertise’, ‘design’ and ‘valence’ sub-dimensions respectively. The ability to classify SQ dimensions at a higher level coupled with support for the measures that make up this higher level, leads the candidate to propose the B&C (2001) model as a unifying theory that acts as a starting point to measuring SQ and the SQ of IS. The candidate also notes, in parallel with the continuing validation and generalization of the IS-Impact model, that there is value in alternatively conceptualizing the IS as a ‘service’ and ultimately triangulating measures of IS SQ with the IS-Impact model. These further efforts are beyond the scope of the candidate’s study. Results from the candidate’s research also suggest that both the disconfirmation and perceptions-only approaches have their merits and the choice of approach would depend on the objective(s) of the study. Should the objective(s) be an overall evaluation of SQ, the perceptions-only approached is more appropriate as this approach is more straightforward and reduces administrative overheads in the process. However, should the objective(s) be to identify SQ gaps (shortfalls), the (measured) disconfirmation approach is more appropriate as this approach has the ability to identify areas that need improvement.

Capability measurement for Inclusive Design

Understanding users' capabilities, needs and expectations is key to the domain of Inclusive Design. Much of the work in the field could be informed and further strengthened by clear, valid and representative data covering the full range of people's capabilities. This article reviews existing data sets and identifies the challenges inherent in measuring capability in a manner that is informative for work in Inclusive Design. The need for a design-relevant capability data set is identified and consideration is given to a variety of capability construct operationalisation issues including questions associated with self-report and performance measures, sampling and the appropriate granularity of measures. The need for further experimental work is identified and a programme of research designed to culminate in the design of a valid and reliable capability survey is described.

The problem of measurement indeterminacy in complex neurobiological movement systems

In the study of complex neurobiological movement systems, measurement indeterminacy has typically been overcome by imposing artificial modelling constraints to reduce the number of unknowns (e.g., reducing all muscle, bone and ligament forces crossing a joint to a single vector). However, this approach prevents human movement scientists from investigating more fully the role, functionality and ubiquity of coordinative structures or functional motor synergies. Advancements in measurement methods and analysis techniques are required if the contribution of individual component parts or degrees of freedom of these task-specific structural units is to be established, thereby effectively solving the indeterminacy problem by reducing the number of unknowns. A further benefit of establishing more of the unknowns is that human movement scientists will be able to gain greater insight into ubiquitous processes of physical self-organising that underpin the formation of coordinative structures and the confluence of organismic, environmental and task constraints that determine the exact morphology of these special-purpose devices.

Uncertainty budget in the measurement of typical airborne number, surface area and mass particle distributions

The effects of particulate matter on environment and public health have been widely studied in recent years. A number of studies in the medical field have tried to identify the specific effect on human health of particulate exposure, but agreement amongst these studies on the relative importance of the particles’ size and its origin with respect to health effects is still lacking. Nevertheless, air quality standards are moving, as the epidemiological attention, towards greater focus on the smaller particles. Current air quality standards only regulate the mass of particulate matter less than 10 μm in aerodynamic diameter (PM10) and less than 2.5 μm (PM2.5). The most reliable method used in measuring Total Suspended Particles (TSP), PM10, PM2.5 and PM1 is the gravimetric method since it directly measures PM concentration, guaranteeing an effective traceability to international standards. This technique however, neglects the possibility to correlate short term intra-day variations of atmospheric parameters that can influence ambient particle concentration and size distribution (emission strengths of particle sources, temperature, relative humidity, wind direction and speed and mixing height) as well as human activity patterns that may also vary over time periods considerably shorter than 24 hours. A continuous method to measure the number size distribution and total number concentration in the range 0.014 – 20 μm is the tandem system constituted by a Scanning Mobility Particle Sizer (SMPS) and an Aerodynamic Particle Sizer (APS). In this paper, an uncertainty budget model of the measurement of airborne particle number, surface area and mass size distributions is proposed and applied for several typical aerosol size distributions. The estimation of such an uncertainty budget presents several difficulties due to i) the complexity of the measurement chain, ii) the fact that SMPS and APS can properly guarantee the traceability to the International System of Measurements only in terms of number concentration. In fact, the surface area and mass concentration must be estimated on the basis of separately determined average density and particle morphology. Keywords: SMPS-APS tandem system, gravimetric reference method, uncertainty budget, ultrafine particles.

Measurement properties of the Australian Women's Activity Survey

Purpose: The Australian Women’s Activity Survey (AWAS) was developed based on a systematic review and qualitative research on how to measure activity patterns of women with young children (WYC). AWAS assesses activity performed across five domains (planned activities, employment, childcare, domestic responsibilities and transport), and intensity levels (sitting, light-intensity, brisk walking, moderate-intensity and vigorous-intensity) in a typical week in the past month. The purpose of this study was to assess the test-retest reliability and criterion validity of the AWAS. Methods: WYC completed the AWAS on two occasions 7-d apart (test-retest reliability protocol) and/or wore an MTI ActiGraph accelerometer for 7-d in between (validity protocol). Forty WYC (mean age 35 ± 5yrs) completed the test-retest reliability protocol and 75 WYC (mean age 33 ± 5yrs) completed the validity protocol. Interclass Correlation Coefficients (ICC) between AWAS administrations and Spearman’s Correlation Coefficients (rs) between AWAS and MTI data were calculated. Results: AWAS showed good test-retest reliability (ICC=0.80 (0.65-0.89)) and acceptable criterion validity (rs= 0.28, p=0.01) for measuring weekly health-enhancing physical activity. AWAS also provided repeatable and valid estimates of sitting time (test-retest reliability ICC=0.42 (0.13-0.64), and criterion validity (rs= 0.32, p=0.006)). Conclusion: The measurement properties of the AWAS are comparable to those reported for existing self-report measures of physical activity. However, AWAS offers a more comprehensive and flexible alternative for accurately assessing different domains and intensities of activity relevant to WYC. Future research should investigate whether the AWAS is a suitable measure of intervention efficacy by examining its sensitivity to change.

Measuring CRM and SCM benefits : a preliminary measurement model

Organizations invest heavily in Customer Relationship Management (CRM) and Supply Chain Management (SCM) systems, and their related infrastructure, presumably expecting positive benefits to the organization. Assessing the benefits of such applications is an important aspect of managing such systems. Considering the salient differences between CRM and SCM applications with other intra-organizational applications, existing Information Systems benefits measurement models and frameworks are ill-suited to gauge benefits of inter-organizational systems. This paper reports the preliminary findings of a measurement model developed to assess benefits of CRM and SCM applications. The preliminary model, which reflects the characteristics of the Analytic Theory, is derived using a review of 55 academic studies and 44 papers from the practice. Six hundred and six identified benefits were then synthesized in to 74 non-overlapping benefits, arranged under six dimensions.

A novel system for the measurement of body shape and construction of seating supports for patients

Patients with severe back deformities can greatly benefit from customized medical seating. Customized medical seating is made by taking measurements of each individual patient and making the seat as per these measurements. The current measuring systems employed by the industry are limited to use in clinics which are generally located only in major population centres. Patients living in remote areas are severely affected by this as the clinics could be far away and inaccessible for these patients. To provide service of customized medical seating requires a new measurement system which is portable so that the system could be transported to the patients in remote areas. The requirements for a new measurement system are analysed to suite the needs of Equipment Technology Services of the Cerebral Palsy League of Queensland. Design for a new measurement system was conceptualised by reviewing systems and technologies in various scientific disciplines. Design for a new system was finalised by optimizing each individual component. The final approach was validated by measuring difficult models and repeating the process to check for process variances. This system has now been adopted for clinical evaluation by ETS Suggestions have been made for further improvements in this new measurement approach.

Ultrasonic measurement of fine head movements in a standard ophthalmic headrest

We aimed to investigate the naturally occurring horizontal plane movements of a head stabilized in a standard ophthalmic headrest and to analyze their magnitude, velocity, spectral characteristics, and correlation to the cardio pulmonary system. Two custom-made air-coupled highly accurate (±2 μm)ultrasound transducers were used to measure the displacements of the head in different horizontal directions with a sampling frequency of 100 Hz. Synchronously to the head movements, an electrocardiogram (ECG) signal was recorded. Three healthy subjects participated in the study. Frequency analysis of the recorded head movements and their velocities was carried out, and functions of coherence between the two displacements and the ECG signal were calculated. Frequency of respiration and the heartbeat were clearly visible in all recorded head movements. The amplitude of head displacements was typically in the range of ±100 μm. The first harmonic of the heartbeat (in the range of 2–3 Hz), rather than its principal frequency, was found to be the dominant frequency of both head movements and their velocities. Coherence analysis showed high interdependence between the considered signals for frequencies of up to 20 Hz. These findings may contribute to the design of better ophthalmic headrests and should help other studies in the decision making of whether to use a heavy headrest or a bite bar.