Formative and reflective measurement and validation mismatch in survey research : an archival analysis of information systems success constructs 1985-2007

The generic IS-success constructs first identified by DeLone and McLean (1992) continue to be widely employed in research. Yet, recent work by Petter et al (2007) has cast doubt on the validity of many mainstream constructs employed in IS research over the past 3 decades; critiquing the almost universal conceptualization and validation of these constructs as reflective when in many studies the measures appear to have been implicitly operationalized as formative. Cited examples of proper specification of the Delone and McLean constructs are few, particularly in light of their extensive employment in IS research. This paper introduces a four-stage formative construct development framework: Conceive > Operationalize > Respond > Validate (CORV). Employing the CORV framework in an archival analysis of research published in top outlets 1985-2007, the paper explores the extent of possible problems with past IS research due to potential misspecification of the four application-related success dimensions: Individual-Impact, Organizational-Impact, System-Quality and Information-Quality. Results suggest major concerns where there is a mismatch of the Respond and Validate stages. A general dearth of attention to the Operationalize and Respond stages in methodological writings is also observed.

The quality of interface design for educational courseware development in Malaysian educational context

Interactive educational courseware has been adopted in diverse education sectors such as primary, secondary, tertiary education, vocational and professional training. In Malaysian educational context, the ministry of education has implemented Smart School Project that aims to increase high level of academic achievement in primary and secondary schools by using interactive educational courseware. However, many researchers have reported that many coursewares fail to accommodate the learner and teacher needs. In particular, the interface design is not appropriately designed in terms of quality of learning. This paper reviews educational courseware development process in terms of defining quality of interface design and suggests a conceptual model of interface design through the integration of design components and interactive learning experience into the development process. As a result, it defines the concept of interactive learning experience in a more practical approach in order to implement each stage of the development process in a seamless and integrated way.

The measurement of broadband ultrasonic attenuation in cancellous bone-a review of the science and technology

The measurement of broadband ultrasonic attenuation (BUA) in cancellous bone at the calcaneus was first described in 1984. The assessment of osteoporosis by BUA has recently been recognized by Universities UK, within its EurekaUK book, as being one of the “100 discoveries and developments in UK Universities that have changed the world” over the past 50 years, covering the whole academic spectrum from the arts and humanities to science and technology. Indeed, BUA technique has been clinically validated and is utilized worldwide, with at least seven commercial systems providing calcaneal BUA measurement. However, a fundamental understanding of the dependence of BUA upon the material and structural properties of cancellous bone is still lacking. This review aims to provide a science- and technology-orientated perspective on the application of BUA to the medical disease of osteoporosis.

Measuring cognitive and dispositional characteristics of creativity in elementary students

This article describes the theoretical underpinning and development of a measurement instrument that provides teachers with a tool to observe the personal creativity characteristics of individual students. The instrument was developed by compiling a list of characteristics derived from the literature to be indicative of the personal characteristics of creative people. The list was then reduced by grouping like characteristics to 9 cognitive and dispositional traits that were considered appropriate for elementary students. The 9-item instrument was then administered in 24 classrooms to 520 Year 6 and Year 7 students. Factor analysis using maximum likelihood extraction with an oblimin rotation revealed a single factor with an eigenvalue greater than 1 and accounting for 63% of the variance. All 9 items on this factor loaded at .72 or greater. The results indicated that the Creativity Checklist has very high internal consistency and is a reliable measurement instrument (a = .93).

An exploratory study of innovation effectiveness measurement between Australian and Thai SMEs

The advantages of using a balanced approach to measurement of overall organisational performance are well-known. We examined the effects of a balanced approach in the more specific domain of measuring innovation effectiveness in 144 small to medium sized companies in Australia and Thailand. We found that there were no differences in the metrics used by Australian and Thai companies. In line with our hypotheses, we found that those SMEs that took a balanced approach were more likely to perceive benefits of implemented innovations than those that used only a financial approach to measurement. The perception of benefits then had a subsequent effect on overall attitudes towards innovation. The study shows the importance of measuring both financial and non-financial indicators of innovation effectiveness within SMEs and discusses ways in which these can be conducted with limited resources.

An empirical study : a role of financial and non-financial performance measurement and perceived innovation effectiveness

Practitioners and academics often assume that investments in innovation will lead to organizational improvements. However, previous research has often shown that implemented innovations fail to realise these potential improvements. On the other hand, organisation, perhaps, has been growing and productive because of the innovation, but traditional measurements have failed to capture that growth. In order to help organizations capture their innovation performance effectively, this study examined the organizations which employ different types of performance measurement and their perception of innovation effectiveness.

Revisiting the conceptualization and measurement of service quality

Since the 1980s, industries and researchers have sought to better understand the quality of services due to the rise in their importance (Brogowicz, Delene and Lyth 1990). More recent developments with online services, coupled with growing recognition of service quality (SQ) as a key contributor to national economies and as an increasingly important competitive differentiator, amplify the need to revisit our understanding of SQ and its measurement. Although ‘SQ’ can be broadly defined as “a global overarching judgment or attitude relating to the overall excellence or superiority of a service” (Parasuraman, Berry and Zeithaml 1988), the term has many interpretations. There has been considerable progress on how to measure SQ perceptions, but little consensus has been achieved on what should be measured. There is agreement that SQ is multi-dimensional, but little agreement as to the nature or content of these dimensions (Brady and Cronin 2001). For example, within the banking sector, there exist multiple SQ models, each consisting of varying dimensions. The existence of multiple conceptions and the lack of a unifying theory bring the credibility of existing conceptions into question, and beg the question of whether it is possible at some higher level to define SQ broadly such that it spans all service types and industries. This research aims to explore the viability of a universal conception of SQ, primarily through a careful re-visitation of the services and SQ literature. The study analyses the strengths and weaknesses of the highly regarded and widely used global SQ model (SERVQUAL) which reflects a single-level approach to SQ measurement. The SERVQUAL model states that customers evaluate SQ (of each service encounter) based on five dimensions namely reliability, assurance, tangibles, empathy and responsibility. SERVQUAL, however, failed to address what needs to be reliable, assured, tangible, empathetic and responsible. This research also addresses a more recent global SQ model from Brady and Cronin (2001); the B&C (2001) model, that has potential to be the successor of SERVQUAL in that it encompasses other global SQ models and addresses the ‘what’ questions that SERVQUAL didn’t. The B&C (2001) model conceives SQ as being multidimensional and multi-level; this hierarchical approach to SQ measurement better reflecting human perceptions. In-line with the initial intention of SERVQUAL, which was developed to be generalizable across industries and service types, this research aims to develop a conceptual understanding of SQ, via literature and reflection, that encompasses the content/nature of factors related to SQ; and addresses the benefits and weaknesses of various SQ measurement approaches (i.e. disconfirmation versus perceptions-only). Such understanding of SQ seeks to transcend industries and service types with the intention of extending our knowledge of SQ and assisting practitioners in understanding and evaluating SQ. The candidate’s research has been conducted within, and seeks to contribute to, the ‘IS-Impact’ research track of the IT Professional Services (ITPS) Research Program at QUT. The vision of the track is “to develop the most widely employed model for benchmarking Information Systems in organizations for the joint benefit of research and practice.” The ‘IS-Impact’ research track has developed an Information Systems (IS) success measurement model, the IS-Impact Model (Gable, Sedera and Chan 2008), which seeks to fulfill the track’s vision. Results of this study will help future researchers in the ‘IS-Impact’ research track address questions such as: • Is SQ an antecedent or consequence of the IS-Impact model or both? • Has SQ already been addressed by existing measures of the IS-Impact model? • Is SQ a separate, new dimension of the IS-Impact model? • Is SQ an alternative conception of the IS? Results from the candidate’s research suggest that SQ dimensions can be classified at a higher level which is encompassed by the B&C (2001) model’s 3 primary dimensions (interaction, physical environment and outcome). The candidate also notes that it might be viable to re-word the ‘physical environment quality’ primary dimension to ‘environment quality’ so as to better encompass both physical and virtual scenarios (E.g: web sites). The candidate does not rule out the global feasibility of the B&C (2001) model’s nine sub-dimensions, however, acknowledges that more work has to be done to better define the sub-dimensions. The candidate observes that the ‘expertise’, ‘design’ and ‘valence’ sub-dimensions are supportive representations of the ‘interaction’, physical environment’ and ‘outcome’ primary dimensions respectively. The latter statement suggests that customers evaluate each primary dimension (or each higher level of SQ classification) namely ‘interaction’, physical environment’ and ‘outcome’ based on the ‘expertise’, ‘design’ and ‘valence’ sub-dimensions respectively. The ability to classify SQ dimensions at a higher level coupled with support for the measures that make up this higher level, leads the candidate to propose the B&C (2001) model as a unifying theory that acts as a starting point to measuring SQ and the SQ of IS. The candidate also notes, in parallel with the continuing validation and generalization of the IS-Impact model, that there is value in alternatively conceptualizing the IS as a ‘service’ and ultimately triangulating measures of IS SQ with the IS-Impact model. These further efforts are beyond the scope of the candidate’s study. Results from the candidate’s research also suggest that both the disconfirmation and perceptions-only approaches have their merits and the choice of approach would depend on the objective(s) of the study. Should the objective(s) be an overall evaluation of SQ, the perceptions-only approached is more appropriate as this approach is more straightforward and reduces administrative overheads in the process. However, should the objective(s) be to identify SQ gaps (shortfalls), the (measured) disconfirmation approach is more appropriate as this approach has the ability to identify areas that need improvement.

Capability measurement for Inclusive Design

Understanding users' capabilities, needs and expectations is key to the domain of Inclusive Design. Much of the work in the field could be informed and further strengthened by clear, valid and representative data covering the full range of people's capabilities. This article reviews existing data sets and identifies the challenges inherent in measuring capability in a manner that is informative for work in Inclusive Design. The need for a design-relevant capability data set is identified and consideration is given to a variety of capability construct operationalisation issues including questions associated with self-report and performance measures, sampling and the appropriate granularity of measures. The need for further experimental work is identified and a programme of research designed to culminate in the design of a valid and reliable capability survey is described.

Nursing students’ knowledge and practice of infection control precautions : an educational intervention

Aim. This paper is a report of the effectiveness of a purpose-designed education program in improving undergraduate nursing students’ understanding and practice of infection control precautions. Background. The severe acute respiratory syndrome outbreak in 2003 highlighted that healthcare workers were under-prepared for such an epidemic. While many in-service education sessions were arranged by institutions in response to the outbreak, preservice nursing education has overlooked preparation for handling such infectious disease epidemics. Method. A quasi-experimental design was used and a 16-hour, purpose-designed infection control education programme was implemented for preservice nursing students in southern Taiwan. Self-administered questionnaires were distributed at three time points during the period September 2005 to April 2006 to examine the sustainability and effectiveness of the intervention. Results. A total of 175 preservice nursing students participated in the study. Following the education programme, students in the intervention group showed a statistically significant improvement across time in their knowledge of these precautions [F(2, 180) = 13Æ53, P < 0Æ001] and confidence in resolving infectionrelated issues [F(1Æ79, 168Æ95) = 3Æ24] when compared with those in the control group. Conclusion. To improve nursing students’ capacity in responding to infectious epidemics, an educational programme that integrates the theme of infection precautions, learning theory and teaching strategies is recommended for all nursing institutes.

The problem of measurement indeterminacy in complex neurobiological movement systems

In the study of complex neurobiological movement systems, measurement indeterminacy has typically been overcome by imposing artificial modelling constraints to reduce the number of unknowns (e.g., reducing all muscle, bone and ligament forces crossing a joint to a single vector). However, this approach prevents human movement scientists from investigating more fully the role, functionality and ubiquity of coordinative structures or functional motor synergies. Advancements in measurement methods and analysis techniques are required if the contribution of individual component parts or degrees of freedom of these task-specific structural units is to be established, thereby effectively solving the indeterminacy problem by reducing the number of unknowns. A further benefit of establishing more of the unknowns is that human movement scientists will be able to gain greater insight into ubiquitous processes of physical self-organising that underpin the formation of coordinative structures and the confluence of organismic, environmental and task constraints that determine the exact morphology of these special-purpose devices.

Novel educational training program for community pharmacists

There is little evidence that workshops alone have a lasting impact on the day-to-day practice of participants. The current paper examined a strategy to increase generalization and maintenance of skills in the natural environment using pseudo-patients and immediate performance feedback to reinforce skills acquisition. A random half of pharmacies (N=30) took part in workshop training aimed at optimizing consumers' use of nonprescription analgesic products. Pharmacies in the training group also received performance feedback on their adherence to the recommended protocol. Feedback occurred immediately after a pseudo-patient visit in which confederates posed as purchasers of analgesics, and combined positive and corrective elements. Trained pharmacists were significantly more accurate at identifying people who misused the medication (P<0.001). The trained pharmacists were more likely than controls to use open-ended questions (P<0.001), assess readiness to change problematic use (P <0.001), and to deliver a brief intervention that was tailored to the person's commitment to alter his/her usage (P <0.001). Participants responded to the feedback positively. Results were consistent with the hypothesis that when workshop is combined with on-site performance feedback, it enhances practitioners' adherence to protocols in the natural setting.

Uncertainty budget in the measurement of typical airborne number, surface area and mass particle distributions

The effects of particulate matter on environment and public health have been widely studied in recent years. A number of studies in the medical field have tried to identify the specific effect on human health of particulate exposure, but agreement amongst these studies on the relative importance of the particles’ size and its origin with respect to health effects is still lacking. Nevertheless, air quality standards are moving, as the epidemiological attention, towards greater focus on the smaller particles. Current air quality standards only regulate the mass of particulate matter less than 10 μm in aerodynamic diameter (PM10) and less than 2.5 μm (PM2.5). The most reliable method used in measuring Total Suspended Particles (TSP), PM10, PM2.5 and PM1 is the gravimetric method since it directly measures PM concentration, guaranteeing an effective traceability to international standards. This technique however, neglects the possibility to correlate short term intra-day variations of atmospheric parameters that can influence ambient particle concentration and size distribution (emission strengths of particle sources, temperature, relative humidity, wind direction and speed and mixing height) as well as human activity patterns that may also vary over time periods considerably shorter than 24 hours. A continuous method to measure the number size distribution and total number concentration in the range 0.014 – 20 μm is the tandem system constituted by a Scanning Mobility Particle Sizer (SMPS) and an Aerodynamic Particle Sizer (APS). In this paper, an uncertainty budget model of the measurement of airborne particle number, surface area and mass size distributions is proposed and applied for several typical aerosol size distributions. The estimation of such an uncertainty budget presents several difficulties due to i) the complexity of the measurement chain, ii) the fact that SMPS and APS can properly guarantee the traceability to the International System of Measurements only in terms of number concentration. In fact, the surface area and mass concentration must be estimated on the basis of separately determined average density and particle morphology. Keywords: SMPS-APS tandem system, gravimetric reference method, uncertainty budget, ultrafine particles.

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

The use of the Index for Inclusion in a regional educational learning community

This project utilised the materials of the Index for Inclusion (Booth & Ainscow, 2002) to enhance the development of a learning community of educators in Education Queensland in 2009. The values, dimensions and indicators of the Index for Inclusion, were incorporated into the professional development package, On the Same Page (Education Queensland, 2008), to enhance its wider purpose to improve inclusive education practices explicit within the P-12 Curriculum Framework (Education Queensland, 2008). The incorporation of the values, dimensions and indicators of the Index enabled deeper reflection by participants about their expectations of students and their resulting teaching practices. The subsequent development of action plans assisted participants to develop “a curriculum for all” (Education Queensland, 2008, p. 9). Deeper reflection, action planning and ‘distance travelled’ in understanding of inclusive education were apparent in the comments by participants and their evaluation of the professional development package.

A Conceptualisation of Courseware Development Process in Malaysian Educational Context

Multimedia-based learning has been accepted as an effective learning tool and has broadly prevailed in various types of education around the world. The Malaysian ministry of education has also adopted this information communication technology (ICT) as the means of an education reformation project called, ‘Smart School’ since 1998, aiming to improve all Malaysian Primary and Secondary students’ learning ability, attitudes, achievement, and further enhance teachers’ teaching performance. As a result, Malaysian Ministry of Education has distributed a number of interactive courseware of the key learning domains such as Mathematics, Science, Bahasa Melayu (Malay language), and English. According to recent reports by Malaysian Ministry of Education (MOE), however, the courseware has not been effectively used in schools, and many researchers point out there are vital issues concerning the interface and interaction design. Within this context, this paper presumes that one of the main reasons could derive from a structural aspect of the course development process that is devaluing or ignoring the importance of interface and interaction design. Therefore, it is imperative to conceptualise the courseware development process in terms of creating interactive and quality learning experiences through defining the stakeholders’ needs in terms of better learning and teaching. Within this context, this paper reviews the current development process and proposes a new concept called the interactive communication component which enables courseware developers to embed interactive and quality learning experiences into their courseware development process. The key objective is to provide opportunities to discuss the courseware development process from the different stakeholders’ perspectives of the educational courseware in a Malaysian context.