Formative and reflective measurement and validation mismatch in survey research : an archival analysis of information systems success constructs 1985-2007

The generic IS-success constructs first identified by DeLone and McLean (1992) continue to be widely employed in research. Yet, recent work by Petter et al (2007) has cast doubt on the validity of many mainstream constructs employed in IS research over the past 3 decades; critiquing the almost universal conceptualization and validation of these constructs as reflective when in many studies the measures appear to have been implicitly operationalized as formative. Cited examples of proper specification of the Delone and McLean constructs are few, particularly in light of their extensive employment in IS research. This paper introduces a four-stage formative construct development framework: Conceive > Operationalize > Respond > Validate (CORV). Employing the CORV framework in an archival analysis of research published in top outlets 1985-2007, the paper explores the extent of possible problems with past IS research due to potential misspecification of the four application-related success dimensions: Individual-Impact, Organizational-Impact, System-Quality and Information-Quality. Results suggest major concerns where there is a mismatch of the Respond and Validate stages. A general dearth of attention to the Operationalize and Respond stages in methodological writings is also observed.

Cues to starting CPAP in obstructive sleep apnoea : development and validation of the cues to CPAP Use Questionnaire (CCUQ)

Background: The reasons that a patient has to start treatment, their “Cues to Action”, are important for determining subsequent health behaviours. Cues to action are an explicit component of the Health Belief Model of CPAP acceptance adherence. At present there is no scale available to measure this construct for individuals with Obstructive Sleep Apnoea (OSA). This paper aims to develop, validate and describe responding patterns within an OSA patient sample to the Cues to CPAP Use Questionnaire (CCUQ).----- Method: Participants were 63 adult patients diagnosed with OSA who had never tried CPAP when initially recruited. The CCUQ was completed at one month after being prescribed CPAP.----- Results: Exploratory factor analysis (EFA) showed a three factor structure of the 9-item CCUQ, with “Health Cues”, “Partner Cues” and “Health Professional Cues” subscales accounting for 59.91% of the total variance. The CCUQ demonstrated modest internal consistency and split-half reliability. The questionnaire is brief and user-friendly, with readability at a 7th grade level. The most frequently endorsed cues for starting CPAP were Health Professional Cues (prompting by the sleep physician) and Health Cues such as tiredness and concern about health outcomes.----- Conclusions: This study validates a measure of an important motivational component of the Health Belief Model. Health Professional Cues and internal Health Cues were reported to be the most important prompts to commence CPAP by this patient sample.

Development and validation of 3-Minute Nutrition Screening (3-MinNS) tool for acute hospital patients in Singapore

It is important to detect and treat malnutrition in hospital patients so as to improve clinical outcome and reduce hospital stay. The aim of this study was to develop and validate a nutrition screening tool with a simple and quick scoring system for acute hospital patients in Singapore. In this study, 818 newly admitted patients aged above 18 years old were screened using five parameters that contribute to the risk of malnutrition. A dietitian blinded to the nutrition screening score assessed the same patients using the reference standard, Subjective Global Assessment (SGA) within 48 hours. The sensitivity and specificity were established using the Receiver Operator Characteristics (ROC) curve and the best cutoff scores determined. The nutrition parameter with the largest Area Under the ROC Curve (AUC) was chosen as the final screening tool, which was named 3-Minute Nutrition Screening (3-MinNS). The combination of the parameters weight loss, intake and muscle wastage (3-MinNS), gave the largest AUC when compared with SGA. Using 3-MinNS, the best cutoff point to identify malnourished patients is three (sensitivity 86%, specificity 83%). The cutoff score to identify subjects at risk of severe malnutrition is five (sensitivity 93%, specificity 86%). 3-Minute Nutrition Screening is a valid, simple and rapid tool to identify patients at risk of malnutrition in Singapore acute hospital patients. It is able to differentiate patients at risk of moderate malnutrition and severe malnutrition for prioritization and management purposes.

Validation of the Edinburgh Postnatal Depression Scale for men, and comparison of item endorsement with their partners

BACKGROUND: The Edinburgh Postnatal Depression Scale (EPDS) has been validated and used extensively in screening for depression in new mothers, both in English speaking and non-English speaking communities. While some studies have reported the use of the EPDS with fathers, none have validated it for this group, and thus the appropriate cut-off score for screening for depression or anxiety caseness for this population is not known. METHODS: Couples were recruited antenatally and interviewed at six weeks postpartum. EPDS scores and distress caseness (depression or anxiety disorders) for 208 fathers and 230 mothers were determined using the Diagnostic Interview Schedule. RESULTS: Analyses of the EPDS for fathers using distress caseness (depression or anxiety disorders) as the criterion shows that a cut-off of 5/6 has optimum receiver operating characteristics. Furthermore acceptable reliability (split-half and internal consistency) and validity (concurrent) coefficients were obtained. For mothers the optimum cut-off screening value to detect distress caseness was 7/8. Item analysis revealed that fathers endorsed seven of the ten items at lower rates to mothers, with the most significant being that referring to crying. CONCLUSIONS: The EPDS is a reliable and valid measure of mood in fathers. Screening for depression or anxiety disorders in fathers requires a two point lower cut-off than screening for depression or anxiety in mothers, and we recommend this cut-off to be 5/6

Development and scientific validation of a neonatal ultrasound axial bone scanner

Preterm infants have an increased risk of low bone mass and subsequent fracture due to limited bone mass accretion in utero and a greater need for bone nutrients. The diagnosis of ostepeonia of prematurity remains difficult as there is no sctreening test which is both sensitive and specific.

Motivation, development and validation of a new spectral greenness index : a spectral dimension related to foliage projective cover

A method is presented for the development of a regional Landsat-5 Thematic Mapper (TM) and Landsat-7 Enhanced Thematic Mapper plus (ETM+) spectral greenness index, coherent with a six-dimensional index set, based on a single ETM+ spectral image of a reference landscape. The first three indices of the set are determined by a polar transformation of the first three principal components of the reference image and relate to scene brightness, percent foliage projective cover (FPC) and water related features. The remaining three principal components, of diminishing significance with respect to the reference image, complete the set. The reference landscape, a 2200 km2 area containing a mix of cattle pasture, native woodland and forest, is located near Injune in South East Queensland, Australia. The indices developed from the reference image were tested using TM spectral images from 19 regionally dispersed areas in Queensland, representative of dissimilar landscapes containing woody vegetation ranging from tall closed forest to low open woodland. Examples of image transformations and two-dimensional feature space plots are used to demonstrate image interpretations related to the first three indices. Coherent, sensible, interpretations of landscape features in images composed of the first three indices can be made in terms of brightness (red), foliage cover (green) and water (blue). A limited comparison is made with similar existing indices. The proposed greenness index was found to be very strongly related to FPC and insensitive to smoke. A novel Bayesian, bounded space, modelling method, was used to validate the greenness index as a good predictor of FPC. Airborne LiDAR (Light Detection and Ranging) estimates of FPC along transects of the 19 sites provided the training and validation data. Other spectral indices from the set were found to be useful as model covariates that could improve FPC predictions. They act to adjust the greenness/FPC relationship to suit different spectral backgrounds. The inclusion of an external meteorological covariate showed that further improvements to regional-scale predictions of FPC could be gained over those based on spectral indices alone.

The development and validation of Australian indices of child development - Part II : validity support

The two outcome indices described in a companion paper (Sanson et al., Child Indicators Research, 2009) were developed using data from the Longitudinal Study of Australian Children (LSAC). These indices, one for infants and the other for 4 year to 5 year old children, were designed to fill the need for parsimonious measures of children’s developmental status to be used in analyses by a broad range of data users and to guide government policy and interventions to support young children’s optimal development. This paper presents evidence from Wave 1data from LSAC to support the validity of these indices and their three domain scores of Physical, Social/Emotional, and Learning. Relationships between the indices and child, maternal, family, and neighborhood factors which are known to relate concurrently to child outcomes were examined. Meaningful associations were found with the selected variables, thereby demonstrating the usefulness of the outcome indices as tools for understanding children’s development in their family and socio-cultural contexts. It is concluded that the outcome indices are valuable tools for increasing understanding of influences on children’s development, and for guiding policy and practice to optimize children’s life chances.

The development and validation of Australian Indices of Child Development — Part I: conceptualisation and development

The Longitudinal Study of Australian Children (LSAC) is a major national study examining the lives of Australian children, using a cross-sequential cohort design and data from parents, children, and teachers for 5,107 infants (3–19 months) and 4,983 children (4–5 years). Its data are publicly accessible and are used by researchers from many disciplinary backgrounds. It contains multiple measures of children’s developmental outcomes as well as a broad range of information on the contexts of their lives. This paper reports on the development of summary outcome indices of child development using the LSAC data. The indices were developed to fill the need for indicators suitable for use by diverse data users in order to guide government policy and interventions which support young children’s optimal development. The concepts underpinning the indices and the methods of their development are presented. Two outcome indices (infant and child) were developed, each consisting of three domains—health and physical development, social and emotional functioning, and learning competency. A total of 16 measures are used to make up these three domains in the Outcome Index for the Child Cohort and six measures for the Infant Cohort. These measures are described and evidence supporting the structure of the domains and their underlying latent constructs is provided for both cohorts. The factorial structure of the Outcome Index is adequate for both cohorts, but was stronger for the child than infant cohort. It is concluded that the LSAC Outcome Index is a parsimonious measure representing the major components of development which is suitable for non-specialist data users. A companion paper (Sanson et al. 2010) presents evidence of the validity of the Index.

Validation of the Australian Propensity for Angry Driving Scale (Aus-PADS)

The present study used a university sample to assess the test-retest reliability and validity of the Australian Propensity for Angry Driving Scale (Aus-PADS). The scale has stability over time, and convergent validity was established, as Aus-PADS scores correlated significantly with established anger and impulsivity measures. Discriminant validity was also established, as Aus-PADS scores did not correlate with Venturesomeness scores. The Aus-PADS has demonstrated criterion validity, as scores were correlated with behavioural measures, such as yelling at other drivers, gesturing at other drivers, and feeling angry but not doing anything. Aus-PADS scores reliably predicted the frequency of these behaviours over and above other study variables. No significant relationship between aggressive driving and crash involvement was observed. It was concluded that the Aus-PADS is a reliable and valid tool appropriate for use in Australian research, and that the potential relationship between aggressive driving and crash involvement warrants further investigation with a more representative (and diverse) driver sample.

Validation of the impact of Health Information Technology (I-HIT) scale : an international collaborative

In 2005, the Healthcare Information Management Systems Society (HIMSS) Nursing Informatics Community developed a survey to measure the impact of health information technology (HIT), the IHIT Scale, on the role of nurses and interdisciplinary communication in hospital settings. In 2007, nursing informatics colleagues from Australia, England, Finland, Ireland, New Zealand, Scotland and the United States formed a research collaborative to validate the IHIT across countries. All teams have completed construct and face validation in their countries. Five out of six teams have initiated reliability testing by practicing nurses. This paper reports the international collaborative’s validation of the IHIT Scale completed to date.

Empirical derivation and validation of a wandering typology

OBJECTIVES: To develop and validate a wandering typology. ---------- DESIGN: Cross-sectional, correlational descriptive design. ---------- SETTING:: Twenty-two nursing homes and six assisted living facilities. ---------- PARTICIPANTS: One hundred forty-two residents with dementia who spoke English, met Diagnostic and Statistical Manual for Mental Disorders, Fourth Edition, criteria for dementia, scored less than 24 on the Mini-Mental State Examination (MMSE), were ambulatory (with or without assistive device), and maintained a stable regime of psychotropic medications were studied. ---------- MEASUREMENTS: Data on wandering were collected using direct observations, plotted serially according to rate and duration to yield 21 parameters, and reduced through factor analysis to four components: high rate, high duration, low to moderate rate and duration, and time of day. Other measures included the MMSE, Minimum Data Set 2.0 mobility items, Cumulative Illness Rating Scale—Geriatric, and tympanic body temperature readings. ---------- RESULTS: Three groups of wanderers were identified through cluster analysis: classic, moderate, and subclinical. MMSE, mobility, and cardiac and upper and lower gastrointestinal problems differed between groups of wanderers and in comparison with nonwanderers. ---------- CONCLUSION: Results have implications for improving identification of wanderers and treatment of possible contributing factors.

Customer-oriented deviance (COD) : a conceptualisation, development and validation of a behavioural measure

Frontline employee behaviours are recognised as vital for achieving a competitive advantage for service organisations. The services marketing literature has comprehensively examined ways to improve frontline employee behaviours in service delivery and recovery. However, limited attention has been paid to frontline employee behaviours that favour customers in ways that go against organisational norms or rules. This study examines these behaviours by introducing a behavioural concept of Customer-Oriented Deviance (COD). COD is defined as, “frontline employees exhibiting extra-role behaviours that they perceive to defy existing expectations or prescribed rules of higher authority through service adaptation, communication and use of resources to benefit customers during interpersonal service encounters.” This thesis develops a COD measure and examines the key determinants of these behaviours from a frontline employee perspective. Existing research on similar behaviours that has originated in the positive deviance and pro-social behaviour domains has limitations and is considered inadequate to examine COD in the services context. The absence of a well-developed body of knowledge on non-conforming service behaviours has implications for both theory and practice. The provision of ‘special favours’ increases customer satisfaction but the over-servicing of customers is also counterproductive for the service delivery and costly for the organisation. Despite these implications of non-conforming service behaviours, there is little understanding about the nature of these behaviours and its key drivers. This research builds on inadequacies in prior research on positive deviance, pro-social and pro-customer literature to develop the theoretical foundation of COD. The concept of positive deviance which has predominantly been used to study organisational behaviours is applied within a services marketing setting. Further, it addresses previous limitations in pro-social and pro-customer behavioural literature that has examined limited forms of behaviours with no clear understanding on the nature of these behaviours. Building upon these literature streams, this research adopts a holistic approach towards the conceptualisation of COD. It addresses previous shortcomings in the literature by providing a well bounded definition, developing a psychometrically sound measure of COD and a conceptually well-founded model of COD. The concept of COD was examined across three separate studies and based on the theoretical foundations of role theory and social identity theory. Study 1 was exploratory and based on in-depth interviews using the Critical Incident Technique (CIT). The aim of Study 1 was to understand the nature of COD and qualitatively identify its key drivers. Thematic analysis was conducted to analyse the data and the two potential dimensions of COD behaviours of Deviant Service Adaptation (DSA) and Deviant Service Communication (DSC) were revealed in the analysis. In addition, themes representing the potential influences of COD were broadly classified as individual factors, situational factors, and organisational factors. Study 2 was a scale development procedure that involved the generation and purification of items for the measure based on two student samples working in customer service roles (Pilot sample, N=278; Initial validation sample, N=231). The results for the reliability and Exploratory Factor Analyses (EFA) on the pilot sample suggested the scale had poor psychometric properties. As a result, major revisions were made in terms of item wordings and new items were developed based on the literature to reflect a new dimension, Deviant Use of Resources (DUR). The revised items were tested on the initial validation sample with the EFA analysis suggesting a four-factor structure of COD. The aim of Study 3 was to further purify the COD measure and test for nomological validity based on its theoretical relationships with key antecedents and similar constructs (key correlates). The theoretical model of COD consisting of nine hypotheses was tested on a retail and hospitality sample of frontline employees (Retail N=311; Hospitality N=305) of a market research panel using an online survey. The data was analysed using Structural Equation Modelling (SEM). The results provided support for a re-specified second-order three-factor model of COD which consists of 11 items. Overall, the COD measure was found to be reliable and valid, demonstrating convergent validity, discriminant validity and marginal partial invariance for the factor loadings. The results showed support for nomological validity, although the antecedents had differing impact on COD across samples. Specifically, empathy and perspective-taking, role conflict, and job autonomy significantly influenced COD in the retail sample, whereas empathy and perspective-taking, risk-taking propensity and role conflict were significant predictors in the hospitality sample. In addition, customer orientation-selling orientation, the altruistic dimension of organisational citizenship behaviours, workplace deviance, and social desirability responding were found to correlate with COD. This research makes several contributions to theory. First, the findings of this thesis extend the literature on positive deviance, pro-social and pro-customer behaviours. Second, the research provides an empirically tested model which describes the antecedents of COD. Third, this research contributes by providing a reliable and valid measure of COD. Finally, the research investigates the differential effects of the key antecedents in different service sectors on COD. The research findings also contribute to services marketing practice. Based on the research findings, service practitioners can better understand the phenomenon of COD and utilise the measurement tool to calibrate COD levels within their organisations. Knowledge on the key determinants of COD will help improve recruitment and training programs and drive internal initiatives within the firm.

Validation of an Australian sign language instrument of outcome measurement for adults in mental health settings

Objective: There are currently no adult mental health outcome measures that have been translated into Australian sign language (Auslan). Without a valid and reliable Auslan outcome measure, empirical research into the efficacy of mental health interventions for sign language users is unattainable. To address this research problem the Outcome Rating Scale (ORS), a measure of general functioning, was translated into Auslan and recorded on to digital video disk for use in clinical settings. The purpose of the present study was therefore to examine the reliability, validity and acceptability of an Auslan version of the ORS (ORS-Auslan). Method: The ORS-Auslan was administered to 44 deaf people who use Auslan as their first language and who identify as members of a deaf community (termed ‘Deaf’ people) on their first presentation to a mental health or counselling facility and to 55 Deaf people in the general community. The community sample also completed an Auslan version of the Depression Anxiety Stress Scale-21 (DASS-21). Results: t-Tests indicated significant differences between the mean scores for the clinical and community sample. Internal consistency was acceptable given the low number of items in the ORS-Auslan. Construct validity was established by significant correlations between total scores on the DASS-21-Auslan and ORS-Auslan. Acceptability of ORS-Auslan was evident in the completion rate of 93% compared with 63% for DASS-21-Auslan. Conclusions: This is the only Auslan outcome measure available that can be used across a wide variety of mental health and clinical settings. The ORS-Auslan provides mental health clinicians with a reliable and valid, brief measure of general functioning that can significantly distinguish between clinical and non-clinical presentations for members of the Deaf community.

Validation of the Drinking Refusal Self-Efficacy Questionnaire - Revised in an Adolescent Sample (DRSEQ-RA)

Adolescent drinking is a significant issue yet valid psychometric tools designed for this group are scarce. The Drinking Refusal Self-Efficacy Questionnaire—Revised Adolescent Version (DRSEQ-RA) is designed to assess an individual's belief in their ability to resist drinking alcohol. The original DRSEQ-R consists of three factors reflecting social pressure refusal self-efficacy, opportunistic refusal self-efficacy and emotional relief refusal self-efficacy. A large sample of 2020 adolescents aged between 12 and 19 years completed the DRSEQ and measures of alcohol consumption in small groups. Using confirmatory factor analysis, the three factor structure was confirmed. All three factors were negatively correlated with both frequency and volume of alcohol consumption. Drinkers reported lower drinking refusal self-efficacy than non-drinkers. Taken together, these results suggest that the adolescent version of the Drinking Refusal Self-Efficacy Questionnaire (DRSEQ-RA) is a reliable and valid measure of drinking refusal self-efficacy.