Validation of multicriteria analysis models

Validation procedures play an important role in establishing the credibility of models, improving their relevance and acceptability. This article reviews the testing of models relevant to environmental and natural resource management with particular emphasis on models used in multicriteria analysis (MCA). Validation efforts for a model used in a MCA catchment management study in North Queensland, Australia, are presented. Determination of face validity is found to be a useful approach in evaluating this model, and sensitivity analysis is useful in checking the stability of the model. (C) 2000 Elsevier Science Ltd. All rights reserved.

A review of two journals found that articles using multivariable logistic regression frequently did not report commonly recommended assumptions

Background and Objective: To examine if commonly recommended assumptions for multivariable logistic regression are addressed in two major epidemiological journals. Methods: Ninety-nine articles from the Journal of Clinical Epidemiology and the American Journal of Epidemiology were surveyed for 10 criteria: six dealing with computation and four with reporting multivariable logistic regression results. Results: Three of the 10 criteria were addressed in 50% or more of the articles. Statistical significance testing or confidence intervals were reported in all articles. Methods for selecting independent variables were described in 82%, and specific procedures used to generate the models were discussed in 65%. Fewer than 50% of the articles indicated if interactions were tested or met the recommended events per independent variable ratio of 10: 1. Fewer than 20% of the articles described conformity to a linear gradient, examined collinearity, reported information on validation procedures, goodness-of-fit, discrimination statistics, or provided complete information on variable coding. There was no significant difference (P >.05) in the proportion of articles meeting the criteria across the two journals. Conclusion: Articles reviewed frequently did not report commonly recommended assumptions for using multivariable logistic regression. (C) 2004 Elsevier Inc. All rights reserved.

The development and evaluation of an incontinence screening questionnaire for female primary care

Although there is a high prevalence of leaking urine among Australian women, there are currently no standardized procedures for screening patients for incontinence in the primary care setting (known in Australia as general practice). In response to this, an incontinence screening questionnaire (ISQ) was developed and evaluated for use in general practice. Eighty-nine women completed an original compilation of 33 items that asked about situations associated with leaking urine, avoidance of leakage, and concern about leakage. Each item was assessed according to its acceptability for the population of female general practice patients, discriminative value, and test-retest reliability. These patients also underwent an objective test of incontinence, the 48-hour pad test, so that the screening items could be validated against an objective classification of incontinence. The study included women who had bladder control problems and those who did not. Eight items on the ISQ were shown to be acceptable to patients, discriminative, reliable, and valid indicators of objective incontinence. Five items were capable of predicting almost 70% of patients who showed objective leakage of urine and misclassified fewer than 15% of these patients. Those five items were selected for inclusion in the (refined) ISQ. (C) 2000 Wiley-Liss, Inc.

Validation of a food-frequency questionnaire assessment of carotenoid and vitamin E intake using weighed food records and plasma biomarkers: The method of triads model

Background: Reliability or validity studies are important for the evaluation of measurement error in dietary assessment methods. An approach to validation known as the method of triads uses triangulation techniques to calculate the validity coefficient of a food-frequency questionnaire (FFQ). Objective: To assess the validity of an FFQ estimates of carotenoid and vitamin E intake against serum biomarker measurements and weighed food records (WFRs), by applying the method of triads. Design: The study population was a sub-sample of adult participants in a randomised controlled trial of beta-carotene and sunscreen in the prevention of skin cancer. Dietary intake was assessed by a self-administered FFQ and a WFR. Nonfasting blood samples were collected and plasma analysed for five carotenoids (alpha-carotene, beta-carotene, beta-cryptoxanthin, lutein, lycopene) and vitamin E. Correlation coefficients were calculated between each of the dietary methods and the validity coefficient was calculated using the method of triads. The 95% confidence intervals for the validity coefficients were estimated using bootstrap sampling. Results: The validity coefficients of the FFQ were highest for alpha-carotene (0.85) and lycopene (0.62), followed by beta- carotene (0.55) and total carotenoids (0.55), while the lowest validity coefficient was for lutein (0.19). The method of triads could not be used for b- cryptoxanthin and vitamin E, as one of the three underlying correlations was negative. Conclusions: Results were similar to other studies of validity using biomarkers and the method of triads. For many dietary factors, the upper limit of the validity coefficients was less than 0.5 and therefore only strong relationships between dietary exposure and disease will be detected.

Translation and validation of the Chinese version of the Charing Cross Venous Ulcer Questionnaire

No Abstract

A state-of-practice questionnaire on verification and validation for concurrent programs

Research in verification and validation (V&V) for concurrent programs can be guided by practitioner information. A survey was therefore run to gain state-of-practice information in this context. The survey presented in this paper collected state-of-practice information on V&V technology in concurrency from 35 respondents. The results of the survey can help refine existing V&V technology by providing a better understanding of the context of V&V technology usage. Responses to questions regarding the motivation for selecting V&V technologies can help refine a systematic approach to V&V technology selection.

Goals and reputations amongst young children - The validation of the importance of goals and reputation enhancement scales

The aim of the present research was to provide school psychologists with valid instruments with which to assess the goals and reputations of young children. This was achieved by ascertaining whether the factor structures and the second-order factor models of the high school versions of the Importance of Goals (Carroll, et al., 1997) and Reputation Enhancement Scales (Carroll, et al., 1999) could be replicated with a primary school sample. Eight hundred and eighty-six 10 to 12 year old children were administered modified versions of the two scales, which were combined and renamed the Children's Activity Questionnaire. For the two scales, the factor structure proved replicable and reliable with the primary school sample. A comparison between the factor loadings of the primary school and the high school samples using the coefficient of congruence procedure demonstrated similarity indicating that the scales are replicable and able to be used with a younger primary school sample. Structural equation modelling indicated that the second-order factor structure of the Importance of Goals Scale was acceptable but this was not the case for the second-order factor structure of the Reputation Enhancement Scale.

The validity of an Australian modification of the AUDIT questionnaire

The Alcohol Use Disorders Identification Test (AUDIT) has been used widely and is reported to be superior to conventional questionnaires in detection of current hazardous and harmful alcohol use. We assessed the validity of an Australian modification of the AUDIT (the AusAUDIT), which has been employed widely in Australian and New Zealand early intervention programmes. We used a cross-sectional study of 370 subjects from the follow-up phase of a randomized controlled trial of early intervention to reduce hazardous alcohol consumption. Scores on the AusAUDIT were compared against 12-month ICD-10 diagnoses of harmful alcohol use and dependence, as determined by the Composite International Diagnostic Interview, and against self-report of alcohol consumption exceeding Australian National Health and Medical Research Council (NH&MRC) recommended limits. AusAUDIT had good internal consistency and discriminated significantly between persons meeting criteria for ICD-10 alcohol use disorders, and drinkers who did not. At currently recommended cut-off scores, AusAUDIT detected more than 85% of people meeting criteria for ICD-10 alcohol use disorders, or drinking over NH&MRC recommended limits, but its specificity was limited (29% in men, and 58% in women for drinking over NH&MRC limits). No subset of questions performed as well as the full AusAUDIT in detection of drinking problems, but the alcohol consumption items provided a reasonable screen for drinking over NH&MRC limits. We conclude that AusAUDIT is effective in detecting problematic drinking, but positive cases should be confirmed by clinical assessment. The findings illustrate the need for validation of questionnaire modifications, and the difficulty in increasing test sensitivity without reducing specificity.

Validation and refinement of survival models for liver retransplantation

Orthotopic liver retransplantation (re-OLT) is highly controversial. The objectives of this study were to determine the validity of a recently developed United Network for Organ Sharing (UNOS) multivariate model using an independent cohort of patients undergoing re-OLT outside the United States, to determine whether incorporation of other variables that were incomplete in the UNOS registry would provide additional prognostic information, to develop new models combining data sets from both cohorts, and to evaluate the validity of the model for end-stage liver disease (MELD) in patients undergoing re-OLT. Two hundred eighty-one adult patients undergoing re-OLT (between 1986 and 1999) at 6 foreign transplant centers comprised the validation cohort. We found good agreement between actual survival and predicted survival in the validation cohort; 1-year patient survival rates in the low-, intermediate-, and high-risk groups (as assigned by the original UNOS model) were 72%, 68%, and 36%, respectively (P < .0001). In the patients for whom the international normalized ratio (INR) of prothrombin time was available, MELD correlated with outcome following re-OLT; the median MELD scores for patients surviving at least 90 days compared with those dying within 90 days were 20.75 versus 25.9, respectively (P = .004). Utilizing both patient cohorts (n = 979), a new model, based on recipient age, total serum bilirubin, creatinine, and interval to re-OLT, was constructed (whole model χ(2) = 105, P < .0001). Using the c-statistic with 30-day, 90-day, 1-year, and 3-year mortality as the end points, the area under the receiver operating characteristic (ROC) curves for 4 different models were compared. In conclusion, prospective validation and use of these models as adjuncts to clinical decision making in the management of patients being considered for re-OLT are warranted.

Validation of self-reported cigarette smoking in a remote Australian Aboriginal community

Objective: To examine the relationship between self-reported tobacco smoking and urinary cotinine concentrations in the setting of a remote Aboriginal community. Methods: In a remote Northern Territory (NT) Aboriginal community the relationship between self-reported tobacco smoking and urinary cotinine concentrations was examined as part of a cross-sectional survey of cardiovascular risk factors. Current tobacco smoking was assessed as part of a questionnaire. The concentration of cotinine and cotinine/creatinine ratio (CCR) in a spot urine sample were used as a biochemical marker of nicotine exposure. Results: A total of 237 people took part in the survey, although completed questionnaires and urine results were available for 184 people. Current tobacco smoking was reported by 161 (69 [95% Cl 63 to 75]%) people, with higher rates among males (84/104, 81 [95% Cl 72 to 88]%) than females (77/129, 60 [95% Cl 51 to 68]%, p

Psychometric properties of the drinking expectancy questionnaire: A review of the factor structure and a proposed new scoring method

Objective: Expectancies about the outcomes of alcohol consumption are widely accepted as important determinants of drinking. This construct is increasingly recognized as a significant element of psychological interventions for alcohol-related problems. Much effort has been invested in producing reliable and valid instruments to measure this construct for research and clinical purposes, but very few have had their factor structure subjected to adequate validation. Among them, the Drinking Expectancies Questionnaire (DEQ) was developed to address some theoretical and design issues with earlier expectancy scales. Exploratory factor analyses, in addition to validity and reliability analyses, were performed when the original questionnaire was developed. The object of this study was to undertake a confirmatory analysis of the factor structure of the DEQ. Method: Confirmatory factor analysis through LISREL 8 was performed using a randomly split sample of 679 drinkers. Results: Results suggested that a new 5-factor model, which differs slightly from the original 6-factor version, was a more robust measure of expectancies. A new method of scoring the DEQ consistent with this factor structure is presented. Conclusions: The present study shows more robust psychometric properties of the DEQ using the new factor structure.