Completeness of Follow-Up Determines Validity of Study Findings: Results of a Prospective Repeated Measures Cohort Study.

BACKGROUND Current reporting guidelines do not call for standardised declaration of follow-up completeness, although study validity depends on the representativeness of measured outcomes. The Follow-Up Index (FUI) describes follow-up completeness at a given study end date as ratio between the investigated and the potential follow-up period. The association between FUI and the accuracy of survival-estimates was investigated. METHODS FUI and Kaplan-Meier estimates were calculated twice for 1207 consecutive patients undergoing aortic repair during an 11-year period: in a scenario A the population's clinical routine follow-up data (available from a prospective registry) was analysed conventionally. For the control scenario B, an independent survey was completed at the predefined study end. To determine the relation between FUI and the accuracy of study findings, discrepancies between scenarios regarding FUI, follow-up duration and cumulative survival-estimates were evaluated using multivariate analyses. RESULTS Scenario A noted 89 deaths (7.4%) during a mean considered follow-up of 30±28months. Scenario B, although analysing the same study period, detected 304 deaths (25.2%, P<0.001) as it scrutinized the complete follow-up period (49±32months). FUI (0.57±0.35 versus 1.00±0, P<0.001) and cumulative survival estimates (78.7% versus 50.7%, P<0.001) differed significantly between scenarios, suggesting that incomplete follow-up information led to underestimation of mortality. Degree of follow-up completeness (i.e. FUI-quartiles and FUI-intervals) correlated directly with accuracy of study findings: underestimation of long-term mortality increased almost linearly by 30% with every 0.1 drop in FUI (adjusted HR 1.30; 95%-CI 1.24;1.36, P<0.001). CONCLUSION Follow-up completeness is a pre-requisite for reliable outcome assessment and should be declared systematically. FUI represents a simple measure suited as reporting standard. Evidence lacking such information must be challenged as potentially flawed by selection bias.

Validity of SYNTAX score II for risk stratification of percutaneous coronary interventions: A patient-level pooled analysis of 5,433 patients enrolled in contemporary coronary stent trials.

OBJECTIVES To assess the clinical profile and long-term mortality in SYNTAX score II based strata of patients who received percutaneous coronary interventions (PCI) in contemporary randomized trials. BACKGROUND The SYNTAX score II was developed in the randomized, all-comers' SYNTAX trial population and is composed by 2 anatomical and 6 clinical variables. The interaction of these variables with the treatment provides individual long-term mortality predictions if a patient undergoes coronary artery bypass grafting (CABG) or PCI. METHODS Patient-level (n=5433) data from 7 contemporary coronary drug-eluting stent (DES) trials were pooled. The mortality for CABG or PCI was estimated for every patient. The difference in mortality estimates for these two revascularization strategies was used to divide the patients into three groups of theoretical treatment recommendations: PCI, CABG or PCI/CABG (the latter means equipoise between CABG and PCI for long term mortality). RESULTS The three groups had marked differences in their baseline characteristics. According to the predicted risk differences, 5115 patients could be treated either by PCI or CABG, 271 should be treated only by PCI and, rarely, CABG (n=47) was recommended. At 3-year follow-up, according to the SYNTAX score II recommendations, patients recommended for CABG had higher mortality compared to the PCI and PCI/CABG groups (17.4%; 6.1% and 5.3%, respectively; P<0.01). CONCLUSIONS The SYNTAX score II demonstrated capability to help in stratifying PCI procedures.

Comparing the factorial structure, invariance, and predictive validity of transtheoretical model constructs for alcohol use across restricted and unrestricted settings

With substance abuse treatment expanding in prisons and jails, understanding how behavior change interacts with a restricted setting becomes more essential. The Transtheoretical Model (TTM) has been used to understand intentional behavior change in unrestricted settings, however, evidence indicates restrictive settings can affect the measurement and structure of the TTM constructs. The present study examined data from problem drinkers at baseline and end-of-treatment from three studies: (1) Project CARE (n = 187) recruited inmates from a large county jail; (2) Project Check-In (n = 116) recruited inmates from a state prison; (3) Project MATCH, a large multi-site alcohol study had two recruitment arms, aftercare (n = 724 pre-treatment and 650 post-treatment) and outpatient (n = 912 pre-treatment and 844 post-treatment). The analyses were conducted using cross-sectional data to test for non-invariance of measures of the TTM constructs: readiness, confidence, temptation, and processes of change (Structural Equation Modeling, SEM) across restricted and unrestricted settings. Two restricted (jail and aftercare) and one unrestricted group (outpatient) entering treatment and one restricted (prison) and two unrestricted groups (aftercare and outpatient) at end-of-treatment were contrasted. In addition TTM end-of-treatment profiles were tested as predictors of 12 month drinking outcomes (Profile Analysis). Although SEM did not indicate structural differences in the overall TTM construct model across setting types, there were factor structure differences on the confidence and temptation constructs at pre-treatment and in the factor structure of the behavioral processes at the end-of-treatment. For pre-treatment temptation and confidence, differences were found in the social situations factor loadings and in the variance for the confidence and temptation latent factors. For the end-of-treatment behavioral processes, differences across the restricted and unrestricted settings were identified in the counter-conditioning and stimulus control factor loadings. The TTM end-of-treatment profiles were not predictive of drinking outcomes in the prison sample. Both pre and post-treatment differences in structure across setting types involved constructs operationalized with behaviors that are limited for those in restricted settings. These studies suggest the TTM is a viable model for explicating addictive behavior change in restricted settings but calls for modification of subscale items that refer to specific behaviors and caution in interpreting the mean differences across setting types for problem drinkers. ^

Evidence of reliability and validity of IPAQ for Mexican-American adults

Physical activity is a key component of life-style modification process which helps to reduce the risk of developing chronic diseases. It is important to have accurate estimates of physical activity to identify sedentary populations where interventions might be helpful. The International Physical Activity Questionnaire (IPAQ) short version has been used to estimate physical activity in diverse populations. However, there is little literature depicting the use of the IPAQ short version in Mexican America population. This study addressed the predictive validity and test-retest reliability of the IPAQ short version in Mexican American adults. The analysis was performed on 97 participants enrolled in the Cameron County Hispanic Cohort. Individuals selected in this study were 18 years of age or older. The predictive validity was evaluated by studying the relationship between physical activity and biomarkers known to be correlated with physical activity, namely, TNF-α, Adiponectin, and HDL. Multiple linear regression analysis was performed to delineate predictive validity. To assess test-retest reliability, two IPAQ-short last seven days questionnaires were interviewer administered to the participants on the same day, approximately two hours apart. Test-Retest reliability of IPAQ was estimated by performing intraclass correlations between the readings at two different time periods. The study showed that the IPAQ – short version used in the above study had acceptable test-retest reliability in the Mexican American population. This study showed that the IPAQ – short version did not have acceptable predictive validity when looking at physical activity and TNF-α, Adiponectin, and HDL in this sample.^

An examination of the cross-cultural reliability and construct validity of a measure of loneliness

Loneliness is a pervasive, rather common experience in American culture, particularly notable among adolescents. However, the phenomenon is not well documented in the cross-cultural psychiatric literature. For psychiatric epidemiology to encompass a wide array of psychopathologic phenomena, it is important to develop useful measures to characterize and classify both non-clinical and clinical dysfunction in diverse subgroups and cultures.^ The goal of this research was to examine the cross-cultural reliability and construct validity of a scale designed to measure loneliness. The Roberts Loneliness Scale (RLS-8) was administered to 4,060 adolescents ages 10-19 years enrolled in high schools along either side of the Texas-Tamaulipas border region between the U.S. and Mexico. Data collected in 1988 from a study focusing on substance use and psychological distress among adolescents in these regions were used to examine the operating characteristics of the RLS-8. A sample stratified by nationality and language, age, gender, and grade was used for analysis.^ Results indicated that in general the RLS-8 has moderate reliability in the U.S. sample, but not in the Mexican sample. Validity analyses demonstrated that there was evidence for convergent validity of the RLS-8 in the U.S. sample, but none in the Mexican sample. Discriminant validity of the measures in neither sample could be established. Based on the factor structure of the RLS-8, two subscales were created and analyzed for construct validity. Evidence for convergent validity was established for both subscales in both national samples. However, the discriminant validity of the measure remains unsubstantiated in both national samples. Also, the dimensionality of the scale is unresolved.^ One primary goal for future cross-cultural research would be to develop and test better defined culture-specific models of loneliness within the two cultures. From such scientific endeavor, measures of loneliness can be developed or reconstructed to classify the phenomenon in the same manner across cultures. Since estimates of prevalence and incidence are contingent upon reliable and valid screening or diagnostic measures, this objective would serve as an important foundation for future psychiatric epidemiologic inquiry into loneliness. ^

AN INVESTIGATION OF THE RELIABILITY AND VALIDITY OF THE CENTER FOR EPIDEMIOLOGIC STUDIES DEPRESSION SCALE IN THREE ETHNIC GROUPS

Epidemiologic studies of mental disorder have called attention to the need for identifying untreated cases and to the inadequacies of the instruments available for this purpose. Accurate case ascertainment devices are the basis of sound epidemiology. Without these, neither case classification nor analytic studies of risk factors is possible.^ The purpose of this research was to examine the reliability and validity of an instrument designed to measure depressive symptoms in community populations--the Center for Epidemiologic Studies Depression Scale (CES-D Scale). Two particular foci of the study were whether or not the scale had the same statistical structure across three ethnic groups and whether or not the magnitude and pattern of rates of symptoms for these groups were affected by one source of response error, that due to response tendencies. The effects of age and education on the pattern and magnitude of rates also were examined. In addition, the reliability and validity of the measures of response tendencies were assessed.^ The study population consisted of residents of Alameda County, California. A stratified sample of approximately 700 whites, blacks and Mexican-Americans was interviewed in the summer and fall of 1978.^ The results of the analysis indicated that the scale was reliable and measured a similar content domain across the three ethnic groups. The unadjusted sex- and ethnic-specific rates of depressive symptoms showed an ethnic pattern for both sexes: rates for whites were lowest, those for Mexican-Americans were highest, and those for blacks were intermediate. Measures of response tendencies--need for social approval, trait desirability, and acquiescence--affected the magnitude of the rates for most comparisons. Likewise, the pattern of rates changed somewhat from that originally observed. The one fairly consistent observation was that rates for Mexican-American women were higher than those for the other two female subgroups in most of the comparisons. These results must be considered in the context of the reliability and validity assessment of the measures of response tendencies which indicated the tenuousness of these measures.^ Age affected the ethnic pattern of rates for men in an inconsistent way; for women, Mexican-Americans continued to have higher rates than whites or blacks in all age categories. Education affected the magnitude of rates for women but not for men. For both men and women, Mexican-Americans had higher rates in all educational strata. Rates for women showed an inverse association with education while those for men did not. ^

Validity of using item response theory to analyze data from the Work Limitations Questionnaire

The Work Limitations Questionnaire (WLQ) is used to determine the amount of work loss and productivity which stem from certain health conditions, including rheumatoid arthritis and cancer. The questionnaire is currently scored using methodology from Classical Test Theory. Item Response Theory, on the other hand, is a theory based on analyzing item responses. This study wanted to determine the validity of using Item Response Theory (IRT), to analyze data from the WLQ. Item responses from 572 employed adults with dysthymia, major depressive disorder (MDD), double depressive disorder (both dysthymia and MDD), rheumatoid arthritis and healthy individuals were used to determine the validity of IRT (Adler et al., 2006).^ PARSCALE, which is IRT software from Scientific Software International, Inc., was used to calculate estimates of the work limitations based on item responses from the WLQ. These estimates, also known as ability estimates, were then correlated with the raw score estimates calculated from the sum of all the items responses. Concurrent validity, which claims a measurement is valid if the correlation between the new measurement and the valid measurement is greater or equal to .90, was used to determine the validity of IRT methodology for the WLQ. Ability estimates from IRT were found to be somewhat highly correlated with the raw scores from the WLQ (above .80). However, the only subscale which had a high enough correlation for IRT to be considered valid was the time management subscale (r = .90). All other subscales, mental/interpersonal, physical, and output, did not produce valid IRT ability estimates.^ An explanation for these lower than expected correlations can be explained by the outliers found in the sample. Also, acquiescent responding (AR) bias, which is caused by the tendency for people to respond the same way to every question on a questionnaire, and the multidimensionality of the questionnaire (the WLQ is composed of four dimensions and thus four different latent variables) probably had a major impact on the IRT estimates. Furthermore, it is possible that the mental/interpersonal dimension violated the monotonocity assumption of IRT causing PARSCALE to fail to run for these estimates. The monotonicity assumption needs to be checked for the mental/interpersonal dimension. Furthermore, the use of multidimensional IRT methods would most likely remove the AR bias and increase the validity of using IRT to analyze data from the WLQ.^

Predictive validity of the personal background and preparation survey and effectiveness of targeted interventions among health science community college students

Existing data, collected from 1st-year students enrolled in a major Health Science Community College in the south central United States, for Fall 2010, Spring 2011, Fall 2011 and Spring 2012 semesters as part of the "Online Navigational Assessment Vehicle, Intervention Guidance, and Targeting of Risks (NAVIGATOR) for Undergraduate Minority Student Success" with CPHS approval number HSC-GEN-07-0158, was used for this thesis. The Personal Background and Preparation Survey (PBPS) and a two-question risk self-assessment subscale were administered to students during their 1st-year orientation. The PBPS total risk score, risk self-assessment total and overall scores, and Under Representative Minority Student (URMS) status were recorded. The purpose of this study is to evaluate and report the predictive validity of the indicators identified above for Adverse Academic Status Events (AASE) and Nonadvancement Adverse Academic Status Events (NAASE) as well as the effectiveness of interventions targeted using the PBPS among a diverse population of health science community college students. The predictive validity of the PBPS for AASE has previously been demonstrated among health science professions and graduate students (Johnson, Johnson, Kim, & McKee, 2009a; Johnson, Johnson, McKee, & Kim, 2009b). Data will be analyzed using binary logistic regression and correlation using SPSS 19 statistical package. Independent variables will include baseline- versus intervention-year treatments, PBPS, risk self-assessment, and URMS status. The dependent variables will be binary AASE and NAASE status. ^ The PBPS was the first reliable diagnostic and prescriptive instrument to establish documented predictive validity for student Adverse Academic Status Events (AASE) among students attending health science professional schools. These results extend the documented validity for the PBPS in predicting AASE to a health science community college student population. Results further demonstrated that interventions introduced using the PBPS were followed by approximately one-third reduction in the odds of Nonadvancement Adverse Academic Status Events (NAASE), controlling for URMS status and risk self-assessment scores. These results indicate interventions introduced using the PBPS may have potential to reduce AASE or attrition among URMS and nonURMS attending health science community colleges on a broader scale; positively impacting costs, shortages, and diversity of health science professionals.^