How to assess the external validity of therapeutic trials: a conceptual approach

Background External validity of study results is an important issue from a clinical point of view. From a methodological point of view, however, the concept of external validity is more complex than it seems to be at first glance. Methods Methodological review to address the concept of external validity. Results External validity refers to the question whether results are generalizable to persons other than the population in the original study. The only formal way to establish the external validity would be to repeat the study for that specific target population. We propose a three-way approach for assessing the external validity for specified target populations. (i) The study population might not be representative for the eligibility criteria that were intended. It should be addressed whether the study population differs from the intended source population with respect to characteristics that influence outcome. (ii) The target population will, by definition, differ from the study population with respect to geographical, temporal and ethnical conditions. Pondering external validity means asking the question whether these differences may influence study results. (iii) It should be assessed whether the study's conclusions can be generalized to target populations that do not meet all the eligibility criteria. Conclusion Judging the external validity of study results cannot be done by applying given eligibility criteria to a single target population. Rather, it is a complex reflection in which prior knowledge, statistical considerations, biological plausibility and eligibility criteria all have place.

Reproducibility and validity of video screen measurements of gait in children with spastic cerebral palsy

PURPOSE: To determine the reproducibility and validity of video screen measurement (VSM) of sagittal plane joint angles during gait. METHODS: 17 children with spastic cerebral palsy walked on a 10m walkway. Videos were recorded and 3d-instrumented gait analysis was performed. Two investigators measured six sagittal joint/segment angles (shank, ankle, knee, hip, pelvis, and trunk) using a custom-made software package. The intra- and interrater reproducibility were expressed by the intraclass correlation coefficient (ICC), standard error of measurements (SEM) and smallest detectable difference (SDD). The agreement between VSM and 3d joint angles was illustrated by Bland-Altman plots and limits of agreement (LoA). RESULTS: Regarding the intrarater reproducibility of VSM, the ICC ranged from 0.99 (shank) to 0.58 (trunk), the SEM from 0.81 degrees (shank) to 5.97 degrees (trunk) and the SDD from 1.80 degrees (shank) to 16.55 degrees (trunk). Regarding the interrater reproducibility, the ICC ranged from 0.99 (shank) to 0.48 (trunk), the SEM from 0.70 degrees (shank) to 6.78 degrees (trunk) and the SDD from 1.95 degrees (shank) to 18.8 degrees (trunk). The LoA between VSM and 3d data ranged from 0.4+/-13.4 degrees (knee extension stance) to 12.0+/-14.6 degrees (ankle dorsiflexion swing). CONCLUSION: When performed by the same observer, VSM mostly allows the detection of relevant changes after an intervention. However, VSM angles differ from 3d-IGA and do not reflect the real sagittal joint position, probably due to the additional movements in the other planes.

Validity of an automatic measure protocol in distal femur for allograft selection from a three-dimensional virtual bone bank system

Osteoarticular allograft is one possible treatment in wide surgical resections with large defects. Performing best osteoarticular allograft selection is of great relevance for optimal exploitation of the bone databank, good surgery outcome and patient’s recovery. Current approaches are, however, very time consuming hindering these points in practice. We present a validation study of a software able to perform automatic bone measurements used to automatically assess the distal femur sizes across a databank. 170 distal femur surfaces were reconstructed from CT data and measured manually using a size measure protocol taking into account the transepicondyler distance (A), anterior-posterior distance in medial condyle (B) and anterior-posterior distance in lateral condyle (C). Intra- and inter-observer studies were conducted and regarded as ground truth measurements. Manual and automatic measures were compared. For the automatic measurements, the correlation coefficients between observer one and automatic method, were of 0.99 for A measure and 0.96 for B and C measures. The average time needed to perform the measurements was of 16 h for both manual measurements, and of 3 min for the automatic method. Results demonstrate the high reliability and, most importantly, high repeatability of the proposed approach, and considerable speed-up on the planning.

Reliability, construct and criterion validity of the KIDSCREEN-10 score: a short measure for children and adolescents' well-being and health-related quality of life

Background To assess the criterion and construct validity of the KIDSCREEN-10 well-being and health-related quality of life (HRQoL) score, a short version of the KIDSCREEN-52 and KIDSCREEN-27 instruments. Methods The child self-report and parent report versions of the KIDSCREEN-10 were tested in a sample of 22,830 European children and adolescents aged 8–18 and their parents (n = 16,237). Correlation with the KIDSCREEN-52 and associations with other generic HRQoL measures, physical and mental health, and socioeconomic status were examined. Score differences by age, gender, and country were investigated. Results Correlations between the 10-item KIDSCREEN score and KIDSCREEN-52 scales ranged from r = 0.24 to 0.72 (r = 0.27–0.72) for the self-report version (proxy-report version). Coefficients below r = 0.5 were observed for the KIDSCREEN-52 dimensions Financial Resources and Being Bullied only. Cronbach alpha was 0.82 (0.78), test–retest reliability was ICC = 0.70 (0.67) for the self- (proxy-)report version. Correlations between other children self-completed HRQoL questionnaires and KIDSCREEN-10 ranged from r = 0.43 to r = 0.63 for the KIDSCREEN children self-report and r = 0.22–0.40 for the KIDSCREEN parent proxy report. Known group differences in HRQoL between physically/mentally healthy and ill children were observed in the KIDSCREEN-10 self and proxy scores. Associations with self-reported psychosomatic complaints were r = −0.52 (−0.36) for the KIDSCREEN-10 self-report (proxy-report). Statistically significant differences in KIDSCREEN-10 self and proxy scores were found by socioeconomic status, age, and gender. Conclusions Our results indicate that the KIDSCREEN-10 provides a valid measure of a general HRQoL factor in children and adolescents, but the instrument does not represent well most of the single dimensions of the original KIDSCREEN-52. Test–retest reliability was slightly below a priori defined thresholds.

Remission in schizophrenia: validity, frequency, predictors, and patients' perspective 5 years later

In March 2005, the Remission in Schizophrenia Working Group (RSWG) proposed a consensus definition of symptomatic remission in schizophrenia and developed specific operational criteria for its assessment. They pointed out, however, that the validity and the relationship to other outcome dimensions required further examination. This article reviews studies on the validity, frequency, and predictors of symptomatic remission in schizophrenia and studies on patients' perspectives. These studies have demonstrated that the RSWG remission criteria appear achievable and sustainable for a significant proportion of patients, and are related to a better overall symptomatic status and functional outcome and, to a less clear extent, to a better quality of life and cognitive performance. However, achieving symptomatic remission is not automatically concurrent with an adequate status in other outcome dimensions. The results of the present review suggest that the RSWG remission criteria are valid and useful. As such, they should be consistently applied in clinical trials. However the lack of consensus definitions of functional remission and adequate quality of life hampers research on their predictive validity on these outcome dimensions. Future research should therefore search for criteria of these dimensions and test whether the RSWG remission criteria consistently predict a "good" outcome with respect to functioning and quality of life.

Measurement issues in trials of pediatric acute diarrheal diseases: a systematic review

BACKGROUND: Worldwide, diarrheal diseases rank second among conditions that afflict children. Despite the disease burden, there is limited consensus on how to define and measure pediatric acute diarrhea in trials. OBJECTIVES: In RCTs of children involving acute diarrhea as the primary outcome, we documented (1) how acute diarrhea and its resolution were defined, (2) all primary outcomes, (3) the psychometric properties of instruments used to measure acute diarrhea and (4) the methodologic quality of included trials, as reported. METHODS: We searched CENTRAL, Embase, Global Health, and Medline from inception to February 2009. English-language RCTs of children younger than 19 years that measured acute diarrhea as a primary outcome were chosen. RESULTS: We identified 138 RCTs reporting on 1 or more primary outcomes related to pediatric acute diarrhea/diseases. Included trials used 64 unique definitions of diarrhea, 69 unique definitions of diarrhea resolution, and 46 unique primary outcomes. The majority of included trials evaluated short-term clinical disease activity (incidence and duration of diarrhea), laboratory outcomes, or a composite of these end points. Thirty-two trials used instruments (eg, single and multidomain scoring systems) to support assessment of disease activity. Of these, 3 trials stated that their instrument was valid; however, none of the trials (or their citations) reported evidence of this validity. The overall methodologic quality of included trials was good. CONCLUSIONS: Even in what would be considered methodologically sound clinical trials, definitions of diarrhea, primary outcomes, and instruments employed in RCTs of pediatric acute diarrhea are heterogeneous, lack evidence of validity, and focus on indices that may not be important to participants.