Methods and representativeness of a European survey in children and adolescents: the KIDSCREEN study

BACKGROUND: The objective of the present study was to compare three different sampling and questionnaire administration methods used in the international KIDSCREEN study in terms of participation, response rates, and external validity. METHODS: Children and adolescents aged 8-18 years were surveyed in 13 European countries using either telephone sampling and mail administration, random sampling of school listings followed by classroom or mail administration, or multistage random sampling of communities and households with self-administration of the survey materials at home. Cooperation, completion, and response rates were compared across countries and survey methods. Data on non-respondents was collected in 8 countries. The population fraction (PF, respondents in each sex-age, or educational level category, divided by the population in the same category from Eurostat census data) and population fraction ratio (PFR, ratio of PF) and their corresponding 95% confidence intervals were used to analyze differences by country between the KIDSCREEN samples and a reference Eurostat population. RESULTS: Response rates by country ranged from 18.9% to 91.2%. Response rates were highest in the school-based surveys (69.0%-91.2%). Sample proportions by age and gender were similar to the reference Eurostat population in most countries, although boys and adolescents were slightly underrepresented (PFR <1). Parents in lower educational categories were less likely to participate (PFR <1 in 5 countries). Parents in higher educational categories were overrepresented when the school and household sampling strategies were used (PFR = 1.78-2.97). CONCLUSION: School-based sampling achieved the highest overall response rates but also produced slightly more biased samples than the other methods. The results suggest that the samples were sufficiently representative to provide reference population values for the KIDSCREEN instrument.

The KIDSCREEN-27 quality of life measure for children and adolescents: psychometric results from a cross-cultural survey in 13 European countries

OBJECTIVE: To assess the construct and criterion validity of the KIDSCREEN-27 health-related quality of life (HRQoL) questionnaire, a shorter version of the KIDSCREEN-52. METHODS: The five-dimensional KIDSCREEN-27 was tested in a sample of 22,827. For criterion validity the correlation with and the percentage explained variance of the scores of the KIDSCREEN-52 instrument were examined. Construct validity was assessed by testing a priori expected associations with other generic HRQoL measures (YQOL-S, PedsQL, CHIP), indicators of physical and mental health, and socioeconomic status. Age and gender differences were investigated. RESULTS: Correlation with corresponding scales of the KIDSCREEN-52 ranged from r = 0.63 to r = 0.96, and r2 ranged from 0.39 to 0.92. Correlations between other HRQoL questionnaires and KIDSCREEN-27 dimensions were moderate to high for those assessing similar constructs (r = 0.36 to 0.63). Statistically significant and sizeable differences between physically and mentally healthy and ill children were found in all KIDSCREEN-27 dimensions together with strong associations with psychosomatic complaints (r = -0.52). Most of the KIDSCREEN-27 dimensions showed a gradient according to socio-economic status, age and gender. CONCLUSIONS: The KIDSCREEN-27 seems to be a valid measure of HRQoL in children and adolescents. Further research is needed to assess longitudinal validity and sensitivity to change.

Testing the structural and cross-cultural validity of the KIDSCREEN-27 quality of life questionnaire

OBJECTIVES: The aim of this study is to assess the structural and cross-cultural validity of the KIDSCREEN-27 questionnaire. METHODS: The 27-item version of the KIDSCREEN instrument was derived from a longer 52-item version and was administered to young people aged 8-18 years in 13 European countries in a cross-sectional survey. Structural and cross-cultural validity were tested using multitrait multi-item analysis, exploratory and confirmatory factor analysis, and Rasch analyses. Zumbo's logistic regression method was applied to assess differential item functioning (DIF) across countries. Reliability was assessed using Cronbach's alpha. RESULTS: Responses were obtained from n = 22,827 respondents (response rate 68.9%). For the combined sample from all countries, exploratory factor analysis with procrustean rotations revealed a five-factor structure which explained 56.9% of the variance. Confirmatory factor analysis indicated an acceptable model fit (RMSEA = 0.068, CFI = 0.960). The unidimensionality of all dimensions was confirmed (INFIT: 0.81-1.15). Differential item functioning (DIF) results across the 13 countries showed that 5 items presented uniform DIF whereas 10 displayed non-uniform DIF. Reliability was acceptable (Cronbach's alpha = 0.78-0.84 for individual dimensions). CONCLUSIONS: There was substantial evidence for the cross-cultural equivalence of the KIDSCREEN-27 across the countries studied and the factor structure was highly replicable in individual countries. Further research is needed to correct scores based on DIF results. The KIDSCREEN-27 is a new short and promising tool for use in clinical and epidemiological studies.

Psychometric properties of the Swiss version of the Nursing Home Survey on Patient Safety Culture

The Nursing Home Survey on Patient Safety Culture (NHSPSC) was specifically developed for nursing homes to assess a facility’s safety climate and it consists of 12 dimensions. After its pilot testing, however, no fur- ther psychometric analyses were performed on the instrument. For this study of safety climate in Swiss nursing home units, the NHSPSC was linguistically adapted to the Swiss context and to address the unit as well as facility level, with the aim of testing aspects of the validity and reliability of the Swiss version before its use in Swiss nursing home units. Psychometric analyses were performed on data from 367 nurs- ing personnel from nine nursing homes in the German-speaking part of Switzerland (response rate = 66%), and content validity (CVI) examined. The statistical influence of unit membership on respondents’ answers, and on their agreement concerning their units’ safety climate, was tested using intraclass corre- lation coefficients (ICCs) and the rWG(J) interrater agreement index. A multilevel exploratory factor analysis (MEFA) with oblimin rotation was applied to examine the questionnaire’s dimensionality. Cronbach’s alpha and Raykov’s rho were calculated to assess factor reliability. The relationship of safety climate dimensions with clinical outcomes was explored. Expert feedback confirmed the relevance of the instru- ment’s items (CVI = 0.93). Personnel showed strong agreement in their perceptions in three dimensions of the questionnaire. ICCs supported a multilevel analysis. MEFA produced nine factors at the within-level (in comparison to 12 in the original version) and two factors at the between-level with satisfactory fit statis- tics. Raykov’s Rho for the single level factors ranged between 0.67 and 0.86. Some safety climate dimen- sions show moderate, but non-significant correlations with the use of bedrails, physical restraint use, and fall-related injuries. The Swiss version of the NHSPSC needs further refinement and testing before its use can be recommended in Swiss nursing homes: its dimensionality needs further clarification, particularly to distinguish items addressing the unit-level safety climate from those at the facility level.

First results on the mandatory notification of dog bite injuries in Switzerland: a questionnaire survey among physicians and veterinarians

In order to expedite targeted interventions, mandatory notification of treated dog bite injuries for Swiss physicians and veterinarians was implemented in 2006. Since the number of notified cases was much lower than expected, the validity of the annually produced statistics with respect to the real situation in Switzerland was discussed controversially. In this study a questionnaire survey among physicians and veterinarians was carried out to evaluate the amount of and reasons for non-compliance with the mandatory notification. 81 % of the physicians and 97 % of the veterinarians reported the treatment of dog bite injuries in the year 2009. Among those, 60 % of the physicians and 41 % of the veterinarians indicated notification of less than 50 % of the treated cases. Our results indicate that the most relevant cases for targeted interventions seem to be notified, but that the statistical analyses have to be interpreted carefully.

Self-Reported Capabilities Among Young Male Adults in Switzerland: Translation and Psychometric Evaluation of a German, French and Italian Version of a Closed Survey Instrument

There is a shortage of empirical applications of the capability approach that employ closed survey instruments to assess self-reported capabilities. However, for those few instruments that have been designed and administered through surveys until now, no psychometric properties (reliability, validity, and factor structure) were reported. The purpose of this study is the assessment of the psychometric properties of three new language versions (German, French, and Italian) of an established (English) set of eight self-reported capability items. The set of items is taken from a previously published British study by Anand and van Hees (J Soc Econ 35(2):268–284, 2006). Our sample consists of 17,152 young male adults aged 18–25 years from the three major language regions in Switzerland. The results indicate good reliability of the three language versions. The results from the exploratory factor analyses suggest a one-dimensional factor structure for seven domain specific items. Furthermore, the results from multiple regression analyses suggest that a global summary item on overall capabilities represents a measurement alternative to the set of seven domain specific capability items. Finally, the results confirm the applicability of the closed capability instrument in a large scale survey questionnaire and represent the first attempt to measure self-reported capabilities in Switzerland.

SwissScoring--a nationwide survey of NEMS assessing practices and its accuracy.

BACKGROUND The nine equivalents of nursing manpower use score (NEMS) is used to evaluate critical care nursing workload and occasionally to define hospital reimbursements. Little is known about the caregivers' accuracy in scoring, about factors affecting this accuracy and how validity of scoring is assured. METHODS Accuracy in NEMS scoring of Swiss critical care nurses was assessed using case vignettes. An online survey was performed to assess training and quality control of NEMS scoring and to collect structural and organizational data of participating intensive care units (ICUs). Aggregated structural and procedural data of the Swiss ICU Minimal Data Set were used for matching. RESULTS Nursing staff from 64 (82%) of the 78 certified adult ICUs participated in this survey. Training and quality control of scoring shows large variability between ICUs. A total of 1378 nurses scored one out of 20 case vignettes: accuracy ranged from 63.7% (intravenous medications) to 99.1% (basic monitoring). Erroneous scoring (8.7% of all items) was more frequent than omitted scoring (3.2%). Mean NEMS per case was 28.0 ± 11.8 points (reference score: 25.7 ± 14.2 points). Mean bias was 2.8 points (95% confidence interval: 1.0-4.7); scores below 37.1 points were generally overestimated. Data from units with a greater nursing management staff showed a higher bias. CONCLUSION Overall, nurses assess the NEMS score within a clinically acceptable range. Lower scores are generally overestimated. Inaccurate assessment was associated with a greater size of the nursing management staff. Swiss head nurses consider themselves motivated to assure appropriate scoring and its validation.

Safety climate in Swiss hospital units: Swiss version of the Safety Climate Survey

RATIONALE, AIMS AND OBJECTIVES Safety climate measurements are a broadly used element of improvement initiatives. In order to provide a sound and easy-to-administer instrument for the use in Swiss hospitals, we translated the Safety Climate Survey into German and French. METHODS After translating the Safety Climate Survey into French and German, a cross-sectional survey study was conducted with health care professionals (HCPs) in operating room (OR) teams and on OR-related wards in 10 Swiss hospitals. Validity of the instrument was examined by means of Cronbach's alpha and missing rates of the single items. Item-descriptive statistics group differences and percentage of 'problematic responses' (PPR) were calculated. RESULTS 3153 HCPs completed the survey (response rate: 63.4%). 1308 individuals were excluded from the analyses because of a profession other than doctor or nurse or invalid answers (n = 1845; nurses = 1321, doctors = 523). Internal consistency of the translated Safety Climate Survey was good (Cronbach's alpha G erman  = 0.86; Cronbach's alpha F rench  = 0.84). Missing rates at item level were rather low (0.23-4.3%). We found significant group differences in safety climate values regarding profession, managerial function, work area and time spent in direct patient care. At item level, 14 out of 21 items showed a PPR higher than 10%. CONCLUSIONS Results indicate that the French and German translations of the Safety Climate Survey might be a useful measurement instrument for safety climate in Swiss hospital units. Analyses at item level allow for differentiating facets of safety climate into more positive and critical safety climate aspects.

Completeness of Follow-Up Determines Validity of Study Findings: Results of a Prospective Repeated Measures Cohort Study.

BACKGROUND Current reporting guidelines do not call for standardised declaration of follow-up completeness, although study validity depends on the representativeness of measured outcomes. The Follow-Up Index (FUI) describes follow-up completeness at a given study end date as ratio between the investigated and the potential follow-up period. The association between FUI and the accuracy of survival-estimates was investigated. METHODS FUI and Kaplan-Meier estimates were calculated twice for 1207 consecutive patients undergoing aortic repair during an 11-year period: in a scenario A the population's clinical routine follow-up data (available from a prospective registry) was analysed conventionally. For the control scenario B, an independent survey was completed at the predefined study end. To determine the relation between FUI and the accuracy of study findings, discrepancies between scenarios regarding FUI, follow-up duration and cumulative survival-estimates were evaluated using multivariate analyses. RESULTS Scenario A noted 89 deaths (7.4%) during a mean considered follow-up of 30±28months. Scenario B, although analysing the same study period, detected 304 deaths (25.2%, P<0.001) as it scrutinized the complete follow-up period (49±32months). FUI (0.57±0.35 versus 1.00±0, P<0.001) and cumulative survival estimates (78.7% versus 50.7%, P<0.001) differed significantly between scenarios, suggesting that incomplete follow-up information led to underestimation of mortality. Degree of follow-up completeness (i.e. FUI-quartiles and FUI-intervals) correlated directly with accuracy of study findings: underestimation of long-term mortality increased almost linearly by 30% with every 0.1 drop in FUI (adjusted HR 1.30; 95%-CI 1.24;1.36, P<0.001). CONCLUSION Follow-up completeness is a pre-requisite for reliable outcome assessment and should be declared systematically. FUI represents a simple measure suited as reporting standard. Evidence lacking such information must be challenged as potentially flawed by selection bias.

MTurk Survey on "Mood and Personality". Documentation

Social desirability and the fear of negative consequences often deter a considerable share of survey respondents from responding truthfully to sensitive questions. Thus, resulting prevalence estimates are biased. Indirect techniques for surveying sensitive questions such as the Randomized Response Technique are intended to mitigate misreporting by providing complete concealment of individual answers. However, it is far from clear whether these indirect techniques actually produce more valid measurements than standard direct questioning. In order to evaluate the validity of different sensitive question techniques we carried out an online validation experiment at Amazon Mechanical Turk in which respondents' self-reports of norm-breaking behavior (cheating in dice games) were validated against observed behavior. This document describes the design of the validation experiment and provides details on the questionnaire, the different sensitive question technique implementations, the field work, and the resulting dataset. The appendix contains a codebook of the data and facsimiles of the questionnaire pages and other survey materials.