Student's academic efficacy or Inefficacy? an example on how to evaluate the psychometric properties of a measuring instrument and evaluate the effects of item wording

This study evaluated the effect of item inversion on the construct validity and reliability of psychometric scales and proposed a theoretical framework for the evaluation of the psychometric properties of data gathered with psychometric instruments. To this propose, we used the Maslach Burnout Inventory, which is the most used psychometric inventory to measure burnout in different professional context (Students, Teachers, Police, Doctors, Nurses, etc…). The version of the MBI used was the MBI-Student Survey (MBI-SS). This inventory is composed of three key dimensions: Exhaustion, Cynicism and Professional Efficacy. The two first dimensions—which have positive formulated items—are moderate to strong positive correlated, and show moderate to strong negative correlations with the 3rd dimension—which has negative formulated items. We tested the hypothesis that, in college students, formulating the 3rd dimension of burnout as Inefficacy (reverting the negatively worded items in the Efficacy dimension) improves the correlation of the 3rd dimension with the other two dimensions, improves its internal consistency, and the overall MBI-SS’ construct validity and reliability. Confirmatory factor analysis results, estimated by Maximum Likelihood, revealed adequate factorial fit for both forms of the MBI-SS (with Efficacy) vs. the MBI-SSi (with Inefficacy). Also both forms showed adequate convergent and discriminant related validity. However, reliability and convergent validity were higher for the MBI-SSi. There were also stronger (positive) correlations between the 3 factors in MBI-SSi than the ones observed in MBI-SS. Results show that positively rewording of the 3rd dimension of the MBI-SS improves its validity and reliability. We therefore propose that the 3rd dimension of the MBI-SS should be named Professional Inefficacy and its items should be positively worded.

Cross-cultural adaptation and psychometric properties of the Brazilian-Portuguese version of the Duke Anticoagulation Satisfaction Scale

Aim. The aim of this study was to evaluate the internal reliability and validity of the BrazilianPortuguese version of Duke Anticoagulation Satisfaction Scale (DASS) among cardiovascular patients. Background. Oral anticoagulation is widely used to prevent and treat thromboembolic events in several conditions, especially in cardiovascular diseases; however, this therapy can induce dissatisfaction and reduce the quality of life. Design. Methodological and cross-sectional research design. Methods. The cultural adaptation of the DASS included the translation and back-translation, discussions with healthcare professionals and patients to ensure conceptual equivalence, semantic evaluation and instrument pretest. The BrazilianPortuguese version of the DASS was tested among subjects followed in a university hospital anticoagulation outpatient clinic. The psychometric properties were assessed by construct validity (convergent, known groups and dimensionality) and internal consistency/reliability (Cronbachs alpha). Results. A total of 180 subjects under oral anticoagulation formed the baseline validation population. DASS total score and SF-36 domain correlations were moderate for General health (r = -0.47, p < 0.01), Vitality (r = -0.44, p < 0.01) and Mental health (r = -0.42, p < 0.01) (convergent). Age and length on oral anticoagulation therapy (in years) were weakly correlated with total DASS score and most of the subscales, except Limitation (r = -0.375, p < 0.01) (Known groups). The Cronbachs alpha coefficient was 0.79 for the total scale, and it ranged from 0.76 (hassles and burdens)0.46 (psychological impact) among the domains, confirming the internal consistency reliability. Conclusions. The BrazilianPortuguese version of the DASS has shown levels of reliability and validity comparable with the original English version. Relevance to clinical practice. Healthcare practitioners and researchers need internationally validated measurement tools to compare outcomes of interventions in clinical management and research tools in oral anticoagulation therapy.

Cross-cultural adaptation and psychometric properties of the Brazilian version of the scale of oral health outcomes for 5-year-old children (SOHO-5)

Abstract Background Most of the instruments available to measure the oral health-related quality of life (OHRQoL) in paediatric populations focus on older children, whereas parental reports are used for very young children. The scale of oral health outcomes for 5-year-old children (SOHO-5) assesses the OHRQoL of very young children through self-reports and parental proxy reports. We aimed to cross-culturally adapt the SOHO-5 to the Brazilian Portuguese language and to assess its reliability and validity. Findings We tested the quality of the cross-cultural adaptation in 2 pilot studies with 40 children aged 5–6 years and their parents. The measurement was tested for reliability and validity on 193 children that attended the paediatric dental screening program at the University of São Paulo. The children were also clinically examined for dental caries. The internal consistency was demonstrated by a Cronbach's alpha coefficient of 0.90 for the children’s self-reports and 0.77 for the parental proxy reports. The test-retest reliability results, which were based on repeated administrations on 159 children, were excellent; the intraclass correlation coefficient was 0.98 for parental and 0.92 for child reports. In general, the construct validity was satisfactory and demonstrated consistent and strong associations between the SOHO-5 and different subjective global ratings of oral health, perceived dental treatment need and overall well-being in both the parental and children’s versions (p < 0.001). The SOHO-5 was also able to clearly discriminate between children with and without a history of dental caries (mean scores: 5.8 and 1.1, respectively; p < 0.001). Conclusion The present study demonstrated that the SOHO-5 exhibits satisfactory psychometric properties and is applicable to 5- to 6-year-old children in Brazil.

Psychometric properties of the Spanish version of the mindful attention awareness scale (MAAS) in patients with fibromyalgia

Background Mindful-based interventions improve functioning and quality of life in fibromyalgia (FM) patients. The aim of the study is to perform a psychometric analysis of the Spanish version of the Mindful Attention Awareness Scale (MAAS) in a sample of patients diagnosed with FM. Methods The following measures were administered to 251 Spanish patients with FM: the Spanish version of MAAS, the Chronic Pain Acceptance Questionnaire, the Pain Catastrophising Scale, the Injustice Experience Questionnaire, the Psychological Inflexibility in Pain Scale, the Fibromyalgia Impact Questionnaire and the Euroqol. Factorial structure was analysed using Confirmatory Factor Analyses (CFA). Cronbach's α coefficient was calculated to examine internal consistency, and the intraclass correlation coefficient (ICC) was calculated to assess the test-retest reliability of the measures. Pearson’s correlation tests were run to evaluate univariate relationships between scores on the MAAS and criterion variables. Results The MAAS scores in our sample were low (M = 56.7; SD = 17.5). CFA confirmed a two-factor structure, with the following fit indices [sbX2 = 172.34 (p < 0.001), CFI = 0.95, GFI = 0.90, SRMR = 0.05, RMSEA = 0.06. MAAS was found to have high internal consistency (Cronbach’s α = 0.90) and adequate test-retest reliability at a 1–2 week interval (ICC = 0.90). It showed significant and expected correlations with the criterion measures with the exception of the Euroqol (Pearson = 0.15). Conclusion Psychometric properties of the Spanish version of the MAAS in patients with FM are adequate. The dimensionality of the MAAS found in this sample and directions for future research are discussed.

Perceived neighborhood problems: multilevel analysis to evaluate psychometric properties in a Southern adult Brazilian population

Abstract Background Physical attributes of the places in which people live, as well as their perceptions of them, may be important health determinants. The perception of place in which people dwell may impact on individual health and may be a more telling indicator for individual health than objective neighborhood characteristics. This paper aims to evaluate psychometric and ecometric properties of a scale on the perceptions of neighborhood problems in adults from Florianopolis, Southern Brazil. Methods Individual, census tract level (per capita monthly familiar income) and neighborhood problems perception (physical and social disorders) variables were investigated. Multilevel models (items nested within persons, persons nested within neighborhoods) were run to assess ecometric properties of variables assessing neighborhood problems. Results The response rate was 85.3%, (1,720 adults). Participants were distributed in 63 census tracts. Two scales were identified using 16 items: Physical Problems and Social Disorder. The ecometric properties of the scales satisfactory: 0.24 to 0.28 for the intra-class correlation and 0.94 to 0.96 for reliability. Higher values on the scales of problems in the physical and social domains were associated with younger age, more length of time residing in the same neighborhood and lower census tract income level. Conclusions The findings support the usefulness of these scales to measure physical and social disorder problems in neighborhoods.

Psychometric properties of the Beck Depression Inventory-II: a comprehensive review

OBJECTIVE: To review the psychometric properties of the Beck Depression Inventory-II (BDI-II) as a self-report measure of depression in a variety of settings and populations. METHODS: Relevant studies of the BDI-II were retrieved through a search of electronic databases, a hand search, and contact with authors. Retained studies (k = 118) were allocated into three groups: non-clinical, psychiatric/institutionalized, and medical samples. RESULTS: The internal consistency was described as around 0.9 and the retest reliability ranged from 0.73 to 0.96. The correlation between BDI-II and the Beck Depression Inventory (BDI-I) was high and substantial overlap with measures of depression and anxiety was reported. The criterion-based validity showed good sensitivity and specificity for detecting depression in comparison to the adopted gold standard. However, the cutoff score to screen for depression varied according to the type of sample. Factor analysis showed a robust dimension of general depression composed by two constructs: cognitive-affective and somatic-vegetative. CONCLUSIONS: The BDI-II is a relevant psychometric instrument, showing high reliability, capacity to discriminate between depressed and non-depressed subjects, and improved concurrent, content, and structural validity. Based on available psychometric evidence, the BDI-II can be viewed as a cost-effective questionnaire for measuring the severity of depression, with broad applicability for research and clinical practice worldwide.

Working Alliance Inventory-Short Revised (WAI-SR): psychometric properties in outpatients and inpatients

The Working Alliance Inventory-Short Revised (WAI-SR) is a recently refined measure of the therapeutic alliance that assesses three key aspects of the therapeutic alliance: (a) agreement on the tasks of therapy, (b) agreement on the goals of therapy and (c) development of an affective bond. The WAI-SR demonstrated good psychometric properties in an initial validation in psychotherapy outpatients in the USA. The generalizability of these findings is limited because in some countries a substantial portion of individual psychotherapy is delivered in inpatient settings. This study investigated and compared the psychometric properties of the WAI-SR in German outpatients (N = 88) and inpatients (N = 243). In both samples reliability (alpha > 0.80) and convergent validity with the Helping Alliance Questionnaire were good (r > 0.64). Confirmatory factor analysis showed acceptable to good model fit for the proposed Bond-Task-Goal model in both samples. Multi-group analysis demonstrated that the same constructs were measured across settings. Alliance ratings of outpatients and inpatients differed regarding the overlap of alliance components and the magnitude of the alliance ratings: The differentiation of the alliance components was poorer in inpatients and they reported lower alliances. Unique aspects of the alliance in inpatient treatment are discussed and a need for further research on the alliance in inpatient settings is pointed out. Overall, the WAI-SR can be recommended for alliance assessment in both settings.

Psychometric properties of the MacNew heart disease health-related quality of life instrument in patients with heart failure

RATIONALE, AIMS AND OBJECTIVES: Heart failure (HF) is a severe chronic disease and impairs health-related quality of life (HRQL). While validated specific HRQL instruments are required for evaluation of treatment and rehabilitation in patients with HF, a single validated measure to document changes in HRQL for patients with different heart disease diagnoses would be invaluable. The purpose of this analysis was the psychometric analysis of the German MacNew Heart Disease Questionnaire (MacNew) in HF patients, which has previously been shown to be reliable and valid in patients with myocardial infarction, angina pectoris and arrhythmia. METHODS: We recruited 89 patients (61.7+/-11.5 years; 84.3% male) in two Austrian and one Swiss cardiology department with documented HF (effect sizes 28.9+/-10.1%). The self-administered MacNew, the Short Form-36 (SF-36) and the Hospital Anxiety and Depression Scale were completed. Internal consistency reliability (Cronbach's alpha), discriminative and evaluative validity were assessed. RESULTS: Cronbach's alpha exceeded 0.80. Each MacNew scale differentiated between patients with and without anxiety (3.9+/-1.0 vs. 5.3+/-0.8, all P<0.001), with and without depression (4.2+/-1.2 vs. 5.2+/-0.9 all P<0.03) and by the SF-36 health transition item (deteriorate=4.39, no change=4.95, improve=5.45, all P<0.02). Evaluative validity was demonstrated with effect sizes >0.70 for a subsample attending a 12-week outpatient rehabilitation programme. CONCLUSIONS: The German language version of the MacNew demonstrates consistently acceptable psychometric properties of reliability, validity and responsiveness in patients with documented HF. Together with previous documentation of reliability, validity and responsive, these findings strengthen the argument for the MacNew as a potential 'core' HRQL measure, at least in the German language.

The self-administered 24-item geriatric pain measure (GPM-24-SA): psychometric properties in three European populations of community-dwelling older adults

OBJECTIVE: To explore the feasibility and psychometric properties of a self-administered version of the 24-item Geriatric Pain Measure (GPM-24-SA). DESIGN: Secondary analysis of baseline data from the Prevention in Older People-Assessment in Generalists' practices trial, an international multi-center study of a health-risk appraisal system. PARTICIPANTS: One thousand seventy-two community dwelling nondisabled older adults self-reporting pain from London, UK; Hamburg, Germany; and Solothurn, Switzerland. OUTCOME MEASURES: GPM-24-SA as part of a multidimensional Health Risk Appraisal Questionnaire including self-reported demographic and health-related information. RESULTS: Among the 1,072 subjects, 655 had complete GPM-24-SA data, 404 had 30% missing GPM-24-SA data. In psychometric analyses across the three European populations with complete GPM-24-SA data, the measure exhibited stable internal consistency, good convergent, divergent and discriminant validity, and produced stable pain measurements. However, factor analysis indicated differences in the GPM-24-SA across sites with discrepancies mainly related to items of a single subscale that failed to load appropriately. Analyses including imputation for subjects with psychometric properties comparable to complete data analyses suggesting that imputation in cases with psychometric evaluation of the GPM-24-SA is needed before it could be recommended for widespread use.

Psychometric properties of the Swiss version of the Nursing Home Survey on Patient Safety Culture

The Nursing Home Survey on Patient Safety Culture (NHSPSC) was specifically developed for nursing homes to assess a facility’s safety climate and it consists of 12 dimensions. After its pilot testing, however, no fur- ther psychometric analyses were performed on the instrument. For this study of safety climate in Swiss nursing home units, the NHSPSC was linguistically adapted to the Swiss context and to address the unit as well as facility level, with the aim of testing aspects of the validity and reliability of the Swiss version before its use in Swiss nursing home units. Psychometric analyses were performed on data from 367 nurs- ing personnel from nine nursing homes in the German-speaking part of Switzerland (response rate = 66%), and content validity (CVI) examined. The statistical influence of unit membership on respondents’ answers, and on their agreement concerning their units’ safety climate, was tested using intraclass corre- lation coefficients (ICCs) and the rWG(J) interrater agreement index. A multilevel exploratory factor analysis (MEFA) with oblimin rotation was applied to examine the questionnaire’s dimensionality. Cronbach’s alpha and Raykov’s rho were calculated to assess factor reliability. The relationship of safety climate dimensions with clinical outcomes was explored. Expert feedback confirmed the relevance of the instru- ment’s items (CVI = 0.93). Personnel showed strong agreement in their perceptions in three dimensions of the questionnaire. ICCs supported a multilevel analysis. MEFA produced nine factors at the within-level (in comparison to 12 in the original version) and two factors at the between-level with satisfactory fit statis- tics. Raykov’s Rho for the single level factors ranged between 0.67 and 0.86. Some safety climate dimen- sions show moderate, but non-significant correlations with the use of bedrails, physical restraint use, and fall-related injuries. The Swiss version of the NHSPSC needs further refinement and testing before its use can be recommended in Swiss nursing homes: its dimensionality needs further clarification, particularly to distinguish items addressing the unit-level safety climate from those at the facility level.

Psychometric properties of the Trauma and Distress Scale, TADS, in an adult community sample in Finland

BACKGROUND: There is increasing evidence that a history of childhood abuse and neglect is not uncommon among individuals who experience mental disorder and that childhood trauma experiences are associated with adult psychopathology. Although several interview and self-report instruments for retrospective trauma assessment have been developed, many focus on sexual abuse (SexAb) rather than on multiple types of trauma or adversity. METHODS: Within the European Prediction of Psychosis Study, the Trauma and Distress Scale (TADS) was developed as a new self-report assessment of multiple types of childhood trauma and distressing experiences. The TADS includes 43 items and, following previous measures including the Childhood Trauma Questionnaire, focuses on five core domains: emotional neglect (EmoNeg), emotional abuse (EmoAb), physical neglect (PhyNeg), physical abuse (PhyAb), and SexAb.This study explores the psychometric properties of the TADS (internal consistency and concurrent validity) in 692 participants drawn from the general population who completed a mailed questionnaire, including the TADS, a depression self-report and questions on help-seeking for mental health problems. Inter-method reliability was examined in a random sample of 100 responders who were reassessed in telephone interviews. RESULTS: After minor revisions of PhyNeg and PhyAb, internal consistencies were good for TADS totals and the domain raw score sums. Intra-class coefficients for TADS total score and the five revised core domains were all good to excellent when compared to the interviewed TADS as a gold standard. In the concurrent validity analyses, the total TADS and its all core domains were significantly associated with depression and help-seeking for mental problems as proxy measures for traumatisation. In addition, robust cutoffs for the total TADS and its domains were calculated. CONCLUSIONS: Our results suggest the TADS as a valid, reliable, and clinically useful instrument for assessing retrospectively reported childhood traumatisation.

Psychometric properties of the School Anxiety Inventory-Short Version in Spanish secondary education students

Background: The School Anxiety Inventory (SAI) can be applied in different fields of psychology. However, due to the inventory’s administration time, it may not be useful in certain situations. To address this concern, the present study developed a short version of the SAI (the SAI-SV). Method: This study examined the reliability and validity evidence drawn from the scores of the School Anxiety Inventory-Short Version (SAI-SV) using a sample of 2,367 (47.91% boys) Spanish secondary school students, ranging from 12 to 18 years of age. To analyze the dimensional structure of the SAI-SV, exploratory and confirmatory factor analyses were applied. Internal consistency and test-retest reliability were calculated for SAISV scores. Results: A correlated three-factor structure related to school situations (Anxiety about Aggression, Anxiety about Social Evaluation, and Anxiety about Academic Failure) and a three-factor structure related to the response systems of anxiety (Physiological Anxiety, Cognitive Anxiety, and Behavioral Anxiety) were identified and supported. The internal consistency and test-retest reliability were determined to be appropriate. Conclusions: The reliability and validity evidence based on the internal structure of SAI-SV scores was satisfactory.

Psychometric properties of the Spanish adaptation of the School Attitude Assessment Survey-Revised

Background: The assessment of attitudes toward school with the objective of identifying adolescents who may be at risk of underachievement has become an important area of research in educational psychology, although few specific tools for their evaluation have been designed to date. One of the instruments available is the School Attitude Assessment Survey-Revised (SAAS-R). Method: The objective of the current research is to test the construct validity and to analyze the psychometric properties of the Spanish version of the SAAS-R. Data were collected from 1,398 students attending different high schools. Students completed the SAAS-R along with measures of the g factor, and academic achievement was obtained from school records. Results: Confirmatory factor analysis, multivariate analysis of variance and analysis of variance tests supported the validity evidence. Conclusions: The results indicate that the Spanish version of the SAAS-R is a useful measure that contributes to identification of underachieving students. Lastly, the results obtained and their implications for education are discussed.