Psychometric assessment of recovery from mild traumatic brain injury

The Rapid Screen of Concussion (RSC) is a brief psychometric test battery, designed to provide a functional criterion to aid clinical diagnosis of mild traumatic brain injury (mTBI). The present research aimed to examine the utility of this instrument for assessing recovery after mTBI. Three studies were conducted. In Study 1, Discriminant Function Analysis was performed to determine how well the RSC differentiated uninjured controls (N¼16), from mTBI patients (N¼22) and moderate to severe TBI patients (N¼14), several months post-injury. As predicted, moderate to severe TBI patients achieved lower scores than the mTBI and control groups. The RSC also successfully differentiated between each of the diagnostic groups, yielding an overall correct classification rate of 75%. Study 2 examined the predictive utility of the RSC in the mTBI sample (N¼22). Acute injury performance on the RSC was correlated with post-injury scores at an average of 5.5 months post-injury. Statistically significant partial correlation coefficients (r¼0.53r¼0.80) were found for each of the subtests, showing that low acute RSC scores were predictive of poor recovery scores on the RSC after mTBI. In the third study, Reliable Change Indices were calculated on the RSC subtests to examine individual patterns of recovery from mTBI. While 17 of the 23 participants made a significant improvement on their acute injury DSST scores (74%), only 13 of 25 made a significant improvement on the Rapid Sentence Judgement Test (52%), highlighting differential recovery of function, and challenging the notion of full recovery from mTBI within 3 months. These overall results offer support for the construct and predictive validity of the RSC and demonstrate that inexpensive tests of brain function may be useful for managing mTBI acutely for prognosis.

Evaluating the properties of a stage-specific self-efficacy scale for physical activity using classical test theory, confirmatory factor analysis and item response modeling

The purpose of this paper was to evaluate the psychometric properties of a stage-specific selfefficacy scale for physical activity with classical test theory (CTT), confirmatory factor analysis (CFA) and item response modeling (IRM). Women who enrolled in the Women On The Move study completed a 20-item stage-specific self-efficacy scale developed for this study [n = 226, 51.1% African-American and 48.9% Hispanic women, mean age = 49.2 (67.0) years, mean body mass index = 29.7 (66.4)]. Three analyses were conducted: (i) a CTT item analysis, (ii) a CFA to validate the factor structure and (iii) an IRM analysis. The CTT item analysis and the CFA results showed that the scale had high internal consistency (ranging from 0.76 to 0.93) and a strong factor structure. Results also showed that the scale could be improved by modifying or eliminating some of the existing items without significantly altering the content of the scale. The IRM results also showed that the scale had few items that targeted high self-efficacy and the stage-specific assumption underlying the scale was rejected. In addition, the IRM analyses found that the five-point response format functioned more like a four-point response format. Overall, employing multiple methods to assess the psychometric properties of the stage-specific self-efficacy scale demonstrated the complimentary nature of these methods and it highlighted the strengths and weaknesses of this scale.

Psychometric properties of the Chinese version of the diabetes empowerment scale

Metabolic control is central to positive clinical outcome in patients with diabetes. Empowerment has been linked to metabolic control in this clinical group. The current study sought to determine key psychometric properties of the Chinese version of the Diabetes Empowerment Scale (C-DES) and to explore the relationship of the C-DES sub-scales to metabolic control in 189 patients with a diagnosis of diabetes. Confirmatory factor analysis established that the five sub-scales of the C-DES offered a highly satisfactory fit to the data. Furthermore, C-DES sub-scales were found to have generally acceptable internal consistency and divergent reliability. However, convergent reliability of C-DES sub-scales could not be established against metabolic control. It is concluded that future research needs to address ambiguities in the relationship between empowerment and metabolic control in order to afford patients an evidenced-based treatment package to assure optimal metabolic control.

An examination of the psychometric properties of the Rosenberg Self-Esteem Scale (RSES) in Chinese acute coronary syndrome (ACS) patients

The psychometric properties of the Rosenberg Self-Esteem Scale (RSES) as a clinical research instrument for acute coronary syndrome (ACS) patients were investigated in a translated Chinese version of the instrument. A confirmatory factor analysis was conducted on the RSES to establish its psychometric properties in 128 ACS patients over two observation points (within 1 week and 6 months post-admission for ACS). Internal and test - retest reliability of the RSES-TOT (all-items) and RSES-POS sub-scale (positively valenced items) were found to be acceptable. The RSES-NEG sub-scale (negatively valenced items) lacked acceptable internal reliability. The underlying factor structure of the RSES comprised two distinct but related factors, though there was inconsistency in best model fit indices at the 1-week observation point. The use of the RSES as two sub-scales (RSES-POS and RSES-NEG) may be clinically useful in evaluating the influence of this important psychological construct on the health outcomes of patients with ACS. Directions for future research are indicated.

+Psychometric evaluation of the MacDQoL individualised measure of the impact of macular degeneration on quality of life

Background: The MacDQoL is an individualised measure of the impact of macular degeneration (MD) on quality of life (QoL). There is preliminary evidence of its psychometric properties and sensitivity to severity of MD. The aim of this study was to carry out further psychometric evaluation with a larger sample and investigate the measure's sensitivity to MD severity. Methods: Patients with MD (n = 156: 99 women, 57 men, mean age 79 ± 13 years), recruited from eye clinics (one NHS, one private) completed the MacDQoL by telephone interview and later underwent a clinic vision assessment including near and distance visual acuity (VA), comfortable near VA, contrast sensitivity, colour recognition, recovery from glare and presence or absence of distortion or scotoma in the central 10° of the visual field. Results: The completion rate for the MacDQoL items was 99.8%. Of the 26 items, three were dropped from the measure due to redundancy. A fourth was retained in the questionnaire but excluded when computing the scale score. Principal components analysis and Cronbach's alpha (0.944) supported combining the remaining 22 items in a single scale. Lower MacDQoL scores, indicating more negative impact of MD on QoL, were associated with poorer distance VA (better eye r = -0.431 p < 0.001; worse eye r = -0.350 p < 0.001; binocular vision r = -0.419 p < 0.001) and near VA (better eye r -0.326 p < 0.001; worse eye r = -0.226 p < 0.001; binocular vision r = -0.326 p < 0.001). Poorer MacDQoL scores were associated with poorer contrast sensitivity (better eye r = 0.392 p < 0.001; binocular vision r = 0.423 p < 0.001), poorer colour recognition (r = 0.417 p < 0.001) and poorer comfortable near VA (r = -0.283, p < 0.001). The MacDQoL differentiated between those with and without binocular scotoma (U = 1244 p < 0.001). Conclusion: The MacDQoL 22-item scale has excellent internal consistency reliability and a single-factor structure. The measure is acceptable to respondents and the generic QoL item, MD-specific QoL item and average weighted impact score are related to several measures of vision. The MacDQoL demonstrates that MD has considerable negative impact on many aspects of QoL, particularly independence, leisure activities, dealing with personal affairs and mobility. The measure may be valuable for use in clinical trials and routine clinical care. © 2005 Mitchell et al; licensee BioMed Central Ltd.

The slope of the psychometric function and non-stationarity of thresholds in spatiotemporal contrast vision

The slope of the two-interval, forced-choice psychometric function (e.g. the Weibull parameter, ß) provides valuable information about the relationship between contrast sensitivity and signal strength. However, little is known about how or whether ß varies with stimulus parameters such as spatiotemporal frequency and stimulus size and shape. A second unresolved issue concerns the best way to estimate the slope of the psychometric function. For example, if an observer is non-stationary (e.g. their threshold drifts between experimental sessions), ß will be underestimated if curve fitting is performed after collapsing the data across experimental sessions. We measured psychometric functions for 2 experienced observers for 14 different spatiotemporal configurations of pulsed or flickering grating patches and bars on each of 8 days. We found ß ˜ 3 to be fairly constant across almost all conditions, consistent with a fixed nonlinear contrast transducer and/or a constant level of intrinsic stimulus uncertainty (e.g. a square law transducer and a low level of intrinsic uncertainty). Our analysis showed that estimating a single ß from results averaged over several experimental sessions was slightly more accurate than averaging multiple estimates from several experimental sessions. However, the small levels of non-stationarity (SD ˜ 0.8 dB) meant that the difference between the estimates was, in practice, negligible.

Development and psychometric tests of the Chinese-version low vision quality of life questionnaire

Background/Aims: To develop and assess the psychometric validity of a Chinese language Vision Health related quality-of-life (VRQoL) measurement instrument for the Chinese visually impaired. Methods: The Low Vision Quality of Life Questionnaire (LVQOL) was translated and adapted into the Chinese-version Low Vision Quality of Life Questionnaire (CLVQOL). The CLVQOL was completed by 100 randomly selected people with low vision (primary group) and 100 people with normal vision (control group). Ninety-four participants from the primary group completed the CLVQOL a second time 2 weeks later (test-retest group). The internal consistency reliability, test-retest reliability, item-internal consistency, item-discrimination validity, construct validity and discriminatory power of the CLVQOL were calculated. Results: The review committee agreed that the CLVQOL replicated the meaning of the LVQOL and was sensitive to cultural differences. The Cronbach's α coefficient and the split-half coefficient for the four scales and total CLVQOL scales were 0.75-0.97. The test-retest reliability as estimated by the intraclass correlations coefficient was 0.69-0.95. Item-internal consistency was >0.4 and item-discrimination validity was generally <0.40. The Varimax rotation factor analysis of the CLVQOL identified four principal factors. the quality-of-life rating of four subscales and the total score of the CLVQOL of the primary group were lower than those of the Control group, both in hospital-based subjects and community-based subjects. Conclusion: The CLVQOL Chinese is a culturally specific vision-related quality-of-life measure instrument. It satisfies conventional psychometric criteria, discriminates visually healthy populations from low vision patients and may be valuable in screening the local community as well as for use in clinical practice or research. © Springer 2005.

Development and psychometric evaluation of a new team effectiveness scale for all types of community adult mental health teams:a mixed-methods approach

Defining 'effectiveness' in the context of community mental health teams (CMHTs) has become increasingly difficult under the current pattern of provision required in National Health Service mental health services in England. The aim of this study was to establish the characteristics of multi-professional team working effectiveness in adult CMHTs to develop a new measure of CMHT effectiveness. The study was conducted between May and November 2010 and comprised two stages. Stage 1 used a formative evaluative approach based on the Productivity Measurement and Enhancement System to develop the scale with multiple stakeholder groups over a series of qualitative workshops held in various locations across England. Stage 2 analysed responses from a cross-sectional survey of 1500 members in 135 CMHTs from 11 Mental Health Trusts in England to determine the scale's psychometric properties. Based on an analysis of its structural validity and reliability, the resultant 20-item scale demonstrated good psychometric properties and captured one overall latent factor of CMHT effectiveness comprising seven dimensions: improved service user well-being, creative problem-solving, continuous care, inter-team working, respect between professionals, engagement with carers and therapeutic relationships with service users. The scale will be of significant value to CMHTs and healthcare commissioners both nationally and internationally for monitoring, evaluating and improving team functioning in practice.

Can psychometric testing be used to explain, predict and measure behavioural competences and the funding resource orchestration of the entrepreneur?

Financing is a critical entrepreneurial activity (Shane et al. 2003) and within the study of entrepreneurship, behaviour has been identified as an area requiring further exploration (Bird et al. 2012). Since 2008 supply side conditions for SMEs have been severe and increasingly entrepreneurs have to bundle or ‘orchestrate’ funding from a variety of sources in order to successfully finance the firm (Wright and Stigliani 2013: p.15). This longitudinal study uses psychometric testing to measure the behavioural competences of a panel of sixty entrepreneurs in the Creative Industries sector. Interviews were conducted over a 3 year period to identify finance finding behaviour. The research takes a pragmatic realism perspective to examine process and the different behavioural competences of entrepreneurs. The predictive qualities of this behaviour are explored in a funding context. The research confirmed a strong behavioural characteristic as validated through interviews and psychometric testing, was an orientation towards engagement and working with other organisations. In a funding context, this manifested itself in entrepreneurs using networks, seeking advice and sharing equity to fund growth. These co-operative, collaborative characteristics are different to the classic image of the entrepreneur as a risk-taker or extrovert. Leadership and achievement orientation were amongst the lowest scores. Three distinctive groups were identified and also shown by subsequent analysis to be a positive contribution to how entrepreneurial behavioural competences can be considered. Belonging to one of these three clusters is a strong predictive indicator of entrepreneurial behaviour – in this context, how entrepreneurs access finance. These Clusters were also proven to have different characteristics in relation to funding outcomes. The study seeks to make a contribution through the development of a methodology for entrepreneurs, policy makers and financial institutions to identify competencies in finding finance and overcome problems in information asymmetry.

Converging evidence that common timing processes underlie temporal-order and simultaneity judgments: a model-based analysis

Perception of simultaneity and temporal order is studied with simultaneity judgment (SJ) and temporal-order judgment (TOJ) tasks. In the former, observers report whether presentation of two stimuli was subjectively simultaneous; in the latter, they report which stimulus was subjectively presented first. SJ and TOJ tasks typically give discrepant results, which has prompted the view that performance is mediated by different processes in each task. We looked at these discrepancies from a model that yields psychometric functions whose parameters characterize the timing, decisional, and response processes involved in SJ and TOJ tasks. We analyzed 12 data sets from published studies in which both tasks had been used in within-subjects designs, all of which had reported differences in performance across tasks. Fitting the model jointly to data from both tasks, we tested the hypothesis that common timing processes sustain simultaneity and temporal order judgments, with differences in performance arising from task-dependent decisional and response processes. The results supported this hypothesis, also showing that model psychometric functions account for aspects of SJ and TOJ data that classical analyses overlook. Implications for research on perception of simultaneity and temporal order are discussed.

Analysis of residuals in contingency tables: another nail in the coffin of conditional approaches to significance testing.

Omnibus tests of significance in contingency tables use statistics of the chi-square type. When the null is rejected, residual analyses are conducted to identify cells in which observed frequencies differ significantly from expected frequencies. Residual analyses are thus conditioned on a significant omnibus test. Conditional approaches have been shown to substantially alter type I error rates in cases involving t tests conditional on the results of a test of equality of variances, or tests of regression coefficients conditional on the results of tests of heteroscedasticity. We show that residual analyses conditional on a significant omnibus test are also affected by this problem, yielding type I error rates that can be up to 6 times larger than nominal rates, depending on the size of the table and the form of the marginal distributions. We explored several unconditional approaches in search for a method that maintains the nominal type I error rate and found out that a bootstrap correction for multiple testing achieved this goal. The validity of this approach is documented for two-way contingency tables in the contexts of tests of independence, tests of homogeneity, and fitting psychometric functions. Computer code in MATLAB and R to conduct these analyses is provided as Supplementary Material.

The Psychometric Properties of the Developmental Reading Assessment

ABSTRACT The purpose of this study was to examine the technical adequacy of the Developmental Reading Assessment (Beaver & Carter, 2004). Internal consistency analysis, factor analysis, and linear regression analyses were used to test whether the DRA is a statistically reliable measuring of reading comprehension for Grades 7 and 8 students. Correlational analyses, decision consistency analyses, and a focus group of experienced Intermediate (Grades 7 and 8) teachers examined whether there is evidence that the results from the DRA provide valid interpretations regarding students’ reading skills and comprehension. Results indicated that, as currently scored, internal consistency is low and skewness of distribution is high. Factor analyses did not replicate those cited by the DRA developers to prove construct validity. Two-way contingency analyses determined that decision consistency did not vary greatly between the DRA, EQAO, scores and report card marks. Views expressed during the focus group echoed many of the challenges to validity found in the statistical analysis. The teachers found that the DRA was somewhat useful, as there were limited alternative reading assessments available for the classroom, but did not endorse it strongly. The study found little evidence that the DRA provides valid interpretations regarding Intermediate students’ reading skills. Indicated changes to the structure and administration procedures of the DRA may ameliorate some of these issues.

Psychometric properties of the Drive for Muscularity Scale in Malay men

The Drive for Muscularity Scale (DMS) is a widely used measure in studies of men’s body image, but few studies have examined its psychometric properties outside English-speaking samples. Here, we assessed the factor structure of a Malay translation of the DMS. A community sample of 159 Malay men from Kuala Lumpur, Malaysia, completed the DMS, along with measures of self-esteem, body appreciation, and muscle discrepancy. Exploratory factor analysis led to the extraction of two factors, differentiating attitudes from behaviours, which mirrors the parent scale. Both factors also loaded on to a higher-order drive for muscularity factor. The subscales of the Malay DMS had adequate internal consistencies and good convergent validity, insofar as significant relationships were reported with self-esteem, body appreciation,muscle discrepancy, and body mass index. These results indicate that the Malay DMS has acceptable psychometric properties and can be used to assess body image concerns in Malay men.

Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements

The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.

Reducing child conduct disordered behaviour and improving parent mental health in disadvantaged families: a 12 month follow-up and cost analysis of a parenting intervention

The effectiveness of the Incredible Years Basic parent programme (IYBP) in reducing child conduct problems and improving parent competencies and mental health was examined in a 12-month follow-up. Pre- to post-intervention service use and related costs were also analysed. A total of 103 families and their children (aged 32–88 months), who previously participated in a randomised controlled trial of the IYBP, took part in a 12-month follow-up assessment. Child and parent behaviour and well-being were measured using psychometric and observational measures. An intention-to-treat analysis was carried out using a one-way repeated measures ANOVA. Pairwise comparisons were subsequently conducted to determine whether treatment outcomes were sustained 1 year post-baseline assessment. Results indicate that post-intervention improvements in child conduct problems, parenting behaviour and parental mental health were maintained. Service use and associated costs continued to decline. The results indicate that parent-focused interventions, implemented in the early years, can result in improvements in child and parent behaviour and well-being 12 months later. A reduced reliance on formal services is also indicated.