The measurement properties of the menorrhagia multi-attribute quality-of-life scale:a psychometric analysis

Menorrhagia, or heavy menstrual bleeding (HMB), is a common gynaecological condition. As the aim of treatment is to improve women's wellbeing and quality of life (QoL), it is necessary to have effective ways to measure this. This study investigated the reliability and validity of the menorrhagia multi-attribute scale (MMAS), a menorrhagia-specific QoL instrument. Participants (n = 431) completed the MMAS and a battery of other tests as part of the baseline assessment of the ECLIPSE (Effectiveness and Cost-effectiveness of Levonorgestrel-containing Intrauterine system in Primary care against Standard trEatment for menorrhagia) trial. Analyses of their responses suggest that the MMAS has good measurement properties and is therefore an appropriate condition-specific instrument to measure the outcome of treatment for HMB. © 2011 The Authors BJOG An International Journal of Obstetrics and Gynaecology © 2011 RCOG.

Development and psychometric tests of the Chinese-version low vision quality of life questionnaire

Background/Aims: To develop and assess the psychometric validity of a Chinese language Vision Health related quality-of-life (VRQoL) measurement instrument for the Chinese visually impaired. Methods: The Low Vision Quality of Life Questionnaire (LVQOL) was translated and adapted into the Chinese-version Low Vision Quality of Life Questionnaire (CLVQOL). The CLVQOL was completed by 100 randomly selected people with low vision (primary group) and 100 people with normal vision (control group). Ninety-four participants from the primary group completed the CLVQOL a second time 2 weeks later (test-retest group). The internal consistency reliability, test-retest reliability, item-internal consistency, item-discrimination validity, construct validity and discriminatory power of the CLVQOL were calculated. Results: The review committee agreed that the CLVQOL replicated the meaning of the LVQOL and was sensitive to cultural differences. The Cronbach's α coefficient and the split-half coefficient for the four scales and total CLVQOL scales were 0.75-0.97. The test-retest reliability as estimated by the intraclass correlations coefficient was 0.69-0.95. Item-internal consistency was >0.4 and item-discrimination validity was generally <0.40. The Varimax rotation factor analysis of the CLVQOL identified four principal factors. the quality-of-life rating of four subscales and the total score of the CLVQOL of the primary group were lower than those of the Control group, both in hospital-based subjects and community-based subjects. Conclusion: The CLVQOL Chinese is a culturally specific vision-related quality-of-life measure instrument. It satisfies conventional psychometric criteria, discriminates visually healthy populations from low vision patients and may be valuable in screening the local community as well as for use in clinical practice or research. © Springer 2005.

Development and psychometric evaluation of a new team effectiveness scale for all types of community adult mental health teams:a mixed-methods approach

Defining 'effectiveness' in the context of community mental health teams (CMHTs) has become increasingly difficult under the current pattern of provision required in National Health Service mental health services in England. The aim of this study was to establish the characteristics of multi-professional team working effectiveness in adult CMHTs to develop a new measure of CMHT effectiveness. The study was conducted between May and November 2010 and comprised two stages. Stage 1 used a formative evaluative approach based on the Productivity Measurement and Enhancement System to develop the scale with multiple stakeholder groups over a series of qualitative workshops held in various locations across England. Stage 2 analysed responses from a cross-sectional survey of 1500 members in 135 CMHTs from 11 Mental Health Trusts in England to determine the scale's psychometric properties. Based on an analysis of its structural validity and reliability, the resultant 20-item scale demonstrated good psychometric properties and captured one overall latent factor of CMHT effectiveness comprising seven dimensions: improved service user well-being, creative problem-solving, continuous care, inter-team working, respect between professionals, engagement with carers and therapeutic relationships with service users. The scale will be of significant value to CMHTs and healthcare commissioners both nationally and internationally for monitoring, evaluating and improving team functioning in practice.

Can psychometric testing be used to explain, predict and measure behavioural competences and the funding resource orchestration of the entrepreneur?

Financing is a critical entrepreneurial activity (Shane et al. 2003) and within the study of entrepreneurship, behaviour has been identified as an area requiring further exploration (Bird et al. 2012). Since 2008 supply side conditions for SMEs have been severe and increasingly entrepreneurs have to bundle or ‘orchestrate’ funding from a variety of sources in order to successfully finance the firm (Wright and Stigliani 2013: p.15). This longitudinal study uses psychometric testing to measure the behavioural competences of a panel of sixty entrepreneurs in the Creative Industries sector. Interviews were conducted over a 3 year period to identify finance finding behaviour. The research takes a pragmatic realism perspective to examine process and the different behavioural competences of entrepreneurs. The predictive qualities of this behaviour are explored in a funding context. The research confirmed a strong behavioural characteristic as validated through interviews and psychometric testing, was an orientation towards engagement and working with other organisations. In a funding context, this manifested itself in entrepreneurs using networks, seeking advice and sharing equity to fund growth. These co-operative, collaborative characteristics are different to the classic image of the entrepreneur as a risk-taker or extrovert. Leadership and achievement orientation were amongst the lowest scores. Three distinctive groups were identified and also shown by subsequent analysis to be a positive contribution to how entrepreneurial behavioural competences can be considered. Belonging to one of these three clusters is a strong predictive indicator of entrepreneurial behaviour – in this context, how entrepreneurs access finance. These Clusters were also proven to have different characteristics in relation to funding outcomes. The study seeks to make a contribution through the development of a methodology for entrepreneurs, policy makers and financial institutions to identify competencies in finding finance and overcome problems in information asymmetry.

Establishing a baseline of psychometric estimates of measurement of intercultural competence of Ukrainian faculty: Setting the stage for domestic and global intercultural understanding

Post-Soviet Ukraine is in a time of upheaval and transition. Internal relations between pro-Western and pro-Russian supporters have deteriorated in the light of recent political events of Euro Revolution, Russia's occupation of the Crimean peninsula, and the militant confrontations in the southeastern regions of the country. In the light of these developments, intercultural competence is greatly needed to alleviate domestic tensions and enable effective intercultural communication with the representatives of different cultures within the country and beyond its borders.^ This study established a baseline of psychometric estimates of intercultural competence of Ukrainian higher education faculty. A sample of 276 professors of different academic majors from one university in Western Ukraine participated in the research. The Global Perspective Inventory (GPI; Merrill, Braskamp, & Braskamp, 2012) was chosen as a research instrument to measure intercultural competence of the faculty members. The GPI takes into account cognitive, intrapersonal, and interpersonal domains, each of which contains two scales reflective of theories of cultural development and intercultural communication – Cognitive-Knowing, Cognitive-Knowledge, Intrapersonal-Identity, Intrapersonal-Affect, Interpersonal-Social Responsibility, and Interpersonal-Social Interaction. Because the research instrument has neither been previously used as a measure of intercultural competence, nor administered in Ukraine, it was cross-validated using a Table of Specification (Newman, Lim, & Pineda, 2013) and two sets of factor analyses. As a result, a modified version of the GPI was created for use in Ukraine.^ Multiple linear regression analyses were used to test relationships between the participants' GPI scores on intercultural competence, and several independent variables that consisted of academic discipline, intercultural experience, and how long the participants taught at the university. The analyses determined a positive relationship between the scores on three out of six scales of the original version and two out of five scales of the modified version of the GPI and all the independent variables simultaneously. The relationship between the faculty responses on the six scales of both GPI versions and the independent variables controlling for each other produced mixed results. A unique role of intercultural professional development in predicting intercultural competence was discussed.^

Fitting model-based psychometric functions to simultaneity and temporal-order judgment data: MATLAB and R routines

Research on temporal-order perception uses temporal-order judgment (TOJ) tasks or synchrony judgment (SJ) tasks in their binary SJ2 or ternary SJ3 variants. In all cases, two stimuli are presented with some temporal delay, and observers judge the order of presentation. Arbitrary psychometric functions are typically fitted to obtain performance measures such as sensitivity or the point of subjective simultaneity, but the parameters of these functions are uninterpretable. We describe routines in MATLAB and R that fit model-based functions whose parameters are interpretable in terms of the processes underlying temporal-order and simultaneity judgments and responses. These functions arise from an independent-channels model assuming arrival latencies with exponential distributions and a trichotomous decision space. Different routines fit data separately for SJ2, SJ3, and TOJ tasks, jointly for any two tasks, or also jointly for the three tasks (for common cases in which two or even the three tasks were used with the same stimuli and participants). Additional routines provide bootstrap p-values and confidence intervals for estimated parameters. A further routine is included that obtains performance measures from the fitted functions. An R package for Windows and source code of the MATLAB and R routines are available as Supplementary Files.

Shifts of the psychometric function: distinguishing bias from perceptual effects.

Morgan, Dillenburger, Raphael, and Solomon have shown that observers can use different response strategies when unsure of their answer, and, thus, they can voluntarily shift the location of the psychometric function estimated with the method of single stimuli (MSS; sometimes also referred to as the single-interval, two-alternative method). They wondered whether MSS could distinguish response bias from a true perceptual effect that would also shift the location of the psychometric function. We demonstrate theoretically that the inability to distinguish response bias from perceptual effects is an inherent shortcoming of MSS, although a three-response format including also an "undecided" response option may solve the problem under restrictive assumptions whose validity cannot be tested with MSS data. We also show that a proper two-alternative forced-choice (2AFC) task with the three-response format is free of all these problems so that bias and perceptual effects can easily be separated out. The use of a three-response 2AFC format is essential to eliminate a confound (response bias) in studies of perceptual effects and, hence, to eliminate a threat to the internal validity of research in this area.

Improving the estimation of psychometric functions in 2AFC discrimination tasks.

Ulrich and Vorberg (2009) presented a method that fits distinct functions for each order of presentation of standard and test stimuli in a two-alternative forced-choice (2AFC) discrimination task, which removes the contaminating influence of order effects from estimates of the difference limen. The two functions are fitted simultaneously under the constraint that their average evaluates to 0.5 when test and standard have the same magnitude, which was regarded as a general property of 2AFC tasks. This constraint implies that physical identity produces indistinguishability, which is valid when test and standard are identical except for magnitude along the dimension of comparison. However, indistinguishability does not occur at physical identity when test and standard differ on dimensions other than that along which they are compared (e.g., vertical and horizontal lines of the same length are not perceived to have the same length). In these cases, the method of Ulrich and Vorberg cannot be used. We propose a generalization of their method for use in such cases and illustrate it with data from a 2AFC experiment involving length discrimination of horizontal and vertical lines. The resultant data could be fitted with our generalization but not with the method of Ulrich and Vorberg. Further extensions of this method are discussed.

Psychometric functions for detection and discrimination with and without flankers.

Recent studies have reported that flanking stimuli broaden the psychometric function and lower detection thresholds. In the present study, we measured psychometric functions for detection and discrimination with and without flankers to investigate whether these effects occur throughout the contrast continuum. Our results confirm that lower detection thresholds with flankers are accompanied by broader psychometric functions. Psychometric functions for discrimination reveal that discrimination thresholds with and without flankers are similar across standard levels, and that the broadening of psychometric functions with flankers disappears as standard contrast increases, to the point that psychometric functions at high standard levels are virtually identical with or without flankers. Threshold-versus-contrast (TvC) curves with flankers only differ from TvC curves without flankers in occasional shallower dippers and lower branches on the left of the dipper, but they run virtually superimposed at high standard levels. We discuss differences between our results and other results in the literature, and how they are likely attributed to the differential vulnerability of alternative psychophysical procedures to the effects of presentation order. We show that different models of flanker facilitation can fit the data equally well, which stresses that succeeding at fitting a model does not validate it in any sense.

Fixed vs. variable noise in 2AFC contrast discrimination: lessons from psychometric functions.

Recent discussion regarding whether the noise that limits 2AFC discrimination performance is fixed or variable has focused either on describing experimental methods that presumably dissociate the effects of response mean and variance or on reanalyzing a published data set with the aim of determining how to solve the question through goodness-of-fit statistics. This paper illustrates that the question cannot be solved by fitting models to data and assessing goodness-of-fit because data on detection and discrimination performance can be indistinguishably fitted by models that assume either type of noise when each is coupled with a convenient form for the transducer function. Thus, success or failure at fitting a transducer model merely illustrates the capability (or lack thereof) of some particular combination of transducer function and variance function to account for the data, but it cannot disclose the nature of the noise. We also comment on some of the issues that have been raised in recent exchange on the topic, namely, the existence of additional constraints for the models, the presence of asymmetric asymptotes, the likelihood of history-dependent noise, and the potential of certain experimental methods to dissociate the effects of response mean and variance.

Bayesian adaptive estimation of arbitrary points on a psychometric function

Bayesian adaptive methods have been extensively used in psychophysics to estimate the point at which performance on a task attains arbitrary percentage levels, although the statistical properties of these estimators have never been assessed. We used simulation techniques to determine the small-sample properties of Bayesian estimators of arbitrary performance points, specifically addressing the issues of bias and precision as a function of the target percentage level. The study covered three major types of psychophysical task (yes-no detection, 2AFC discrimination and 2AFC detection) and explored the entire range of target performance levels allowed for by each task. Other factors included in the study were the form and parameters of the actual psychometric function Psi, the form and parameters of the model function M assumed in the Bayesian method, and the location of Psi within the parameter space. Our results indicate that Bayesian adaptive methods render unbiased estimators of any arbitrary point on psi only when M=Psi, and otherwise they yield bias whose magnitude can be considerable as the target level moves away from the midpoint of the range of Psi. The standard error of the estimator also increases as the target level approaches extreme values whether or not M=Psi. Contrary to widespread belief, neither the performance level at which bias is null nor that at which standard error is minimal can be predicted by the sweat factor. A closed-form expression nevertheless gives a reasonable fit to data describing the dependence of standard error on number of trials and target level, which allows determination of the number of trials that must be administered to obtain estimates with prescribed precision.

The Psychometric Properties of the Developmental Reading Assessment

ABSTRACT The purpose of this study was to examine the technical adequacy of the Developmental Reading Assessment (Beaver & Carter, 2004). Internal consistency analysis, factor analysis, and linear regression analyses were used to test whether the DRA is a statistically reliable measuring of reading comprehension for Grades 7 and 8 students. Correlational analyses, decision consistency analyses, and a focus group of experienced Intermediate (Grades 7 and 8) teachers examined whether there is evidence that the results from the DRA provide valid interpretations regarding students’ reading skills and comprehension. Results indicated that, as currently scored, internal consistency is low and skewness of distribution is high. Factor analyses did not replicate those cited by the DRA developers to prove construct validity. Two-way contingency analyses determined that decision consistency did not vary greatly between the DRA, EQAO, scores and report card marks. Views expressed during the focus group echoed many of the challenges to validity found in the statistical analysis. The teachers found that the DRA was somewhat useful, as there were limited alternative reading assessments available for the classroom, but did not endorse it strongly. The study found little evidence that the DRA provides valid interpretations regarding Intermediate students’ reading skills. Indicated changes to the structure and administration procedures of the DRA may ameliorate some of these issues.

Psychometric properties of the Drive for Muscularity Scale in Malay men

The Drive for Muscularity Scale (DMS) is a widely used measure in studies of men’s body image, but few studies have examined its psychometric properties outside English-speaking samples. Here, we assessed the factor structure of a Malay translation of the DMS. A community sample of 159 Malay men from Kuala Lumpur, Malaysia, completed the DMS, along with measures of self-esteem, body appreciation, and muscle discrepancy. Exploratory factor analysis led to the extraction of two factors, differentiating attitudes from behaviours, which mirrors the parent scale. Both factors also loaded on to a higher-order drive for muscularity factor. The subscales of the Malay DMS had adequate internal consistencies and good convergent validity, insofar as significant relationships were reported with self-esteem, body appreciation,muscle discrepancy, and body mass index. These results indicate that the Malay DMS has acceptable psychometric properties and can be used to assess body image concerns in Malay men.

Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements

The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.

Testing the psychometric properties of two self-report clinical competence scales for nursing students.

Background: It is important to assess the clinical competence of nursing students to gauge their educational needs. Competence can be measured by self-assessment tools; however, Anema and McCoy (2010) contend that currently available measures should be further psychometrically tested.
Aim: To test the psychometric properties of Nursing Competencies Questionnaire (NCQ) and Self-Efficacy in Clinical Performance (SECP) clinical competence scales.
Method: A non-randomly selected sample of n=248 2nd year nursing students completed NCQ, SECP and demographic questionnaires (June and September 2013). Mokken Scaling Analysis (MSA) was used to investigate structural validity and scale properties; convergent and discriminant validity and reliability were also tested for each scale.
Results: MSA analysis identified that the NCQ is a unidimensional scale with strong scale scalability coefficients Hs =0.581; but limited item rankability HT =0.367. The SECP scale MSA suggested that the scale could be potentially split into two unidimensional scales (SECP28 and SECP7), each with good/reasonable scalablity psychometric properties as summed scales but negligible/very limited scale rankability (SECP28: Hs = 0.55, HT=0.211; SECP7: Hs = 0.61, HT=0.049). Analysis of between cohort differences and NCQ/SECP scores produced evidence of discriminant and convergent validity; good internal reliability was also found: NCQ α = 0.93, SECP28 α = 0.96 and SECP7 α=0.89.

Discussion: In line with previous research further evidence of the NCQ’s reliability and validity was demonstrated. However, as the SECP findings are new and the sample small with reference to Straat and colleagues (2014), the SECP results should be interpreted with caution and verified on a second sample.
Conclusions: Measurement of perceived self-competence could start early in a nursing programme to support students’ development of clinical competence. Further testing of the SECP scale with larger nursing student samples from different programme years is indicated.

References:
Anema, M., G and McCoy, JK. (2010) Competency-Based Nursing Education: Guide to Achieving Outstanding Learner Outcomes. New York: Springer.
Straat, JH., van der Ark, LA and Sijtsma, K. (2014) Minimum Sample Size Requirements for Mokken Scale Analysis Educational and Psychological Measurement 74 (5), 809-822.