Converging evidence that common timing processes underlie temporal-order and simultaneity judgments: a model-based analysis

Perception of simultaneity and temporal order is studied with simultaneity judgment (SJ) and temporal-order judgment (TOJ) tasks. In the former, observers report whether presentation of two stimuli was subjectively simultaneous; in the latter, they report which stimulus was subjectively presented first. SJ and TOJ tasks typically give discrepant results, which has prompted the view that performance is mediated by different processes in each task. We looked at these discrepancies from a model that yields psychometric functions whose parameters characterize the timing, decisional, and response processes involved in SJ and TOJ tasks. We analyzed 12 data sets from published studies in which both tasks had been used in within-subjects designs, all of which had reported differences in performance across tasks. Fitting the model jointly to data from both tasks, we tested the hypothesis that common timing processes sustain simultaneity and temporal order judgments, with differences in performance arising from task-dependent decisional and response processes. The results supported this hypothesis, also showing that model psychometric functions account for aspects of SJ and TOJ data that classical analyses overlook. Implications for research on perception of simultaneity and temporal order are discussed.

Analysis of residuals in contingency tables: another nail in the coffin of conditional approaches to significance testing.

Omnibus tests of significance in contingency tables use statistics of the chi-square type. When the null is rejected, residual analyses are conducted to identify cells in which observed frequencies differ significantly from expected frequencies. Residual analyses are thus conditioned on a significant omnibus test. Conditional approaches have been shown to substantially alter type I error rates in cases involving t tests conditional on the results of a test of equality of variances, or tests of regression coefficients conditional on the results of tests of heteroscedasticity. We show that residual analyses conditional on a significant omnibus test are also affected by this problem, yielding type I error rates that can be up to 6 times larger than nominal rates, depending on the size of the table and the form of the marginal distributions. We explored several unconditional approaches in search for a method that maintains the nominal type I error rate and found out that a bootstrap correction for multiple testing achieved this goal. The validity of this approach is documented for two-way contingency tables in the contexts of tests of independence, tests of homogeneity, and fitting psychometric functions. Computer code in MATLAB and R to conduct these analyses is provided as Supplementary Material.

The Psychometric Properties of the Developmental Reading Assessment

ABSTRACT The purpose of this study was to examine the technical adequacy of the Developmental Reading Assessment (Beaver & Carter, 2004). Internal consistency analysis, factor analysis, and linear regression analyses were used to test whether the DRA is a statistically reliable measuring of reading comprehension for Grades 7 and 8 students. Correlational analyses, decision consistency analyses, and a focus group of experienced Intermediate (Grades 7 and 8) teachers examined whether there is evidence that the results from the DRA provide valid interpretations regarding students’ reading skills and comprehension. Results indicated that, as currently scored, internal consistency is low and skewness of distribution is high. Factor analyses did not replicate those cited by the DRA developers to prove construct validity. Two-way contingency analyses determined that decision consistency did not vary greatly between the DRA, EQAO, scores and report card marks. Views expressed during the focus group echoed many of the challenges to validity found in the statistical analysis. The teachers found that the DRA was somewhat useful, as there were limited alternative reading assessments available for the classroom, but did not endorse it strongly. The study found little evidence that the DRA provides valid interpretations regarding Intermediate students’ reading skills. Indicated changes to the structure and administration procedures of the DRA may ameliorate some of these issues.

Psychometric properties of the Drive for Muscularity Scale in Malay men

The Drive for Muscularity Scale (DMS) is a widely used measure in studies of men’s body image, but few studies have examined its psychometric properties outside English-speaking samples. Here, we assessed the factor structure of a Malay translation of the DMS. A community sample of 159 Malay men from Kuala Lumpur, Malaysia, completed the DMS, along with measures of self-esteem, body appreciation, and muscle discrepancy. Exploratory factor analysis led to the extraction of two factors, differentiating attitudes from behaviours, which mirrors the parent scale. Both factors also loaded on to a higher-order drive for muscularity factor. The subscales of the Malay DMS had adequate internal consistencies and good convergent validity, insofar as significant relationships were reported with self-esteem, body appreciation,muscle discrepancy, and body mass index. These results indicate that the Malay DMS has acceptable psychometric properties and can be used to assess body image concerns in Malay men.

Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements

The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.

Reducing child conduct disordered behaviour and improving parent mental health in disadvantaged families: a 12 month follow-up and cost analysis of a parenting intervention

The effectiveness of the Incredible Years Basic parent programme (IYBP) in reducing child conduct problems and improving parent competencies and mental health was examined in a 12-month follow-up. Pre- to post-intervention service use and related costs were also analysed. A total of 103 families and their children (aged 32–88 months), who previously participated in a randomised controlled trial of the IYBP, took part in a 12-month follow-up assessment. Child and parent behaviour and well-being were measured using psychometric and observational measures. An intention-to-treat analysis was carried out using a one-way repeated measures ANOVA. Pairwise comparisons were subsequently conducted to determine whether treatment outcomes were sustained 1 year post-baseline assessment. Results indicate that post-intervention improvements in child conduct problems, parenting behaviour and parental mental health were maintained. Service use and associated costs continued to decline. The results indicate that parent-focused interventions, implemented in the early years, can result in improvements in child and parent behaviour and well-being 12 months later. A reduced reliance on formal services is also indicated.

Exploring the effects of a universal classroom management training programme on teacher and child behaviour: a group randomised controlled trial and cost analysis

Teachers frequently struggle to cope with conduct problems in the classroom. The aim of this study was to assess the effectiveness of the Incredible Years Teacher Classroom Management Training Programme for improving teacher competencies and child adjustment. The study involved a group randomised controlled trial which included 22 teachers and 217 children (102 boys and 115 girls). The average age of children included in the study was 5.3 years (standard deviation = 0.89). Teachers were randomly allocated to an intervention group (n = 11 teachers; 110 children) or a waiting-list control group (n = 11; 107 children). The sample also included 63 ‘high-risk’ children (33 intervention; 30 control), who scored above the cut-off (>12) on the Strengths and Difficulties Questionnaire for abnormal socioemotional and behavioural difficulties. Teacher and child behaviours were assessed at baseline and 6 months later using psychometric and observational measures. Programme delivery costs were also analysed. Results showed positive changes in teachers’ self-reported use of positive classroom management strategies (effect size = 0.56), as well as negative classroom management strategies (effect size = −0.43). Teacher reports also highlight improvements in the classroom behaviour of the high-risk group of children, while the estimated cost of delivering the Incredible Years Teacher Classroom Management Training Programme was modest. However, analyses of teacher and child observations were largely non-significant. A need for further research exploring the effectiveness and cost-effectiveness of the Incredible Years Teacher Classroom Management Training Programme is indicated.

Testing the psychometric properties of two self-report clinical competence scales for nursing students.

Background: It is important to assess the clinical competence of nursing students to gauge their educational needs. Competence can be measured by self-assessment tools; however, Anema and McCoy (2010) contend that currently available measures should be further psychometrically tested.
Aim: To test the psychometric properties of Nursing Competencies Questionnaire (NCQ) and Self-Efficacy in Clinical Performance (SECP) clinical competence scales.
Method: A non-randomly selected sample of n=248 2nd year nursing students completed NCQ, SECP and demographic questionnaires (June and September 2013). Mokken Scaling Analysis (MSA) was used to investigate structural validity and scale properties; convergent and discriminant validity and reliability were also tested for each scale.
Results: MSA analysis identified that the NCQ is a unidimensional scale with strong scale scalability coefficients Hs =0.581; but limited item rankability HT =0.367. The SECP scale MSA suggested that the scale could be potentially split into two unidimensional scales (SECP28 and SECP7), each with good/reasonable scalablity psychometric properties as summed scales but negligible/very limited scale rankability (SECP28: Hs = 0.55, HT=0.211; SECP7: Hs = 0.61, HT=0.049). Analysis of between cohort differences and NCQ/SECP scores produced evidence of discriminant and convergent validity; good internal reliability was also found: NCQ α = 0.93, SECP28 α = 0.96 and SECP7 α=0.89.

Discussion: In line with previous research further evidence of the NCQ’s reliability and validity was demonstrated. However, as the SECP findings are new and the sample small with reference to Straat and colleagues (2014), the SECP results should be interpreted with caution and verified on a second sample.
Conclusions: Measurement of perceived self-competence could start early in a nursing programme to support students’ development of clinical competence. Further testing of the SECP scale with larger nursing student samples from different programme years is indicated.

References:
Anema, M., G and McCoy, JK. (2010) Competency-Based Nursing Education: Guide to Achieving Outstanding Learner Outcomes. New York: Springer.
Straat, JH., van der Ark, LA and Sijtsma, K. (2014) Minimum Sample Size Requirements for Mokken Scale Analysis Educational and Psychological Measurement 74 (5), 809-822.

Testing the psychometric properties of Kidscreen-27 with Irish children of low socio-economic status

Background

Kidscreen-27 was developed as part of a cross-cultural European Union-funded project to standardise the measurement of children’s health-related quality of life. Yet, research has reported mixed evidence for the hypothesised 5-factor model, and no confirmatory factor analysis (CFA) has been conducted on the instrument with children of low socio-economic status (SES) across Ireland (Northern and Republic).

Method

The data for this study were collected as part of a clustered randomised controlled trial. A total of 663 (347 male, 315 female) 8–9-year-old children (M = 8.74, SD = .50) of low SES took part. A 5- and modified 7-factor CFA models were specified using the maximum likelihood estimation. A nested Chi-square difference test was conducted to compare the fit of the models. Internal consistency and floor and ceiling effects were also examined.

Results

CFA found that the hypothesised 5-factor model was an unacceptable fit. However, the modified 7-factor model was supported. A nested Chi-square difference test confirmed that the fit of the 7-factor model was significantly better than that of the 5-factor model. Internal consistency was unacceptable for just one scale. Ceiling effects were present in all but one of the factors.

Conclusions

Future research should apply the 7-factor model with children of low socio-economic status. Such efforts would help monitor the health status of the population.

Parental involvement in sport: psychometric development and empirical test of a theoretical model

The purposes of the present multistudy were to develop and provide initial construct validity for measures based on the model of parental involvement in sport (Study 1) and examine structural relationships among the constructs of the model (Study 2). In Study 1 (nparents = 342, nathletes = 223), a confirmatory factor analysis was used to verify the psychometric properties of the measures. Content and construct validity were evaluated, as well individual and composite reliability. Multi-group analysis with two independent samples provided evidence of factorial invariance. In Study 2 (nparents = 754, nathletes = 438), structural equation modeling analysis supported the hypothesised model in which athletes’ perceptions of parents’ behaviours mediated the relationship between parents’ reported behaviours and the athletes’ psychological variables conducive to their achievement in sport. The findings provide support for the parental involvement in sport model and demonstrate the role of perceptions of parents’ behaviours on young athletes’ cognitions in sport.

Development and preliminary psychometric properties of the Care Experience Feedback Improvement Tool (CEFIT).

Objective To develop a structurally valid and reliable, yet brief measure of patient experience of hospital quality of care, the Care Experience Feedback Improvement Tool (CEFIT). Also, to examine aspects of utility of CEFIT. Background Measuring quality improvement at the clinical interface has become a necessary component of healthcare measurement and improvement plans, but the effectiveness of measuring such complexity is dependent on the purpose and utility of the instrument used. Methods CEFIT was designed from a theoretical model, derived from the literature and a content validity index (CVI) procedure. A telephone population surveyed 802 eligible participants (healthcare experience within the previous 12 months) to complete CEFIT. Internal consistency reliability was tested using Cronbach's α. Principal component analysis was conducted to examine the factor structure and determine structural validity. Quality criteria were applied to judge aspects of utility. Results CVI found a statistically significant proportion of agreement between patient and practitioner experts for CEFIT construction. 802 eligible participants answered the CEFIT questions. Cronbach's α coefficient for internal consistency indicated high reliability (0.78). Interitem (question) total correlations (0.28–0.73) were used to establish the final instrument. Principal component analysis identified one factor accounting for 57.3% variance. Quality critique rated CEFIT as fair for content validity, excellent for structural validity, good for cost, poor for acceptability and good for educational impact. Conclusions CEFIT offers a brief yet structurally sound measure of patient experience of quality of care. The briefness of the 5-item instrument arguably offers high utility in practice. Further studies are needed to explore the utility of CEFIT to provide a robust basis for feedback to local clinical teams and drive quality improvement in the provision of care experience for patients. Further development of aspects of utility is also required.

Children's Separation Anxiety Scale (CSAS): Psychometric Properties

This study describes the psychometric properties of the Children's Separation Anxiety Scale (CSAS), which assesses separation anxiety symptoms in childhood. Participants in Study 1 were 1,908 schoolchildren aged between 8 and 11. Exploratory factor analysis identified four factors: worry about separation, distress from separation, opposition to separation, and calm at separation, which explained 46.91% of the variance. In Study 2, 6,016 children aged 8–11 participated. The factor model in Study 1 was validated by confirmatory factor analysis. The internal consistency (α = 0.82) and temporal stability (r = 0.83) of the instrument were good. The convergent and discriminant validity were evaluated by means of correlations with other measures of separation anxiety, childhood anxiety, depression and anger. Sensitivity of the scale was 85% and its specificity, 95%. The results support the reliability and validity of the CSAS.

Psychometric properties of the oral health impact profile and new methodological approach

Objective: Evaluate the validity, reliability, and factorial invariance of the complete Portuguese version of the Oral Health Impact Profile (OHIP) and its short version (OHIP-14). Methods: A total of 1,162 adults enrolled in the Faculty of Dentistry of Araraquara/UNESP participated in the study; 73.1% were women; and the mean age was 40.7 ± 16.3 yr. We conducted a confirmatory factor analysis, where χ2/df, comparative fit index, goodness of fit index, and root mean square error of approximation were used as indices of goodness of fit. The convergent validity was judged from the average variance extracted and the composite reliability, and the internal consistency was estimated by Cronbach standardized alpha. The stability of the models was evaluated by multigroup analysis in independent samples (test and validation) and between users and nonusers of dental prosthesis. Results: We found best-fitting models for the OHIP-14 and among dental prosthesis users. The convergent validity was below adequate values for the factors “functional limitation” and “physical pain” for the complete version and for the factors “functional limitation” and “psychological discomfort” for the OHIP-14. Values of composite reliability and internal consistency were below adequate in the OHIP-14 for the factors “functional limitation” and “psychological discomfort.” We detected strong invariance between test and validation samples of the full version and weak invariance for OHIP-14. The models for users and nonusers of dental prosthesis were not invariant for both versions. Conclusion: The reduced version of the OHIP was parsimonious, reliable, and valid to capture the construct “impact of oral health on quality of life,” which was more pronounced in prosthesis users.

Development and psychometric evaluation of the basic electrocardiogram interpretation self-efficacy scale

Purpose: Research suggests that nurses and nursing students lack competence in basic electrocardiogram (ECG) interpretation. Self-efficacy is considered to be paramount in the development of one's competence. The aim of this study was to develop and psychometrically evaluate a scale to assess self-efficacy of nursing students in basic ECG interpretation. Materials and methods: Observational cross-sectional study with a convenience sample of 293 nursing students. The basic ECG interpretation self-efficacy scale (ECG-SES) was developed and psychometrically tested in terms of reliability (internal consistency and temporal stability) and validity (content, criterion and construct). The ECG-SES’ internal consistency was explored by calculating the Cronbach's alpha coefficient (α); its temporal stability was investigated by calculating the Pearson correlation coefficient (r) between the participants’ results on a test–retest separated by a 4-week interval. The content validity index of the items (I-CVI) and the scale (S-CVI) was calculated based on the reviews of a panel of 16 experts. Criterion validity was explored by correlating the participants’ results on the ECG-SES with their results on the New General Self-Efficacy Scale (NGSE). 1 Construct validity was investigated by performing Principal Component Analysis (PCA) and known-group analysis. Results: The excellent reliability of the ECG-SES was evidenced by its internal consistency (α = 0.98) and its temporal stability at the 4-week re-test (r = 0.81; p < 0.01). The ECG-SES’ content validity was also excellent (all items’ I-CVI = 0.94–1; S-CVI = 0.99). A strong, significant correlation between the NGSE and the ECG-SES (r = 0.70; p < 0.01) showed its criterion validity. Corroborating the ECG-SES’ construct validity, PCA revealed that all its items loaded on a single factor that explained 74.6% of the total variance found. Furthermore, known-groups analysis showed the ECG-SES’ ability to detect expected differences in self-efficacy between groups with different training experiences (p < 0.01). Conclusion: The ECG-SES showed excellent psychometric properties for measuring the self-efficacy of nursing students in basic ECG interpretation.