Examining the Neurocognitive Validity of Commercially Available, Smartphone-Based Puzzle Games

Cognitive assessment typically involves assessing a person’s cognitive performance in unfamiliar and ar-guably unnatural clinical surroundings. User-centred approaches to assessment and monitoring, driven by issues such as enjoyability and familiarity, are largely absent. Everyday technologies, for example, smartphones represent an opportunity to obtain an objective assessment of a person’s cognitive capabili-ties in a non-threatening, discreet and familiar way, e.g. by everyday puzzle games undertaken as a leisure activity at home. We examined the strength of relationships that exist between performance on common puzzle games and standard measures of neuropsychological performance. Twenty-nine participants, aged 50 - 65 years, completed a comprehensive neuropsychological test battery and played three smart-phone-based puzzle games in triplicate: a picture puzzle [Matches Plus], a word puzzle [Jumbline] and a number puzzle [Sudoku]. As anticipated, a priori, significant correlations were observed between scores on a picture puzzle and visual memory test (r = 0.49; p = 0.007); a word puzzle and estimated verbal IQ (r = 0.53; p = 0.003) and verbal learning (r = 0.30; p = 0.039) tests; and a number puzzle and reason-ing/problem solving test (r = 0.42; p = 0.023). Further analyses making allowance for multiple compari-sons identified a significant unanticipated correlation (r = 0.49; p = 0.007) between number puzzle scores and a measure of nonverbal working memory. Performance on these smartphone-based games was in-dicative of relative cognitive ability across several cognitive domains at a fixed time point. Smart-phone-based, everyday puzzle games may offer a valid, portable measure of assessing and monitoring cognition in older adults.

Measuring math anxiety in Italian college and high school students: Validity, reliability and gender invariance of the Abbreviated Math Anxiety Scale (AMAS)

Given that the ability to manage numbers is essential in a modern society, mathematics anxiety – which has been demonstrated to have unfortunate consequences in terms of mastery of math – has become a subject of increasing interest, and the need to accurately measure it has arisen. One of the widely employed scales to measure math anxiety is the Abbreviated Math Anxiety Scale (AMAS) (Hopko, Mahadevan, Bare & Hunt, 2003). The first aim of the present paper was to confirm the factor structure of the AMAS when administered to Italian high school and college students, and to test the invariance of the scale across educational levels. Additionally, we assessed the reliability and validity of the Italian version of the scale. Finally, we tested the invariance of the AMAS across genders. The overall findings provide evidence for the validity and reliability of the AMAS when administered to Italian students.

"...and they won land among the Picts by friendly treaty or the sword": How a re-examination of early historical sources and an analysis of early medieval settlement in north Co. Antrim confirms the validity of traditional accounts of Dál Riatic migration to Scotland from Ulster

Much recent scholarship has been critical of the concept of a Dál Riatic migration to, or colonisation of, Argyll. Scepticism of the accuracy of the early medieval accounts of this population movement, arguing that these are late amendments to early sources, coupled with an apparent lack of archaeological evidence for such a migration have led to its rejection. It is argued here, however, that this rejection has been based on too narrow a reading of historical sources and that there are several early accounts which, while differing in detail, agree on one point of substance, that the origin of Scottish Dál Riata lies in Ireland. Also, the use of archaeological evidence to suggest no migration to Argyll by the Dál Riata is flawed, misunderstanding the nature of early migrations and how they might be archaeologically identified, and it's proposed that there is actually quite a lot of evidence for migration to Argyll by the Dál Riata, in the form of settlement and artefactural evidence, but that it is to be found in Ireland through the mechanism of counterstream migration, rather than in Scotland.

Dimensionality, Reliability and Validity of the InterRAI Depression Rating Scale in a Canadian Palliative Care Population

A reliable and valid instrument is needed to screen for depression in palliative patients. The interRAI Depression Rating Scale (DRS) is based on seven items in the interRAI Palliative Care instrument. This study is the first to explore the dimensionality, reliability and validity of the DRS in a palliative population. Palliative home care patients (n = 5,175) residing in Ontario (Canada) were assessed with the interRAI Palliative Care instrument. Exploratory factor analysis and Mokken scale analysis were used to identify candidate conceptual models and evaluate scale homogeneity/performance. Confirmatory factor analysis compared models using standard goodness-of-fit indices. Convergent and divergent validity were investigated by examining polychoric correlations between the DRS and other items. The “known groups” test determined if the DRS meaningfully distinguished among client subgroups. The non-hierarchical two factor model showed acceptable fit with the data, and ordinal alpha coefficients of 0.83 and 0.82 were observed for the two DRS subscales. Omega hierarchical (ωh) was 0.78 for the bifactor model, with the general factor explaining three quarters of the common variance. Despite the multidimensionality evident in the factor analyses, bifactor modelling and the Mokken homogeneity coefficient (0.34) suggest that the DRS is a coherent scale that captures important information on sub-constructs of depression (e.g., somatic symptoms). Higher correlations were seen between the DRS and mood and psychosocial well-being items, and lower correlations with functional status and demographic variables. The DRS distinguished in the expected manner for known risk factors (e.g., social support, pain). The results suggest that the DRS is primarily unidimensional and reliable for use in screening for depression in palliative care patients.

Validity of the Global Physical Activity Questionnaire (GPAQ) in assessing levels and change in moderate-vigorous physical activity and sedentary behaviour

Background

Feasible, cost-effective instruments are required for the surveillance of moderate-to-vigorous physical activity (MVPA) and sedentary behaviour (SB) and to assess the effects of interventions. However, the evidence base for the validity and reliability of the World Health Organisation-endorsed Global Physical Activity Questionnaire (GPAQ) is limited. We aimed to assess the validity of the GPAQ, compared to accelerometer data in measuring and assessing change in MVPA and SB.

Participants (n = 101) were selected randomly from an on-going research study, stratified by level of physical activity (low, moderate or highly active, based on the GPAQ) and sex. Participants wore an accelerometer (Actigraph GT3X) for seven days and completed a GPAQ on Day 7. This protocol was repeated for a random sub-sample at a second time point, 3–6 months later. Analysis involved Wilcoxon-signed rank tests for differences in measures, Bland-Altman analysis for the agreement between measures for median MVPA and SB mins/day, and Spearman’s rho coefficient for criterion validity and extent of change.

95 participants completed baseline measurements (44 females, 51 males; mean age 44 years, (SD 14); measurements of change were calculated for 41 (21 females, 20 males; mean age 46 years, (SD 14). There was moderate agreement between GPAQ and accelerometer for MVPA mins/day (r = 0.48) and poor agreement for SB (r = 0.19). The absolute mean difference (self-report minus accelerometer) for MVPA was −0.8 mins/day and 348.7 mins/day for SB; and negative bias was found to exist, with those people who were more physically active over-reporting their level of MVPA: those who were more sedentary were less likely to under-report their level of SB. Results for agreement in change over time showed moderate correlation (r = 0.52, p = 0.12) for MVPA and poor correlation for SB (r = −0.024, p = 0.916).

Levels of agreement with objective measurements indicate the GPAQ is a valid measure of MVPA and change in MVPA but is a less valid measure of current levels and change in SB. Thus, GPAQ appears to be an appropriate measure for assessing the effectiveness of interventions to promote MVPA.

A quantum measurement paradigm for educational predicates: Implications for validity in educational measurement

The outcomes of educational assessments undoubtedly have real implications for students, teachers, schools and education in the widest sense. Assessment results are, for example, used to award qualifications that determine future educational or vocational pathways of students. The results obtained by students in assessments are also used to gauge individual teacher quality, to hold schools to account for the standards achieved by their students, and to compare international education systems. Given the current high-stakes nature of educational assessment, it is imperative that the measurement practices involved have stable philosophical foundations. However, this paper casts doubt on the theoretical underpinnings of contemporary educational measurement models. Aspects of Wittgenstein’s later philosophy and Bohr’s philosophy of quantum theory are used to argue that a quantum theoretical rather than a Newtonian model is appropriate for educational measurement, and the associated implications for the concept of validity are elucidated. Whilst it is acknowledged that the transition to a quantum theoretical framework would not lead to the demise of educational assessment, it is argued that, where practical, current high-stakes assessments should be reformed to become as ‘low-stakes’ as possible. The paper also undermines some of the pro high-stakes testing rhetoric that has a tendency to afflict education.

The validity of critical thinking tests for predicting degree performance: A longitudinal study

This study explored the validity of using critical thinking tests to predict final psychology degree marks over and above that already predicted by traditional admission exams (A-levels). Participants were a longitudinal sample of 109 psychology students from a university in the United Kingdom. The outcome measures were: total degree marks; and end of year marks. The predictor measures were: university admission exam results (A-levels); critical thinking test scores (skills & dispositions); and non-verbal intelligence scores. Hierarchical regressions showed A-levels significantly predicted 10% of the final degree score and the 11-item measure of ‘Inference skills’ from the California Critical Thinking Skills Test significantly predicted an additional 6% of degree outcome variance. The findings from this study should inform decisions about the precise measurement constructs included in aptitude tests used in the higher education admission process.

Content Validity in Landscape Valuation: A transdisciplinary approach to valuing the landscape of the peninsula of Sorrento, Italy.

This paper addresses the representation of landscape complexity in stated preferences research. It integrates landscape ecology and landscape economics and conducts the landscape analysis in a three-dimensional space to provide ecologically meaningful quantitative landscape indicators that are used as variables for the monetary valuation of landscape in a stated preferences study. Expected heterogeneity in taste intensity across respondents is addressed with a mixed logit model in Willingness to Pay space. The results suggest that the integration of landscape ecology metrics in a stated preferences model provides useful insights for valuing landscape and landscape changes

Establishing the validity of English GP Patient Survey items evaluating out-of-hours care

Background A 2014 national audit used the English General Practice Patient Survey (GPPS) to compare service users’ experience of out-of-hours general practitioner (GP) services, yet there is no published evidence on the validity of these GPPS items. Objectives Establish the construct and concurrent validity of GPPS items evaluating service users’ experience of GP out-of-hours care. Methods Cross-sectional postal survey of service users (n=1396) of six English out-of-hours providers. Participants reported on four GPPS items evaluating out-of-hours care (three items modified following cognitive interviews with service users), and 14 evaluative items from the Out-of-hours Patient Questionnaire (OPQ). Construct validity was assessed through correlations between any reliable (Cochran's α>0.7) scales, as suggested by a principal component analysis of the modified GPPS items, with the ‘entry access’ (four items) and ‘consultation satisfaction’ (10 items) OPQ subscales. Concurrent validity was determined by investigating whether each modified GPPS item was associated with thematically related items from the OPQ using linear regressions. Results The modified GPPS item-set formed a single scale (α=0.77), which summarised the two-component structure of the OPQ moderately well; explaining 39.7% of variation in the ‘entry access’ scores (r=0.63) and 44.0% of variation in the ‘consultation satisfaction’ scores (r=0.66), demonstrating acceptable construct validity. Concurrent validity was verified as each modified GPPS item was highly associated with a distinct set of related items from the OPQ. Conclusions Minor modifications are required for the English GPPS items evaluating out-of-hours care to improve comprehension by service users. A modified question set was demonstrated to comprise a valid measure of service users’ overall satisfaction with out-of-hours care received. This demonstrates the potential for the use of as few as four items in benchmarking providers and assisting services in identifying, implementing and assessing quality improvement initiatives.