Comparing poverty and deprivation dynamics:Issues of reliability and validity

In this paper we seek to establish if earlier findings relating to the relationship between income poverty persistence and deprivation persistence could be due to a failure to take measurement error into account. To address this question, we apply a model of dynamics incorporating structural and error components. Our analysis shows a general similarity between latent poverty and deprivation dynamics. In both cases we substantially over-estimate the probability of exiting from poverty or deprivation. We observe a striking similarity across dimensions for both observed and latent outcomes. In both cases levels of poverty and deprivation persistence are higher for the latent case. However, there is no evidence that earlier results relating to the differences in the determinants of poverty and deprivation persistence are a consequence of differential patterns of reliability. Taking measurement error into account seems more likely to accentuate rather than diminish the contrasts highlighted by earlier research. Since longitudinal differences relating to poverty and deprivation cannot be accounted for by measurement error, it seems that we must accept that we are confronted with issues relating to validity rather than reliability. Even where we measure these dimensions over reasonable periods of time and allow for measurement error, they continue to tap relatively distinct phenomenon. Thus, if measures of persistent poverty are to constitute an important component of EU social indicators, a strong case can be made for including parallel measures of deprivation persistence and continuing to explore the relationship between them. © Springer Science+Business Media, Inc. 2006.

Content validity of the Illness Perception Questionnaire-Revised for people with Type 2 Diabetes: A Think-Aloud study

Objectives: To access the cognitions of adults with type 2 diabetes whilst completing items on the Illness Perceptions Questionnaire – Revised (IPQ-R). To determine whether these cognitions are congruent with the meaning of items and subscales as interpreted by researchers and clinicians using the IPQ-R and to identify the nature and extent of problems that individuals experience when completing the IPQ-R.
Design: Participants (n=36) were recruited from a primary care diabetes clinic and a hospital diabetes clinic. They were asked to complete the IPQ-R using a ‘think-aloud’ methodology.
Main Outcome Measures: Transcripts were analysed to identify instances where participants expressed problems with item completion, or where there was inconsistency between verbal and written responses.
Results: The most problematic subscales were those of ‘personal control’ and ‘consequences’.
Conclusion: Generally, participants found the IPQ-R unproblematic. However, participants had problems with the concept of ‘cure’ and ‘symptoms’ in the context of type 2 diabetes, and with the negative phrasing used in some items. These findings have important implications for the interpretation of IPQ-R scores, particularly when the IPQ-R is used as the basis for individualised interventions among people with type 2 diabetes.

Is it painful or not?:Discriminant validity of the Behavioral Indicators of Infant Pain (BIIP) scale

To evaluate the ability of the Behavioral Indicators of Infant Pain (BIIP) scale to discriminate between skin-breaking and nonskin breaking procedures, and to identify sensitized pain responses in preterm infants in the neonatal intensive care unit (NICU).

Examining the Neurocognitive Validity of Commercially Available, Smartphone-Based Puzzle Games

Cognitive assessment typically involves assessing a person’s cognitive performance in unfamiliar and ar-guably unnatural clinical surroundings. User-centred approaches to assessment and monitoring, driven by issues such as enjoyability and familiarity, are largely absent. Everyday technologies, for example, smartphones represent an opportunity to obtain an objective assessment of a person’s cognitive capabili-ties in a non-threatening, discreet and familiar way, e.g. by everyday puzzle games undertaken as a leisure activity at home. We examined the strength of relationships that exist between performance on common puzzle games and standard measures of neuropsychological performance. Twenty-nine participants, aged 50 - 65 years, completed a comprehensive neuropsychological test battery and played three smart-phone-based puzzle games in triplicate: a picture puzzle [Matches Plus], a word puzzle [Jumbline] and a number puzzle [Sudoku]. As anticipated, a priori, significant correlations were observed between scores on a picture puzzle and visual memory test (r = 0.49; p = 0.007); a word puzzle and estimated verbal IQ (r = 0.53; p = 0.003) and verbal learning (r = 0.30; p = 0.039) tests; and a number puzzle and reason-ing/problem solving test (r = 0.42; p = 0.023). Further analyses making allowance for multiple compari-sons identified a significant unanticipated correlation (r = 0.49; p = 0.007) between number puzzle scores and a measure of nonverbal working memory. Performance on these smartphone-based games was in-dicative of relative cognitive ability across several cognitive domains at a fixed time point. Smart-phone-based, everyday puzzle games may offer a valid, portable measure of assessing and monitoring cognition in older adults.

Our Construct or Theirs?

Identity is relational and a construct, and is expressed in a myriad of ways. For example, material culture and its pluralist meanings have been readily manipulated by humans in a prehistoric context in order to construct personal and group identities. Artefacts were often from or reminiscent of far-flung places and were used to demonstrate membership of an (imagined) regional, or European community. Earthworks frequently archive maximum visual impact through elaborate ramparts and entrances with the minimum amount of effort, indicating that the construction of identities were as much in the eye of the perceivor, as of the perceived. Variations in domestic architectural style also demonstrate the malleability of identity, and the prolonged, intermittent use of particular places for specific functions indicates that the identity of place is just as important in our archaeological understanding as the identity of people. By using a wide range of case studies, both temporally and spatially, these thought processes may be explored further and diachronic and geographic patterns in expressions of identity investigated.

Measuring math anxiety in Italian college and high school students: Validity, reliability and gender invariance of the Abbreviated Math Anxiety Scale (AMAS)

Given that the ability to manage numbers is essential in a modern society, mathematics anxiety – which has been demonstrated to have unfortunate consequences in terms of mastery of math – has become a subject of increasing interest, and the need to accurately measure it has arisen. One of the widely employed scales to measure math anxiety is the Abbreviated Math Anxiety Scale (AMAS) (Hopko, Mahadevan, Bare & Hunt, 2003). The first aim of the present paper was to confirm the factor structure of the AMAS when administered to Italian high school and college students, and to test the invariance of the scale across educational levels. Additionally, we assessed the reliability and validity of the Italian version of the scale. Finally, we tested the invariance of the AMAS across genders. The overall findings provide evidence for the validity and reliability of the AMAS when administered to Italian students.

"...and they won land among the Picts by friendly treaty or the sword": How a re-examination of early historical sources and an analysis of early medieval settlement in north Co. Antrim confirms the validity of traditional accounts of Dál Riatic migration to Scotland from Ulster

Much recent scholarship has been critical of the concept of a Dál Riatic migration to, or colonisation of, Argyll. Scepticism of the accuracy of the early medieval accounts of this population movement, arguing that these are late amendments to early sources, coupled with an apparent lack of archaeological evidence for such a migration have led to its rejection. It is argued here, however, that this rejection has been based on too narrow a reading of historical sources and that there are several early accounts which, while differing in detail, agree on one point of substance, that the origin of Scottish Dál Riata lies in Ireland. Also, the use of archaeological evidence to suggest no migration to Argyll by the Dál Riata is flawed, misunderstanding the nature of early migrations and how they might be archaeologically identified, and it's proposed that there is actually quite a lot of evidence for migration to Argyll by the Dál Riata, in the form of settlement and artefactural evidence, but that it is to be found in Ireland through the mechanism of counterstream migration, rather than in Scotland.

Dimensionality, Reliability and Validity of the InterRAI Depression Rating Scale in a Canadian Palliative Care Population

A reliable and valid instrument is needed to screen for depression in palliative patients. The interRAI Depression Rating Scale (DRS) is based on seven items in the interRAI Palliative Care instrument. This study is the first to explore the dimensionality, reliability and validity of the DRS in a palliative population. Palliative home care patients (n = 5,175) residing in Ontario (Canada) were assessed with the interRAI Palliative Care instrument. Exploratory factor analysis and Mokken scale analysis were used to identify candidate conceptual models and evaluate scale homogeneity/performance. Confirmatory factor analysis compared models using standard goodness-of-fit indices. Convergent and divergent validity were investigated by examining polychoric correlations between the DRS and other items. The “known groups” test determined if the DRS meaningfully distinguished among client subgroups. The non-hierarchical two factor model showed acceptable fit with the data, and ordinal alpha coefficients of 0.83 and 0.82 were observed for the two DRS subscales. Omega hierarchical (ωh) was 0.78 for the bifactor model, with the general factor explaining three quarters of the common variance. Despite the multidimensionality evident in the factor analyses, bifactor modelling and the Mokken homogeneity coefficient (0.34) suggest that the DRS is a coherent scale that captures important information on sub-constructs of depression (e.g., somatic symptoms). Higher correlations were seen between the DRS and mood and psychosocial well-being items, and lower correlations with functional status and demographic variables. The DRS distinguished in the expected manner for known risk factors (e.g., social support, pain). The results suggest that the DRS is primarily unidimensional and reliable for use in screening for depression in palliative care patients.

Validity of the Global Physical Activity Questionnaire (GPAQ) in assessing levels and change in moderate-vigorous physical activity and sedentary behaviour

Background

Feasible, cost-effective instruments are required for the surveillance of moderate-to-vigorous physical activity (MVPA) and sedentary behaviour (SB) and to assess the effects of interventions. However, the evidence base for the validity and reliability of the World Health Organisation-endorsed Global Physical Activity Questionnaire (GPAQ) is limited. We aimed to assess the validity of the GPAQ, compared to accelerometer data in measuring and assessing change in MVPA and SB.

Participants (n = 101) were selected randomly from an on-going research study, stratified by level of physical activity (low, moderate or highly active, based on the GPAQ) and sex. Participants wore an accelerometer (Actigraph GT3X) for seven days and completed a GPAQ on Day 7. This protocol was repeated for a random sub-sample at a second time point, 3–6 months later. Analysis involved Wilcoxon-signed rank tests for differences in measures, Bland-Altman analysis for the agreement between measures for median MVPA and SB mins/day, and Spearman’s rho coefficient for criterion validity and extent of change.

95 participants completed baseline measurements (44 females, 51 males; mean age 44 years, (SD 14); measurements of change were calculated for 41 (21 females, 20 males; mean age 46 years, (SD 14). There was moderate agreement between GPAQ and accelerometer for MVPA mins/day (r = 0.48) and poor agreement for SB (r = 0.19). The absolute mean difference (self-report minus accelerometer) for MVPA was −0.8 mins/day and 348.7 mins/day for SB; and negative bias was found to exist, with those people who were more physically active over-reporting their level of MVPA: those who were more sedentary were less likely to under-report their level of SB. Results for agreement in change over time showed moderate correlation (r = 0.52, p = 0.12) for MVPA and poor correlation for SB (r = −0.024, p = 0.916).

Levels of agreement with objective measurements indicate the GPAQ is a valid measure of MVPA and change in MVPA but is a less valid measure of current levels and change in SB. Thus, GPAQ appears to be an appropriate measure for assessing the effectiveness of interventions to promote MVPA.

A quantum measurement paradigm for educational predicates: Implications for validity in educational measurement

The outcomes of educational assessments undoubtedly have real implications for students, teachers, schools and education in the widest sense. Assessment results are, for example, used to award qualifications that determine future educational or vocational pathways of students. The results obtained by students in assessments are also used to gauge individual teacher quality, to hold schools to account for the standards achieved by their students, and to compare international education systems. Given the current high-stakes nature of educational assessment, it is imperative that the measurement practices involved have stable philosophical foundations. However, this paper casts doubt on the theoretical underpinnings of contemporary educational measurement models. Aspects of Wittgenstein’s later philosophy and Bohr’s philosophy of quantum theory are used to argue that a quantum theoretical rather than a Newtonian model is appropriate for educational measurement, and the associated implications for the concept of validity are elucidated. Whilst it is acknowledged that the transition to a quantum theoretical framework would not lead to the demise of educational assessment, it is argued that, where practical, current high-stakes assessments should be reformed to become as ‘low-stakes’ as possible. The paper also undermines some of the pro high-stakes testing rhetoric that has a tendency to afflict education.