Internal consistency and validity of the Stroke Impact Scale 2.0 (SIS 2.0) and SIS - 16 in an Australian sample

Background and Purpose: The impact of stroke is multidimensional however standard stroke measures do not discriminate well when stroke patients are less physically impaired. The Stroke Impact Scale 2.0 (SIS 2.0) is a multidimensional measure of the impact of stroke but its'' psychometric properties require further testing. The SIS-16 is a measure of physical functioning designed to be more sensitive to differences in physical functioning than current stroke outcome measures but there is only preliminary information detailing its'' reliability and validity. The current study examined the internal consistency and validity of the SIS 2.0 and SIS-16 in an Australian sample of stroke patients. Methods: The SIS 2.0, SIS-16, World Health Organization Bref-Scale (WHOQOL-BREF) and Zung''s Self-Rating Depression Scale (SDS) were completed by 74 stroke patients in rural Victoria, Australia.
Results: The item convergent validity index indicated good item convergence of the SIS-16 and SIS 2.0 domains. The item discriminant validity index had only adequate divergence for most SIS 2.0 domains. Internal consistencies of the SIS-16 and SIS 2.0 domains were acceptable (agr = 0.87–0.95). Correlations between the SIS-16 and SIS 2.0 and the WHOQOL-BREF and SDS supported the convergent and discriminant validity of the SIS-16 and all the dimensions of the SIS 2.0 except lsquoParticipationrsquo which lacked discriminant validity. Conclusions: The SIS 2.0 and SIS-16 had good psychometric properties with support for the internal consistency and validity of both measures.

The social validity of interventions for promoting preschool childrens peer interactions

This analogue study investigated the social validity of classroom interventions designed to promote peer interactions in young children. The influences of child characteristics and diagnostic labels on social validity ratings were examined, as were the relationships among three components of social validity. Forty-two preschool teachers completed a 15-item social validity scale that yielded scores pertaining to intervention goals, procedures and effects. Participants rated three types of intervention, which varied according to individuals targeted and intensity of programming procedures, with reference to one of three case descriptions of children with social interaction problems. Half the participants were provided with the child’s diagnostic label. Results indicated that social validity is affected by interactions between intervention methods and child characteristics. The inclusion of diagnostic labels in the case descriptions had no effect on social validity ratings. Strong positive relationships were found among the components of social validity. Implications for the use of empirically validated interventions in inclusive preschools are discussed.

Development, reliability and validity of an instrument measuring the attitudes and knowledge of emergency department staff regarding the emergency nurse practitioner role

Introduction: The Emergency Department (ED) at The Northern Hospital is currently participating in the Victorian Department of Human Services funded Emergency Nurse Practitioner Project. This project aims to develop, implement and evaluate the Emergency Nurse Practitioner role in Victorian EDs. This led to a need to develop a specific data collection tool called The Northern Emergency Nurse Practitioner Staff Survey to examine the knowledge and attitudes of ED medical and nursing staff. This paper describes the development of The Northern Emergency Nurse Practitioner Staff Survey and presents the results of reliability and validity studies. Method: Twenty-five items were developed and piloted on a sample of 58 ED medical and nursing staff. Content and face validity were established by expert panel review. Reliability was established by tests of unidimensionality, exploratory factor analysis and internal consistency. Results: Four items were discarded because of low item to total correlation. Exploratory factor analysis of the remaining items revealed five factors with eigenvalues >1 and acceptable correlation coefficients that explained 76.7% of the variance. Cronbach’s coefficent α for these items was 0.926 indicating a high degree of internal consistency. The factors were titled to reflect the content domain of the items in each factor and the factors arranged in a logical sequence to form the final version of The Northern Emergency Nurse Practitioner Survey.

Early detection of emotional and behavioural problems in children with diabetes: the validity of the child health questionnaire as a screening instrument

Aims: To assess the validity of the Child Health Questionnaire (CHQ) as a screening tool for detecting 'at risk' emotional and behavioural  maladjustment in children with diabetes, using the Behaviour Assessment System for Children (BASC) as a gold standard measure. Methods: CHQ and BASC were administered to 103 parents of children with Type 1 diabetes, aged 7–12 years. Sub-scales of the two measures were compared using Pearson's bivariate correlations. CHQ sensitivity and specificity cut-points were optimized against the BASC borderline category using receiver operating characteristic curves. Results: The BASC Externalizing Problems scale correlated strongly with CHQ Behaviour, Global Behaviour, Mental Health, Family Activities and Family Cohesion scales (r-values −0.68, −0.54, −0.51, −0.59, and −0.42, respectively). BASC Internalizing Problems scale correlated strongly with CHQ Behaviour, Mental Health and Family Cohesion scales (r-values −0.40, −0.43 and −0.45, respectively). Using receiver operating characteristic curve analysis, the CHQ Mental Health scale most effectively identified children classified as borderline on the BASC Internalizing Problems scale (sensitivity 87%, specificity 78%), while the CHQ Global Behaviour scale most effectively identified children classified as borderline on the BASC Externalizing Problems scale (sensitivity 73%, specificity 82%). Conclusions: Significant correlations were seen between the CHQ Global Behaviour and Mental Health scales and the BASC Externalizing and Internalizing scales, respectively. Sequential use of the CHQ, as a screening tool, followed by an established mental health measure such as the BASC, may help identify children with diabetes 'at risk' for chronic maladjustment and poor health outcomes.

The factorial validity of emotional intelligence : an unresolved issue

An unresolved but pertinent issue in the field of emotional intelligence (EI) is factorial validity. Numerous studies have investigated this issue (Gignac, 2005; Mayer, Salovey, Caruso, & Sitarenios, 2003; Petrides & Furnham, 2000; Saklofske, Austin, & Minski, 2003), but most are based on correlations among subscale scores from relevant measures, making the implicit assumption that subscale scores are unidimensional, rather than questioning the structure of subscales themselves. Accordingly, the present study adopts the Anderson and Gerbing (1988) two-step strategy of first considering the structure within subscales before examining the relationship between subscales. An evaluation was undertaken using the Emotional Intelligence Scale (EIS, Schutte et al., 1998), the Work Profile Questionnaire – Emotional Intelligence Version (WQPei, Cameron, 1999) and the Mayer–Salovey–Caruso Emotional Intelligence Test (MSCEIT V.2., Mayer, Salovey, & Caruso, 1999b). Results were characterised by instability, heterogeneity and inconsistency. Specifically, the EIS was not found to form the homogenous structure postulated by authors. Similarly, support was not found for the seven factor model of the WPQei. Large discrepancies exist between the one, two and four factor models described by Mayer et al. (2003) for the MSCEIT V.2. and the 21 components revealed at the primary level in the current analyses. Additionally, reliability statistics for the MSCEIT V.2. were less than optimal. Questions remain regarding the clarity, reliability and validity of the instruments examined.

Spurious precision : variability in procedural validity of diagnostic procedures in psychotic disorders and implications for research and treatment

Very few studies have quantified the level of agreement among alternative diagnostic procedures that use a common set of fixed operational criteria. The authors examined the procedural validity of four independent methods of assigning DSM-III-R diagnoses of psychotic disorders. METHOD: The research was conducted as a satellite study to the DSM-IV Field Trial for Schizophrenia and Related Psychotic Disorders. The setting was the National Health and Medical Research Council Schizophrenia Research Unit's Early Psychosis Prevention and Intervention Centre, which focuses on first-episode psychosis. Consecutively admitted patients (N = 50) were assessed by independent raters who used four different procedures to determine a DSM-III-R diagnosis. These procedures were 1) the diagnostic instrument developed for the DSM-IV field trial, 2) the Royal Park Multidiagnostic Instrument for Psychosis, 3) the Munich Diagnostic Checklists, and 4) a consensus DSM-III-R diagnosis assigned by a team of clinician researchers who were expert in the use of diagnostic criteria. RESULTS: Concordance between pairs of diagnostic procedures was only moderate. Corresponding levels of percent agreement, however, ranged from 66% to 76%, with converse misclassification rates of 24%-34% (assuming one procedure to be "correct"). CONCLUSIONS: These findings have significant research and clinical implications. Despite the introduction of operationally defined diagnoses, there remained an appreciable level of differential classification or misclassification arising from variability in the method of assigning the diagnostic criteria rather than the criteria themselves. Such misclassification may impede neurobiological research and have harmful clinical effects on patients with first-episode psychosis.

The procedural validity of retrospective case note diagnosis

Objective: Deriving diagnoses from retrospective case note examination is a common practice in psychiatric research. The Operational Criteria (OPCRIT) diagnostic checklist is essentially a checklist built up of operational criteria defined by a comprehensive glossary and is designed to assign reliable diagnoses from case notes. However, the validity of such a procedure compared with procedures involving prospective assessment has never been tested. We examined the procedural validity of the OPCRIT diagnostic system in relation to four other diagnostic procedures mostly employing prospectively gathered information.

Method: Three experienced psychopathology raters rated the case notes and clinical abstracts, using the OPCRIT method of diagnostic assignment, of 50 subjects who had participated in an early procedural validity study as an adjunct to the DSM-IV Field Trial for psychotic disorders. The setting was the Early Psychosis Prevention and Intervention Centre (EPPIC), which focuses on first episode psychosis.

Results: The pairwise concordance with the other procedures for DSM-III-R diagnoses assigned by OPCRIT using ratings derived from either the clinical abstracts or the case notes was found to be only poor to moderate when compared with the pairwise concordance of the four other procedures. The per cent agreement between OPCRIT clinical abstracts diagnoses and the other procedures ranged from 49% to 60% with kappa values between 0.30 and 0.45, and for OPCRIT case note diagnoses and the other procedures the per cent agreement range was between 44% and 57% and the kappa values were between 0.35 and 0.49.

Conclusions: The procedural validity of diagnoses assigned via the application of checklists of operational criteria to case notes and clinical abstracts alone is unacceptably poor. Such sources need to be buttressed by other data, particularly direct patient interview and informant material.

Assessing the validity of impact pathways for child labour and well-being in social life cycle assessment

Background, aim and scope: Assuming that the goal of social life cycle assessment (SLCA) is to assess damage and benefits on its ‘area of protection’ (AoP) as accurately as possible, it follows that the impact pathways, describing the cause effect relationship between indicator and the AoP, should have a consistent theoretical foundation so the inventory results can be associated with a predictable damage or benefit to the AoP. This article uses two concrete examples from the work on SLCA to analyse to what extent this is the case in current practice. One considers whether indicators included in SLCA approaches can validly assess impacts on the well-being of the stakeholder, whereas the other example addresses whether the ‘incidence of child labour’ is a valid measure for impacts on the AoPs.

Materials and methods: The theoretical basis for the impact pathway between the relevant indicators and the AoPs is analysed drawing on research from relevant scientific fields.

Results:   The examples show a lack of valid impact pathways in both examples. The first example shows that depending on the definition of ‘well-being’, the assessment of impacts on well-being of the stakeholder cannot be performed exclusively with the type of indicators which are presently used in SLCA approaches. The second example shows that the mere fact that a child is working tells little about how this may damage or benefit the AoPs, implying that the normally used indicator; ‘incidence of child labour’ lacks validity in relation to predicting damage or benefit on the AoPs of SLCA.

Discussion: New indicators are proposed to mitigate the problem of invalid impact pathways. However, several problems arise relating to difficulties in getting data, the usability of the new indicators in management situations, and, in relation to example one, boundary setting issues.

Conclusions: The article shows that it is possible to assess the validity of the impact pathways in SLCA. It thereby point to the possibility of utilising the same framework that underpins the environmental LCA in this regard. It also shows that in relation to both of the specific examples investigated, the validity of the impact pathways may be improved by adopting other indicators, which does, however, come with a considerable ‘price’.

Recommendations and perspectives: It is argued that there is a need for analysing impact pathways of other impact categories often included in SLCA in order to establish indicators that better reflect actual damage or benefit to the AoPs.

Reliability and validity of the system for observing children's activity and relationships during play (SOCARP)

Background: Children frequently engage in diverse activities that are broadly defined as play, but little research has documented children’s activity levels during play and how they are influenced by social contexts. Assessing potentially modifiable conditions that influence play behavior is needed to design optimal physical activity interventions.

Methods: System for Observing Children’s Activity and Relationships during Play (SOCARP) was developed to simultaneously assess children’s physical activity, social group sizes, activity type, and social behavior during play. One hundred and fourteen children (48 boys, 66 girls; 42% overweight) from 8 elementary schools were observed during recess over 24 days, with 12 days videotaped for reliability purposes. Ninety-nine children wore a uni-axial accelerometer during their observation period.

Results: Estimated energy expenditure rates from SOCARP observations and mean accelerometer counts were significantly correlated (r = .67; P < .01), and interobserver reliabilities (ie, percentage agreement) for activity level (89%), group size (88%), activity type (90%) and interactions (88%) met acceptable criteria. Both physical activity and social interactions were influenced by group size, activity type, and child gender and body weight status.

Conclusions: SOCARP is a valid and reliable observation system for assessing physical activity and play behavior in a recess context.

The convergent validity of two sensory processing scales used with school - age children : comparing the sensory profile and the sensory processing measure

Abstract: Aim: To investigate the convergent validity between the Sensory Profile, the Sensory Profile School Companion, and the Home and Main Classroom Forms of the Sensory Processing Measure.

Method: Thirty mothers completed the Sensory Profile and the Sensory Processing Measure - Home Form on one child each. Nineteen teachers of the same children completed the Sensory Profile School Companion and the Sensory Processing Measure - Main Classroom Form.

Results: The Sensory Profile and the Sensory Processing Measure - Home Form were significantly correlated (rho=0.86, p less-than .01). The Sensory Profile School Companion and Sensory Processing Measure - Main Classroom Form were also significantly correlated (rho=.74, p less-than .01).

Conclusion: The two sets of sensory processing scales had moderate levels of convergent validity.

The validity of the CGI severity and improvement scales as measures of clinical effectiveness suitable for routine clinical use

Objective The Clinical Global Impression Scale (CGI) is established as a core metric in psychiatric research. This study aims to test the validity of CGI as a clinical outcome measure suitable for routine use in a private inpatient setting.

Methods The CGI was added to a standard battery of routine outcome measures in a private psychiatric hospital. Data were collected on consecutive admissions over a period of 24 months, which included clinical diagnosis, demographics, service utilization and four routine measures (CGI, HoNOS, MHQ-14 and DASS-21) at both admission and discharge. Descriptive and comparative data analyses were performed.

Results Of 786 admissions in total, there were 624 and 614 CGI-S ratings completed at the point of admission and discharge, respectively, and 610 completed CGI-I ratings. The admission and discharge CGI-S scores were correlated (r = 0.40), and the indirect improvement measures obtained from their differences were highly correlated with the direct CGI-I scores (r = 0.71). The CGI results reflected similar trends seen in the other three outcome measures.

Conclusions The CGI is a valid clinical outcome measure suitable for routine use in an inpatient setting. It offers a number of advantages, including its established utility in psychiatric research, sensitivity to change, quick and simple administration, utility across diagnostic groupings, and reliability in the hands of skilled clinicians.

The validity of the 21-item version of the depression anxiety stress scales as a routine outcome measure

Objective: This study aimed to test the validity of the 21-item Depression Anxiety Stress Scales (DASS-21) as a routine clinical outcome measure in the private in-patient setting. We hypothesized that it would be a suitable routine outcome instrument in this setting.

Method: All in-patients treated at a private psychiatric hospital over a period of 24 months were included in the study. Data were collected on demographics, service utilization, diagnosis and a set of four routine measures both at admission and discharge. These measures consisted of the Clinical Global Impressions (CGI) scales, Health of the Nation Outcome Scales (HoNOS), the Mental Health Questionnaire (MHQ-14) and DASS-21. The results of these measures were compared.

Results: Of 786 admissions in total, the number of fully completed (ie paired admission and discharge) data sets for the DASS-21 depression, anxiety and stress subscales were 337, 328 and 347, respectively. All subscales showed statistically significant reductions in mean scores from admission to discharge (P < 0.001) and were significantly correlated with all MHQ-14 subscales and significantly related to CGI scale categories. The total DASS-21 and total HoNOS scores were also significantly correlated.

Conclusions: The findings from the present study support the validity of DASS-21 as a routine clinical outcome measure in the private in-patient setting.

Convergent validity of two visual motor integration tests

Background: Occupational therapists often assess visual motor integration (VMI) skills. It is, therefore, imperative that therapists use VMI tests with robust measurement properties.

Objective: This study examined the convergent validity of two VMI tests used to assess children, adolescents and adults.

Method: Three groups of healthy participants (n = 153) completed the Beery-Buktenica Developmental Test of Visual-Motor Integration (DTVMI) and the Full Range Test of Visual Motor Integration (FRTVMI). Seventy-three children aged 5-10 years (37 males and 36 females; mean age 7.5 years, SD = 2.20), 19 adolescents aged 11-17 years (8 males and 11 females; 13.1 years, SD = 2.16), and 61 adults (18 males and 43 females; mean age 31.82 years, SD = 11.20) completed the DTVMI and the FRTVMI. Spearman rho correlation coefficients were used to investigate whether each pair of the VMI test scores for each of the three participant age groups were associated.

Results: The Spearman rho correlation coefficients between all three versions of the DTVMI and FRTVMI were statistically significant. For the child group, the correlation coefficient was rho = 0.70 (p<0.000), while the correlation between the VMI scores obtained by the adolescent group on the two tests was rho = 0.77 (p<0.000). For the adult participant group, the correlation coefficient between the DTVMI and the FRTVMI was rho = 0.70 (p<0.000).

Conclusion: The VMI scores obtained by the three participant age groups on the DTVMI and the FRTVMI were all significantly correlated with each other. Overall, the DTVMI and the FRTVMI exhibited large levels of convergent validity with each other, indicating that the two tests appear to measure similar visual-motor integration constructs.

The long-term predictive validity of the consumers' need for uniqueness scale

This research examines the long-term ability of the consumers’ need for uniqueness measure to predict consumers’ valued possessions, shopping behaviors, and experiences. Consumers’ need for uniqueness reflects individual differences in counterconformity motivation (i.e., the pursuit of differentness relative to others) that is manifest in consumer responses. This research reports on 2 longitudinal assessments of the scale's predictive validity using data collected at 4 stages from a consumer mail-survey sample. One year following initial responses to the trait measure, respondents participated in a second survey in which they photographed and rank ordered their valued possessions. In a third survey, the photographs served as cues to obtain responses to ratings of possession benefits and to questions regarding acquisition of the possession. Overall, the results support a number of hypotheses regarding the relation of consumers’ need for uniqueness to salient possession benefits, types of goods comprising consumers’ valued possessions, and shopping behaviors associated with acquiring valued possessions. Two years following initial responses to the trait measure, members of the consumer mail panel completed a fourth survey, indicating their participation in various uniqueness-enhancing consumer activities. Results support the stability and predictive validity of the measure over the 2-year period. Because the design of possessions and endorsed uniqueness-enhancing activities considered in the consumer mail panel studies varied across individuals, an additional study was conducted that demonstrated the scale's ability to predict individuals’ counterconformity responses to the same choice situation.

Reliability and validity of psychosocial and environmental correlates measures of physical activity and screen-based behaviors among Chinese children in Hong Kong

Background : Insufficient participation in physical activity and excessive screen time have been observed among Chinese children. The role of social and environmental factors in shaping physical activity and sedentary behaviors among Chinese children is under-investigated. The purpose of the present study was to assess the reliability and validity of a questionnaire to measure child- and parent-reported psychosocial and environmental correlates of physical activity and screen-based behaviors among Chinese children in Hong Kong.

Methods : A total of 303 schoolchildren aged 9-14 years and their parents volunteered to participate in this study and 160 of them completed the questionnaire twice within an interval of 10 days. Intraclass correlation coefficients (ICCs), kappa statistics, and percent agreement were performed to evaluate test-retest reliability of the continuous and categorical variables, respectively. Exploratory factor analyses (EFAs) were conducted to assess convergent validity of the emergent scales. Cronbach's alpha and ICCs were performed to assess internal and test-retest reliability of the emergent scales. Criterion validity was assessed by correlating psychosocial and environmental measures with self-reported physical activity and screen-based behaviors, measured by a validated questionnaire.

Results : Reliability statistics for both child- and parent-reported continuous variables showed acceptable consistency for all of the ICC values greater than 0.70. Kappa statistics showed fair to perfect test-retest reliability for the categorical items. Adequate internal consistency and test-retest reliability were observed in most of the emergent scales. Criterion validity assessed by correlating psychosocial and environmental measures with child-reported physical activity found associations with physical activity in the self-efficacy scale (r = 0.25, P < 0.05), the peer support for physical activity scale (r = 0.25, P < 0.05) and home physical activity environmental (r = 0.14, P < 0.05). Children's screen-based behaviors were associated with the family support for physical activity scale (r = -0.22, P < 0.05) and parental role modeling of TV (r = 0.12, P = 0.053).

Conclusions : The findings provide psychometric support for using this questionnaire for examining psychosocial and environmental correlates of physical activity and screen-based behaviors among Chinese children in Hong Kong. Further research is needed to develop more robust measures based on the current questionnaire, especially for peer influence on physical activity and parental rules on screen-based behaviors.