A psychometric analysis of reliability and validity of the index of learning styles (ILS)

Prior literature showed that Felder and Silverman learning styles model (FSLSM) was widely adopted to cater to individual styles of learners whether in traditional or Technology Enhanced Learning (TEL). In order to infer this model, the Index of Learning Styles (ILS) instrument was proposed. This research aims to analyse the soundness of this instrument in an Arabic sample. Data were integrated from different courses and years. A total of 259 engineering students participated voluntarily in the study. The reliability was analysed by applying internal construct reliability, inter-scale correlation, and total item correlation. The construct validity was also considered by running factor analysis. The overall results indicated that the reliability and validity of perception and input dimensions were moderately supported, whereas processing and understanding dimensions showed low internal-construct consistency and their items were weakly loaded in the associated constructs. Generally, the instrument needs further effort to improve its soundness. However, considering the consistency of the produced results of engineering students irrespective of cross-cultural differences, it can be adopted to diagnose learning styles.

Comparative performance of UK PFI and non-PFI hospitals

Since the first PFI hospital was established in 1994, many debates centred on the value for money and risk transfer in PFIs. Little concern is shown with PFI hospitals’ performance in delivering healthcare. Exploratory research was carried out to compare PFI with non‐PFI hospital performance. Five performance indicators were analysed to compare differences between PFI and non‐PFI hospitals, namely the length of waiting, the length of stay, MRSA infection rate, C difficile infection rate and patient experience. Data was collected from various government bodies. The results show that only some indexes measuring patient experience emerge statistically significant. This leads to a conclusion that PFI hospitals may not perform better than non‐PFI hospitals but they are not worse than non‐PFI hospitals in the delivery of services. However, future research needs to pay attention to reliability and validity of data sets currently available to undertake comparison.

Estimating reliability and resolution of probability forecasts through decomposition of the empirical score

References (20)Cited By (1)Export CitationAboutAbstract Proper scoring rules provide a useful means to evaluate probabilistic forecasts. Independent from scoring rules, it has been argued that reliability and resolution are desirable forecast attributes. The mathematical expectation value of the score allows for a decomposition into reliability and resolution related terms, demonstrating a relationship between scoring rules and reliability/resolution. A similar decomposition holds for the empirical (i.e. sample average) score over an archive of forecast–observation pairs. This empirical decomposition though provides a too optimistic estimate of the potential score (i.e. the optimum score which could be obtained through recalibration), showing that a forecast assessment based solely on the empirical resolution and reliability terms will be misleading. The differences between the theoretical and empirical decomposition are investigated, and specific recommendations are given how to obtain better estimators of reliability and resolution in the case of the Brier and Ignorance scoring rule.

Online dietary intake estimation : reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record

Background: Advances in nutritional assessment are continuing to embrace developments in computer technology. The online Food4Me food frequency questionnaire (FFQ) was created as an electronic system for the collection of nutrient intake data. To ensure its accuracy in assessing both nutrient and food group intake, further validation against data obtained using a reliable, but independent, instrument and assessment of its reproducibility are required. Objective: The aim was to assess the reproducibility and validity of the Food4Me FFQ against a 4-day weighed food record (WFR). Methods: Reproducibility of the Food4Me FFQ was assessed using test-retest methodology by asking participants to complete the FFQ on 2 occasions 4 weeks apart. To assess the validity of the Food4Me FFQ against the 4-day WFR, half the participants were also asked to complete a 4-day WFR 1 week after the first administration of the Food4Me FFQ. Level of agreement between nutrient and food group intakes estimated by the repeated Food4Me FFQ and the Food4Me FFQ and 4-day WFR were evaluated using Bland-Altman methodology and classification into quartiles of daily intake. Crude unadjusted correlation coefficients were also calculated for nutrient and food group intakes. Results: In total, 100 people participated in the assessment of reproducibility (mean age 32, SD 12 years), and 49 of these (mean age 27, SD 8 years) also took part in the assessment of validity. Crude unadjusted correlations for repeated Food4Me FFQ ranged from .65 (vitamin D) to .90 (alcohol). The mean cross-classification into “exact agreement plus adjacent” was 92% for both nutrient and food group intakes, and Bland-Altman plots showed good agreement for energy-adjusted macronutrient intakes. Agreement between the Food4Me FFQ and 4-day WFR varied, with crude unadjusted correlations ranging from .23 (vitamin D) to .65 (protein, % total energy) for nutrient intakes and .11 (soups, sauces and miscellaneous foods) to .73 (yogurts) for food group intake. The mean cross-classification into “exact agreement plus adjacent” was 80% and 78% for nutrient and food group intake, respectively. There were no significant differences between energy intakes estimated using the Food4Me FFQ and 4-day WFR, and Bland-Altman plots showed good agreement for both energy and energy-controlled nutrient intakes. Conclusions: The results demonstrate that the online Food4Me FFQ is reproducible for assessing nutrient and food group intake and has moderate agreement with the 4-day WFR for assessing energy and energy-adjusted nutrient intakes. The Food4Me FFQ is a suitable online tool for assessing dietary intake in healthy adults.

Associations between interpretation bias and depression in adolescents

Interpretation biases have been shown to play a role in adult depression and are a target in cognitive behavioural therapy. Adolescence is a key risk period for the development of depression and a period of rapid cognitive and emotional development but little research has investigated the relationship between interpretation biases and depression in adolescents. This study adapted a measure of interpretation bias, the Ambiguous Scenarios Test for Depression, for adolescents and evaluated its reliability and validity. A community sample of 206 young people aged 12 to 18 years completed a validated measure of depression symptoms (Mood and Feelings Questionnaires) and the adapted Ambiguous Scenarios Test. The Ambiguous Scenarios Test for Depression in Adolescents had good internal consistency and split half reliability. Depression symptoms were associated with participants’ ratings of the valence of ambiguous situations and with interpretation biases. Importantly, symptoms of depression and anxiety were independently associated with interpretation bias. This research suggests that interpretation biases can be measured in this age group, that negative interpretation biases exist in adolescents and that these are associated with depression symptoms.

Formative assessment in teacher education: the development of a diagnostic language test for trainee teachers of German

This article describes the development and validation of a diagnostic test of German and its integration in a programme of formative assessment during a one-year initial teacher-training course. The test focuses on linguistic aspects that cause difficulty for trainee teachers of German as a foreign language and assesses implicit and explicit grammatical knowledge as well as students' confidence in this knowledge. Administration of the test to 57 German speakers in four groups (first-year undergraduates, fourth-year undergraduates, postgraduate trainees, and native speakers) provided evidence of its reliability and validity.

A new rapid detection threshold method for use with older adults: reducing fatigue whilst maintaining accuracy

Taste and smell detection threshold measurements are frequently time consuming especially when the method involves reversing the concentrations presented to replicate and improve accuracy of results. These multiple replications are likely to cause sensory and cognitive fatigue which may be more pronounced in elderly populations. A new rapid detection threshold methodology was developed that quickly located the likely position of each individuals sensory detection threshold then refined this by providing multiple concentrations around this point to determine their threshold. This study evaluates the reliability and validity of this method. Findings indicate that this new rapid detection threshold methodology was appropriate to identify differences in sensory detection thresholds between different populations and has positive benefits in providing a shorter assessment of detection thresholds. The results indicated that this method is appropriate at determining individual as well as group detection thresholds.

The Cognitive Behaviour Therapy Scale for Children and Young People (CBTS-CYP): development and psychometric properties

Background: There is increased interest in developing training in cognitive behaviour therapy (CBT) with children and young people. However, the assessment of clinical competence has relied upon the use of measures such as the Cognitive Therapy Scale-Revised (CTSR: Blackburn et al., 2001) which has been validated to assess competence with adults. The appropriateness of this measure to assess competence when working with children and young people has been questioned. Aim: This paper describes the development and initial evaluation of the Cognitive Behaviour Therapy Scale for Children and Young People (CBTSCYP) developed specifically to assess competence in CBT with children and young people. Method: A cross section of child CBT practitioners (n = 61) were consulted to establish face validity. Internal reliability, convergent validity and discriminative ability were assessed in two studies. In the first, 12 assessors independently rated a single video using both the Cognitive Behaviour Therapy Scale for Children and Young People (CBTS-CYP) and Cognitive Therapy Scale-Revised (CTS-Revised: Blackburn et al., 2001). In the second, 48 different recordings of CBT undertaken with children and young people were rated on both the CBTS-CYP and CTS-R. Results: Face validity and internal reliability of the CBTS-CYP were high, and convergent validity with the CTS-R was good. The CBTS-CYP compared well with the CTSR in discriminative ability. Conclusion: The CBTS-CYP provides an appropriate way of assessing competence in using CBT with children and young people. Further work is required to assess robustness with younger children and the impact of group training in reducing interrater variations.

Development and validation of the Morphing Fear Questionnaire (MFQ)

Morphing fears (also called transformation obsessions) involve concerns that a person may become contaminated by and acquire undesirable characteristics of others. These symptoms are found in patients with OCD and are thought to be related to mental contamination. Given the high levels of distress and interference morphing fears can cause, a reliable and valid assessment measure is needed. This article describes the development and evaluation of the Morphing Fear Questionnaire (MFQ), a 13-item measure designed to assess for the presence and severity of morphing fears. A sample of 900 participants took part in the research. Of these, 140 reported having a current diagnosis of OCD (SR-OCD) and 760 reported never having had OCD (N-OCD; of whom 24 reported a diagnosis of an anxiety disorder and 23 reported a diagnosis of depression). Factor structure, reliability, and construct and criterion related validity were investigated. Exploratory and confirmatory factor analyses supported a one-factor structure replicable across the N-OCD and SR-OCD group. The MFQ was found to have high internal consistency and good temporal stability, and showed significantly greater associations with convergent measures (assessing obsessive-compulsive symptoms, mental contamination, thought-action fusion and magical thinking) than with divergent measures (assessing depression and anxiety). Moreover, the MFQ successfully discriminated between the SR-OCD sample and the N-OCD group, anxiety disorder sample, and depression sample. These findings suggest that the MFQ has sound psychometric properties and that it can be used to assess morphing fear. Clinical implications are discussed.

Integrating methods for developing sustainability indicators that can facilitate learning and action

Bossel's (2001) systems-based approach for deriving comprehensive indicator sets provides one of the most holistic frameworks for developing sustainability indicators. It ensures that indicators cover all important aspects of system viability, performance, and sustainability, and recognizes that a system cannot be assessed in isolation from the systems upon which it depends and which in turn depend upon it. In this reply, we show how Bossel's approach is part of a wider convergence toward integrating participatory and reductionist approaches to measure progress toward sustainable development. However, we also show that further integration of these approaches may be able to improve the accuracy and reliability of indicators to better stimulate community learning and action. Only through active community involvement can indicators facilitate progress toward sustainable development goals. To engage communities effectively in the application of indicators, these communities must be actively involved in developing, and even in proposing, indicators. The accuracy, reliability, and sensitivity of the indicators derived from local communities can be ensured through an iterative process of empirical and community evaluation. Communities are unlikely to invest in measuring sustainability indicators unless monitoring provides immediate and clear benefits. However, in the context of goals, targets, and/or baselines, sustainability indicators can more effectively contribute to a process of development that matches local priorities and engages the interests of local people.

Reliability in the whole life cycle of building systems

Purpose – The purpose of this research is to show that reliability analysis and its implementation will lead to an improved whole life performance of the building systems, and hence their life cycle costs (LCC). Design/methodology/approach – This paper analyses reliability impacts on the whole life cycle of building systems, and reviews the up-to-date approaches adopted in UK construction, based on questionnaires designed to investigate the use of reliability within the industry. Findings – Approaches to reliability design and maintainability design have been introduced from the operating environment level, system structural level and component level, and a scheduled maintenance logic tree is modified based on the model developed by Pride. Different stages of the whole life cycle of building services systems, reliability-associated factors should be considered to ensure the system's whole life performance. It is suggested that data analysis should be applied in reliability design, maintainability design, and maintenance policy development. Originality/value – The paper presents important factors in different stages of the whole life cycle of the systems, and reliability and maintainability design approaches which can be helpful for building services system designers. The survey from the questionnaires provides the designers with understanding of key impacting factors.

Critical reliability issues for building services systems

This paper addresses two critical issues associated with reliability and maintenance of building services systems. The first is the ratio of operating and/or maintenance costs to initial costs for building services systems. It is an important parameter for life cycle costing and maintenance policy development. The second is the proportion of items among building services systems that need preventive maintenance. In this paper, we estimate the ratios based on a cost dataset. It suggests that correctly estimating the ratio be important but using a constant ratio in life cycle costing may result in wrong decisions. It also estimates the proportion of preventive maintenance for building services systems on the basis of the distribution of failure patterns.