Can less be more? Comparison of an 8-item placement quality measure with the 50-item Dundee Ready Educational Environment Measure (DREEM)

Clinical clerks learn more than they are taught and not all they learn can be measured. As a result, curriculum leaders evaluate clinical educational environments. The quantitative Dundee Ready Environment Measure (DREEM) is a de facto standard for that purpose. Its 50 items and 5 subscales were developed by consensus. Reasoning that an instrument would perform best if it were underpinned by a clearly conceptualized link between environment and learning as well as psychometric evidence, we developed the mixed methods Manchester Clinical Placement Index (MCPI), eliminated redundant items, and published validity evidence for its 8 item and 2 subscale structure. Here, we set out to compare MCPI with DREEM. 104 students on full-time clinical placements completed both measures three times during a single academic year. There was good agreement and at least as good discrimination between placements with the smaller MCPI. Total MCPI scores and the mean score of its 5-item learning environment subscale allowed ten raters to distinguish between the quality of educational environments. Twenty raters were needed for the 3-item MCPI training subscale and the DREEM scale and its subscales. MCPI compares favourably with DREEM in that one-sixth the number of items perform at least as well psychometrically, it provides formative free text data, and it is founded on the widely shared assumption that communities of practice make good learning environments.

Development of the Consumer Refrigerator Safety Questionnaire (CRSQ): A Measure of Consumer Perceptions and Practices

Food preparation and storage behaviors in the home deviating from the ‘best practice’ food safety recommendations may result in food borne illnesses. Currently, there are limited tools available to fully evaluate the consumer knowledge, perceptions and behavior in the area of refrigerator safety. The current study aimed to develop a valid and reliable tool in the form of a questionnaire (CFSQCRSQ) for assessing systematically all these aspects. Items relating to refrigerator safety knowledge (n=17), perceptions (n=46), reported behavior (n=30) were developed and pilot tested by an expert reference group and various consumer groups to assess face and content validity (n=20), item difficulty and item consistency (n=55) and construct validity (n=23). The findings showed that the CFSQCRSQ has acceptable face and content validity with acceptable levels of item difficulty. Item consistency was observed for 12 out of 15 refrigerator safety knowledge. Further, all five of the subscales of consumer perceptions of refrigerator safety practices relating to risk of developing foodborne disease food poisoning showed acceptable internal consistency (Cronbach’s α value > 0.8). Construct validity of the CFSQCRSQ was shown to be very good (p=0.022). The CFSQCRSQ exhibited acceptable test-retest reliability at 14 days with majority of knowledge items (93.3%) and reported behavior items (96.4%) having correlation coefficients of greater than 0.70. Overall, the CFSQCRSQ was deemed valid and reliable in assessing refrigerator safety knowledge and behavior and therefore has the potential for future use in identifying groups of individuals at increased risk of deviating from recommended refrigerator safety practices as well as the assessment of refrigerator safety knowledge, behavior for use before and after an intervention.

Systematic review of tools to measure outcomes for young children with autism spectrum disorder

BACKGROUND: The needs of children with autism spectrum disorder (ASD) are complex and this is reflected in the number and diversity of outcomes assessed and measurement tools used to collect evidence about children's progress. Relevant outcomes include improvement in core ASD impairments, such as communication, social awareness, sensory sensitivities and repetitiveness; skills such as social functioning and play; participation outcomes such as social inclusion; and parent and family impact.

OBJECTIVES: To examine the measurement properties of tools used to measure progress and outcomes in children with ASD up to the age of 6 years. To identify outcome areas regarded as important by people with ASD and parents.

METHODS: The MeASURe (Measurement in Autism Spectrum disorder Under Review) research collaboration included ASD experts and review methodologists. We undertook systematic review of tools used in ASD early intervention and observational studies from 1992 to 2013; systematic review, using the COSMIN checklist (Consensus-based Standards for the selection of health Measurement Instruments) of papers addressing the measurement properties of identified tools in children with ASD; and synthesis of evidence and gaps. The review design and process was informed throughout by consultation with stakeholders including parents, young people with ASD, clinicians and researchers.

RESULTS: The conceptual framework developed for the review was drawn from the International Classification of Functioning, Disability and Health, including the domains 'Impairments', 'Activity Level Indicators', 'Participation', and 'Family Measures'. In review 1, 10,154 papers were sifted - 3091 by full text - and data extracted from 184; in total, 131 tools were identified, excluding observational coding, study-specific measures and those not in English. In review 2, 2665 papers were sifted and data concerning measurement properties of 57 (43%) tools were extracted from 128 papers. Evidence for the measurement properties of the reviewed tools was combined with information about their accessibility and presentation. Twelve tools were identified as having the strongest supporting evidence, the majority measuring autism characteristics and problem behaviour. The patchy evidence and limited scope of outcomes measured mean these tools do not constitute a 'recommended battery' for use. In particular, there is little evidence that the identified tools would be good at detecting change in intervention studies. The obvious gaps in available outcome measurement include well-being and participation outcomes for children, and family quality-of-life outcomes, domains particularly valued by our informants (young people with ASD and parents).

CONCLUSIONS: This is the first systematic review of the quality and appropriateness of tools designed to monitor progress and outcomes of young children with ASD. Although it was not possible to recommend fully robust tools at this stage, the review consolidates what is known about the field and will act as a benchmark for future developments. With input from parents and other stakeholders, recommendations are made about priority targets for research.

FUTURE WORK: Priorities include development of a tool to measure child quality of life in ASD, and validation of a potential primary outcome tool for trials of early social communication intervention.

STUDY REGISTRATION: This study is registered as PROSPERO CRD42012002223.

FUNDING: The National Institute for Health Research Health Technology Assessment programme.

Systematic Review of the Measurement Properties of Tools Used to Measure Behaviour Problems in Young Children with Autism

Background
Behaviour problems are common in young children with autism spectrum disorder (ASD). There are many different tools used to measure behavior problems but little is known about their validity for the population.

Objectives
To evaluate the measurement properties of behaviour problems tools used in evaluation of intervention or observational research studies with children with ASD up to the age of six years.

Methods
Behaviour measurement tools were identified as part of a larger, two stage, systematic review. First, sixteen major electronic databases, as well as grey literature and research registers were searched, and tools used listed and categorized. Second, using methodological filters, we searched for articles examining the measurement properties of the tools in use with young children with ASD in ERIC, MEDLINE, EMBASE, CINAHL, and PsycINFO. The quality of these papers was then evaluated using the COSMIN checklist.

Results
We identified twelve tools which had been used to measure behaviour problems in young children with ASD, and fifteen studies which investigated the measurement properties of six of these tools. There was no evidence available for the remaining six tools. Two questionnaires were found to be the most robust in their measurement properties, the Child Behavior Checklist and the Home Situations Questionnaire—Pervasive Developmental Disorders version.

Conclusions
We found patchy evidence on reliability and validity, for only a few of the tools used to measure behaviour problems in young children with ASD. More systematic research is required on measurement properties of tools for use in this population, in particular to establish responsiveness to change which is essential in measurement of outcomes of intervention.

PROSPERO Registration Number

CRD42012002223

An Empirical Evaluation of the Reversal Theory State Measure: Tracking Changes in the Experience of Video Commercials of Three Leading Running Brands

Abstract of paper delivered at the 17th International Reversal Theory Conference, Day 3, session 4, 01.07.15

An Empirical Evaluation of the Reversal Theory State Measure Using Three Running Brand Video Commercials

This paper reports on the first known empirical use of the Reversal Theory State Measure (RTSM) since its publication by Desselles et al. (2014). The RTSM was employed to track responses to three purposely-selected video commercials in a between-subjects design. Results of the study provide empirical support for the central conceptual premise of reversal theory, the experience of metamotivational reversals and the ability of the RTSM to capture them. The RTSM was also found to be psychometrically sound after adjustments were made to two of its three component subscales. Detailed account and rationale is provided for the analytical process of assessing the psychometric robustness of the RTSM, with a number of techniques and interpretations relating to component structure and reliability discussed. Agreeability and critique of the two available versions of the RTSM – the bundled and the branched – is also examined. Researchers are encouraged to assist development of the RTSM through further use, taking into account the analysis and recommendations presented.

A microelectrode impedance method to measure interaction of cells

An impedance method was developed to determine how immune system cells (hemocyte) interact with intruder cells (parasites). When the hemocyte cells interact with the parasites, they cause a defensive reaction and the parasites start to aggregate in clusters. The level of aggregation is a measure of the host-parasite interaction, and provides information about the efficiency of the immune system response. The cell aggregation is monitored using a set of microelectrodes. The impedance spectrum is measured between each individual microelectrode and a large reference electrode. As the cells starts to aggregate and settle down towards the microelectrode array the impedance of the system is changed. It is shown that the system impedance is very sensitive to the level of cell aggregation and can be used to monitor in real time the interaction between hemocyte cells and parasites.

Differing methodologies to measure microbe dispersal during hand drying

Background: Having previously investigated the dispersal by different hand drying methods of a chemical indicator, fungi and bacteria on the hands of users, this new study assessed the potential for viral dispersal. Aims/Objectives: To determine differences between hand drying methods in their capacity to disperse viruses on the hands of users to other occupants of public washrooms and into the washroom environment. Method: A harmless virus was used to artificially contaminate the hands of participants prior to using three different hand drying devices (jet air dryer, warm air dryer, paper towel dispenser). Viral dispersal was assessed at different heights and distances from the hand drying devices and also at different times after use by means of an air sampler. Results: The jet air dryer was shown to produce significantly more dispersal of virus than the warm air dryer or paper towels. After use of the jet air dryer, high numbers of virus were detected at a range of heights with maximum numbers between 0.61 and 1.22 metres. Virus was also detected at distances of up to 3 metres from the jet air dryer and in the air for up to 15 minutes after its use. The warm air dryer and paper towel dispenser produced low or zero viral counts at different heights, different distances and times after use. Conclusion: Jet air dryers have a greater potential than other hand drying methods to disperse viruses on the hands and contaminate other occupants of a public washroom and the washroom environment.

How to Measure the Deterrence Effects of Merger Policy: Frequency or Composition?

We show that the number of merger proposals (frequency-based deterrence) is a more appropriate indicator of underlying changes in merger policy than the relative anti-competitiveness of merger proposals (composition-based deterrence). This has strong implications for the empirical analysis of the deterrence effects of merger policy enforcement, and potential implications regarding how to reduce anti-competitive merger proposals.

Use of Distress and Depression Thermometers to Measure Psychosocial Morbidity Among Southern European Cancer Patients

GOALS OF WORK: Recent literature has indicated the need for rapid evaluation of psychosocial issues secondary to cancer. Because of the problems of routine use of psychometric instruments, short instruments such as visual analogue scales or one-item 0-10 scales have been developed as valid assessment alternatives. PATIENTS AND METHODS: A study was conducted to examine the role of two 0-10 scales in measuring emotional stress (distress thermometer, DT) and depressed mood (mood thermometer, MT), respectively, in a multicenter study carried out in southern European countries (Italy, Portugal, Spain, and Switzerland). A convenience sample of 312 cancer outpatients completed the DT and MT and the Hospital Anxiety Depression Scale (HADS). MAIN RESULTS: DT was more significantly associated HADS anxiety than HADS depression while MT was related both to HADS anxiety and depression. The correlation of MT with HADS was higher than DT. A cutoff point >4 on the DT maximized sensitivity (65%) and specificity (79%) for general psychosocial morbidity while a cutoff >5 identified more severe "caseness" (sensitivity=70%; specificity=73%). On the MT, sensitivity and specificity for general psychosocial morbidity were 85% and 72% by using the cutoff score >3. A score >4 on the MT was associated with a sensitivity of 78% and a specificity of 77% in detecting more severe caseness. CONCLUSIONS: Two simple instruments, the DT and the MT, were found to have acceptable levels of sensitivity and specificity in detecting psychosocial morbidity. Compared to the HADS, however, the mood MT performed better than the DT.

Impact of Syncope on Quality of Life: Validation of a Measure in Patients Undergoing Tilt Testing

INTRODUCTION AND OBJECTIVES: Recurrent syncope has a significant impact on quality of life. The development of measurement scales to assess this impact that are easy to use in clinical settings is crucial. The objective of the present study is a preliminary validation of the Impact of Syncope on Quality of Life questionnaire for the Portuguese population. METHODS: The instrument underwent a process of translation, validation, analysis of cultural appropriateness and cognitive debriefing. A population of 39 patients with a history of recurrent syncope (>1 year) who underwent tilt testing, aged 52.1 ± 16.4 years (21-83), 43.5% male, most in active employment (n=18) or retired (n=13), constituted a convenience sample. The resulting Portuguese version is similar to the original, with 12 items in a single aggregate score, and underwent statistical validation, with assessment of reliability, validity and stability over time. RESULTS: With regard to reliability, the internal consistency of the scale is 0.9. Assessment of convergent and discriminant validity showed statistically significant results (p<0.01). Regarding stability over time, a test-retest of this instrument at six months after tilt testing with 22 patients of the sample who had not undergone any clinical intervention found no statistically significant changes in quality of life. CONCLUSIONS: The results indicate that this instrument is of value for assessing quality of life in patients with recurrent syncope in Portugal.

Validação e aplicação clínica de uma escala de cognição social na esquizofrenia = Validation and clinical use of a measure of social cognition in schizophrenia

RESUMO: A cognição social encontra-se frequentemente alterada na esquizofrenia. Esta alteração relaciona-se com a diminuição do funcionamento social,caracterizando-se quer por défices quer por vieses cognitivos sociais. No entanto, existem poucos instrumentos fiáveis e válidos para avaliar a cognição social na esquizofrenia, nomeadamente capazes de medir os vieses cognitivos sociais e a cognição social auto-relevante. Adicionalmente, as bases biológicas da disfunção social não estão totalmente esclarecidas. Evidências recentes sugerem que o peptídeo oxitocina (OXT) influencia o funcionamento social, e que esta relação poderá ser mediada pela cognição social. Este Trabalho de Projecto descreve a contribuição do autor para o desenvolvimento e avaliação psicométrica inicial de um novo instrumento de avaliação da cognição social, e a utilidade desta escala na investigação das associações entre a OXT e a capacidade e vieses cognitivos sociais. A Waiting Room Task (WRT), uma escala constituída por 26 vídeos sequenciais que simulam a experiência de observar outra pessoa numa sala de espera, foi administrada num estudo transversal com 61 doentes com esquizofrenia e 20 controlos saudáveis. Observou-se uma menor capacidade cognitiva social e um aumento dos vieses cognitivos sociais nos doentes com esquizofrenia, comparativamente aos controlos. Nos controlos e doentes com delírios, o desempenho na WRT correlacionou-se significativamente com os níveis de OXT. Esta correlação não se observou nos doentes sem delírios, sugerindo que o papel da OXT na cognição social poderá encontrar-se atenuado neste grupo. Estes achados fornecem suporte inicial para a adequação da WRT como instrumento de avaliação da cognição social na esquizofrenia, podendo ainda ser útil na investigação da sua base biológica. ------------ ABSTRACT: Social cognition is often impaired in schizophrenia. This impairment is related to poor social functioning and is characterized by both social cognitive deficits and biases. However, there are few reliable and valid measures of social cognition in schizophrenia, particularly measures of social cognitive bias and of self-relevant social cognition. Also, the biological bases of social dysfunction are not well understood. Emerging evidence suggests that the peptide oxytocin (OXT) influences social functioning, and that this relationship may be mediated by social cognition. This Research Project describes the author’s contribution to the development and initial psychometric testing of a new measure of social cognition, and the utility of this instrument to examine associations between OXT and social cognitive capacity and bias. The Waiting Room Task WRT), a video-based test comprising 26 sequential videos simulating the experience of facing another person in a waiting room, was administered in a cross-sectional study involving 61 patients with schizophrenia and 20 healthy controls. Social cognitive capacity was lower and social cognitive bias was increased in patients with schizophrenia compared with controls. Among controls and patients with delusions, performance on the WRT was significantly correlated with OXT level. This correlation was not found in patients without delusions suggesting that OXT’s role in social cognition may be blunted in this group. These findings provide initial support for the adequacy of the WRT as a measure for assessing social cognition in schizophrenia that may also be useful in understanding its biological underpinnings.

Tradução e validação para Portugal do Patient Activation Measure 13 : em pessoas com Diabetes Mellitus tipo 2

RESUMO - Introdução: A Diabetes Mellitus Tipo 2 (DM2) tem uma elevada prevalência em todo o mundo, com impacto significativo a nível de Saúde Pública, na vida dos doentes e nos custos que lhe são associados. O Patiente Activation Measure 13 (PAM13) é um questionário que possibilita a avaliação das crenças, conhecimentos, motivação e capacidades de uma pessoa em relação à sua saúde; pelo que a sua utilização na DM2 é pertinente. Objetivos: Traduzir o PAM 13, versão curta, para Português de Portugal; Estabelecer as propriedades psicométricas da versão Portuguesa do PAM 13 (PAM13-P); Validar a PAM 13-P num grupo de pessoas com DM2. Material e métodos: O processo de tradução e adaptação cultural do questionário foi composto pelas fases: 1-Tradução, 2-Reconciliação e síntese, 3-Back translation, 4-Rever e sintetizar a Back translation, 5-Harmonização, 6-Revisão do comité de peritos, 7-Cognitive debriefing e 8-Avaliação final. Para validar o PAM13-P realizou-se um estudo observacional transversal analítico com uma amostra de conveniência, de pessoas com DM2, seguidas na Associação Protectora dos Diabéticos de Portugal. O questionário foi de autopreenchimento e foi consultado o processo clínico para obtenção da HbA1c. O tratamento estatístico foi realizado através do SPSS 21® e Winsteps v3.8.1®. Resultados e discussão: O processo de tradução e adaptação cultural foi realizado de acordo com as guidelines. Foram realizados 3 painéis de e-Delphi, com 21 participantes de áreas distintas, tendo-se obtido bons níveis de concordância. As principais modificações realizadas ao questionário foram a simplificação da linguagem e dos itens, obtendo-se as equivalências necessárias. O PAM13-P foi aplicado a 201 pessoas, sendo que a taxa de resposta foi de 83%. Na amostra analisada 57.3% eram homens. Obtiveram-se as médias de idade 67.1 anos e de duração de diabetes 17.3 anos. A média do score do PAM foi 58.5±10.1(41.8-90.5) pontos e 49,7% da amostra estava no nível 3 de ativação. Relativamente aos itens verificou-se que os itens mais difíceis foram o 13(56.1) e o 8(55.4) e o com menor dificuldade foi 4(38.5). As categorias de resposta tiveram um bom ajuste ao Modelo de Rasch. O ajustamento dos itens foi infit entre 0.779-1.177 e outfit entre 0.794-1.315. A fiabilidade dos indivíduos variou entre 0.77(real) e 0.83(modelo) e dos itens foi de 0.97 (real e modelo). O Alfa de Cronbach foi bom (α=0.82). Estas estatísticas foram semelhantes aos da validação do PAM13. Existiu uma relação entre o score do PAM e os itens de validação em 51%. Das variáveis analisadas, existia um relação do nível de ativação com a idade e com a HbA1c. Conclusões: O PAM13-P foi traduzido e adaptado culturalmente para Português e foi validado em pessoas com DM2, sendo as propriedades psicométricas boas.