Can less be more? Comparison of an 8-item placement quality measure with the 50-item Dundee Ready Educational Environment Measure (DREEM)

Clinical clerks learn more than they are taught and not all they learn can be measured. As a result, curriculum leaders evaluate clinical educational environments. The quantitative Dundee Ready Environment Measure (DREEM) is a de facto standard for that purpose. Its 50 items and 5 subscales were developed by consensus. Reasoning that an instrument would perform best if it were underpinned by a clearly conceptualized link between environment and learning as well as psychometric evidence, we developed the mixed methods Manchester Clinical Placement Index (MCPI), eliminated redundant items, and published validity evidence for its 8 item and 2 subscale structure. Here, we set out to compare MCPI with DREEM. 104 students on full-time clinical placements completed both measures three times during a single academic year. There was good agreement and at least as good discrimination between placements with the smaller MCPI. Total MCPI scores and the mean score of its 5-item learning environment subscale allowed ten raters to distinguish between the quality of educational environments. Twenty raters were needed for the 3-item MCPI training subscale and the DREEM scale and its subscales. MCPI compares favourably with DREEM in that one-sixth the number of items perform at least as well psychometrically, it provides formative free text data, and it is founded on the widely shared assumption that communities of practice make good learning environments.

Aceitação e rejeição entre pares e habilidades sociais em universitários

Nesse estudo, investigaram-se as associações entre habilidades sociais e aceitação-rejeição em estudantes universitários. Foram estudados 126 estudantes universitários do curso de Educação Física de uma instituição de ensino superior particular de uma cidade do interior do estado de São Paulo. Para fins de avaliação foi utilizado um Inventário de Habilidades Sociais (IHS) e uma Medida Sociométrica que avalia a aceitação ou rejeição do estudante por seus colegas de classe para as atividades de sair e estudar. A aceitação para ambas as situações não foi explicada por nenhuma faceta das habilidades sociais para os homens. Já nas mulheres, foi explicada somente pela medida de autoexposição a desconhecidos. Essa pesquisa elucida algumas das condutas socialmente hábeis de homens e mulheres universitários que podem ser empregadas em treinamentos em habilidades sociais para se evitar a rejeição entre pares, e maximizar a aceitação.

Elaboração e validação de um instrumento de avaliação da disfagia orofaríngea na paralisia cerebral

Pós-graduação em Fonoaudiologia - FFC

Validade de conteúdo e processos de respostas de um instrumento de rastreio para disfagia orofaríngea no Acidente Vascular Encefálico

Pós-graduação em Fonoaudiologia - FFC

Evaluating Measurement Invariance with Censored Ordinal Data: A Monte Carlo Comparison of Alternative Model Estimators and Scales of Measurement

Evaluations of measurement invariance provide essential construct validity evidence. However, the quality of such evidence is partly dependent upon the validity of the resulting statistical conclusions. The presence of Type I or Type II errors can render measurement invariance conclusions meaningless. The purpose of this study was to determine the effects of categorization and censoring on the behavior of the chi-square/likelihood ratio test statistic and two alternative fit indices (CFI and RMSEA) under the context of evaluating measurement invariance. Monte Carlo simulation was used to examine Type I error and power rates for the (a) overall test statistic/fit indices, and (b) change in test statistic/fit indices. Data were generated according to a multiple-group single-factor CFA model across 40 conditions that varied by sample size, strength of item factor loadings, and categorization thresholds. Seven different combinations of model estimators (ML, Yuan-Bentler scaled ML, and WLSMV) and specified measurement scales (continuous, censored, and categorical) were used to analyze each of the simulation conditions. As hypothesized, non-normality increased Type I error rates for the continuous scale of measurement and did not affect error rates for the categorical scale of measurement. Maximum likelihood estimation combined with a categorical scale of measurement resulted in more correct statistical conclusions than the other analysis combinations. For the continuous and censored scales of measurement, the Yuan-Bentler scaled ML resulted in more correct conclusions than normal-theory ML. The censored measurement scale did not offer any advantages over the continuous measurement scale. Comparing across fit statistics and indices, the chi-square-based test statistics were preferred over the alternative fit indices, and ΔRMSEA was preferred over ΔCFI. Results from this study should be used to inform the modeling decisions of applied researchers. However, no single analysis combination can be recommended for all situations. Therefore, it is essential that researchers consider the context and purpose of their analyses.

Entwicklung eines deutschsprachigen Fragebogens zur Durchführung von Multisource-Feedback in der ärztlichen Weiterbildung

Fragestellung/Einleitung: Multisource-Feedback (MSF) ist ein anerkanntes Instrument zur Überprüfung und Verbesserung der ärztlichen Tätigkeit [1]. Es beinhaltet Feedback, das von MitarbeiterInnen verschiedener Tätigkeitsbereiche und verschiedener Hierarchiestufen gegeben wird. Das Feedback wird anonym mithilfe eines Fragebogens gegeben, der verschiedene Kriterien der ärztlichen Kompetenz beschreibt. Das Feedback wird anschlieβend für die zu beurteilenden ÄrztInnen in einem Gespräch von einer/m SupervisorIn zusammengefasst. Bislang existiert kein deutschsprachiger Fragebogen für Multisource-Feedback für die ärztliche Tätigkeit. Unsere Zielsetzung war es daher, einen deutschsprachigen Fragebogen zu erstellen und diesen bzgl. relevanter Validitätskriterien zu untersuchen. Methoden: Zur Erstellung des Fragebogens sammelten wir die beste verfügbare Evidenz der entsprechenden Literatur. Wir wählten einen validierten englischen Fragebogen, der bereits in der Weiterbildung in Groβbritannien angewendet wird [2] und den wichtigsten Kriterien entspricht. Dieser wurde übersetzt und in einigen Bereichen erweitert, um ihn sprachlichen Gegebenheiten und lokalen Bedürfnissen anzupassen. Bezüglich der Validität wurden zwei Kriterien untersucht: Inhaltsvalidität (content validity evidence) und Antwortprozesse (response process validity evidence). Um die Inhaltsvalidität zu untersuchen, wurde in einer Expertenrunde diskutiert, ob der übersetzte Fragebogen die erwarteten Kompetenzen widerspiegelt. Im Anschluss wurden die Antwortprozesse mithilfe eines sog. „think-alouds“ mit ÄrztInnen in Weiterbildung und ihren AusbilderInnen untersucht. Ergebnisse: Der resultierende Fragebogen umfasst 20 Fragen. Davon sind 15 Items den Bereichen „Klinische Fähigkeiten“, „Umgang mit Patienten“, „Umgang mit Kollegen“ und „Arbeitsweise“ zuzuordnen. Diese Fragen werden auf einer fünfstufigen Likert-Skala beantwortet. Zusätzlich bietet jede Frage die Möglichkeit, einen Freitext zu besonderen Stärken und Schwächen der KandidatInnen aufzuführen. Weiterhin gibt es fünf globale Fragen zu Stärken und Verbesserungsmöglichkeiten, äuβeren Einflüssen, den Arbeitsbedingungen und nach Zweifeln an der Gesundheit oder Integrität des Arztes/ der Ärztin. In der Expertenrunde wurde der Fragebogen als für den deutschsprachigen Raum ohne Einschränkungen anwendbar eingeschätzt. Die Analyse der Antwortprozesse führte zu kleineren sprachlichen Anpassungen und bestätigt, dass der Fragebogen verständlich und eindeutig zu beantworten ist und das gewählte Konstrukt der ärztlichen Tätigkeit vollständig umschreibt. Diskussion/Schlussfolgerung: Wir entwickelten einen deutschsprachigen Fragebogen zur Durchführung von Multisource-Feedback in der ärztlichen Weiterbildung. Wir fanden Hinweise für die Validität dieses Fragebogens bzgl. des Inhalts und der Antwortprozesse. Zusätzliche Untersuchungen zur Validität wie z.B. die durch den Fragebogen entstehenden Auswirkungen (consequences) sind vorgesehen. Dieser Fragebogen könnte zum breiteren Einsatz von MSF in der ärztlichen Weiterbildung auch im deutschsprachigen Raum beitragen. This is an Open Access article distributed under the terms of the Creative Commons Attribution License. You are free: to Share - to copy, distribute and transmit the work, provided the original author and source are credited. See license information at http://creativecommons.org/licenses/by-nc-nd/3.0/.

Improving the Assessment of Communication Skills at the Swiss Federal Licensing Exam: Results of a Student's Needs Assessment

Introduction With a three year project the assessment of communication skills within the Swiss Federal Licensing Examinations (FLE) shall be improved. As a first step a needs assessment among communication experts and medical students of the Swiss Medical Faculties will be performed. In this presentation the results of the students’ needs assessment will be presented. Methods A bilingual student’s online questionnaire will be developed by an expert panel taking relevant literature, the Swiss Catalogue of Learning Objectives and other consensus statements for communication (e.g., the European and Basler consensus statements) into account. With a think aloud study response process validity evidence will be sought. The questionnaire will focus on the following topics related to communication skills: (1) What has been taught?, (2) What has been assessed in the faculty exams?, (3) What has been assessed in the FLE?, (4) What should have been assessed in the FLE and how should the assessment be improved? Results Results of the students’ needs assessment will be available by the end of 2015 and be presented. Conclusions/ Take-home message We hope for valuable input for improving the assessment of communications skills within the FLE also from the students’ side. Results of the needs assessment from the students and experts will be combined and taken as input for an international expert symposium on how to improve the communication skills assessment within the FLE.

O Teste de Apercepção Temática (TAT) em adultos: dados normativos para o sistema morvaliano

No contexto da pesquisa e utilização prática das técnicas de autoexpressão, o Teste de Apercepção Temática (TAT) apresenta reconhecidas vantagens. Apesar de linhas de pesquisa bastante desenvolvidas internacionalmente, não existem estudos atuais sobre as propriedades deste instrumento no Brasil, levantando a necessidade de conhecer seus alcances e limites nesse contexto cultural. Desse modo, o desenvolvimento de normas que descrevam o desempenho típico de uma população não clínica se mostra um importante passo inicial rumo à retomada da pesquisa com o TAT e sua adoção por profissionais. Tendo em vista que a validação das técnicas de autoexpressão se refere aos sistemas de classificação e não aos instrumentos em si, o presente estudo adotou, dentre a miríade de sistemas de categorização das histórias do TAT descritos na literatura, o sistema proposto por Monique Morval, que foi desenvolvido para estudantes e profissionais com pouca experiência com o TAT, utilizando pressupostos da Personologia de Henry Murray, Psicologia Cognitiva e Psicologia do Ego, possibilitando leituras objetivas e qualitativas do material. Considerando o potencial de utilização desse sistema por parte dos psicólogos brasileiros - visto sua semelhança com o atual sistema aprovado para uso pelo Sistema de Avaliação dos Testes Psicológicos (SATEPSI) do Conselho Federal de Psicologia - foram feitos aprimoramentos no assim chamado sistema morvaliano, possibilitando sua adoção no estudo normativo proposto pelo presente projeto. Tais aprimoramentos consistiram na revisão e adição de categorias e subcategorias, a partir de trabalhos brasileiros e internacionais sobre o instrumento. Desta forma, o presente estudo teve por objetivo desenvolver normas para o TAT (Sistema Morvaliano) em adultos, a partir de uma amostra não clínica, aleatória e estratificada por critérios de idade, sexo, escolaridade e nível socioeconômico (NSE) da cidade de Ribeirão Preto (SP). Os participantes foram 100 adultos (25 a 44 anos de idade), selecionados através de visita a domicílios sorteados em bairros de diferentes NSE. Foram realizadas entrevistas de avaliação psicológica individuais, utilizando um roteiro de entrevista semiestruturada, o Teste de Inteligência Geral Não-Verbal (TIG-NV) e 20 cartões do TAT, pré-selecionados de acordo com as recomendações do manual original do instrumento. As histórias do TAT foram categorizadas no sistema morvaliano revisado e seus dados passaram por análises estatísticas e sínteses qualitativas em busca de diferenças de desempenho relacionadas ao sexo, escolaridade e NSE. Os resultados mostram poucas diferenças entre os grupos que justifiquem a elaboração de normas específicas; apesar disso, foi identificada uma tendência (ainda que pequena) de o nível de escolaridade e o NSE favorecerem um melhor desempenho nas variáveis relativas à organização formal das histórias. São discutidas as implicações dos dados obtidos em termos do perfil normativo de desempenho nas variáveis do sistema morvaliano, suas evidências preliminares de validade e futuras direções para estudos sobre o TAT nesse sistema.

Psychometric properties of the School Anxiety Inventory-Short Version in Spanish secondary education students

Background: The School Anxiety Inventory (SAI) can be applied in different fields of psychology. However, due to the inventory’s administration time, it may not be useful in certain situations. To address this concern, the present study developed a short version of the SAI (the SAI-SV). Method: This study examined the reliability and validity evidence drawn from the scores of the School Anxiety Inventory-Short Version (SAI-SV) using a sample of 2,367 (47.91% boys) Spanish secondary school students, ranging from 12 to 18 years of age. To analyze the dimensional structure of the SAI-SV, exploratory and confirmatory factor analyses were applied. Internal consistency and test-retest reliability were calculated for SAISV scores. Results: A correlated three-factor structure related to school situations (Anxiety about Aggression, Anxiety about Social Evaluation, and Anxiety about Academic Failure) and a three-factor structure related to the response systems of anxiety (Physiological Anxiety, Cognitive Anxiety, and Behavioral Anxiety) were identified and supported. The internal consistency and test-retest reliability were determined to be appropriate. Conclusions: The reliability and validity evidence based on the internal structure of SAI-SV scores was satisfactory.

Psychometric properties of the Spanish adaptation of the School Attitude Assessment Survey-Revised

Background: The assessment of attitudes toward school with the objective of identifying adolescents who may be at risk of underachievement has become an important area of research in educational psychology, although few specific tools for their evaluation have been designed to date. One of the instruments available is the School Attitude Assessment Survey-Revised (SAAS-R). Method: The objective of the current research is to test the construct validity and to analyze the psychometric properties of the Spanish version of the SAAS-R. Data were collected from 1,398 students attending different high schools. Students completed the SAAS-R along with measures of the g factor, and academic achievement was obtained from school records. Results: Confirmatory factor analysis, multivariate analysis of variance and analysis of variance tests supported the validity evidence. Conclusions: The results indicate that the Spanish version of the SAAS-R is a useful measure that contributes to identification of underachieving students. Lastly, the results obtained and their implications for education are discussed.

Leveraging employee creativity through high performance work systems:a multilevel perspective

Research has looked at single rather than a configuration of human resource management (HRM) practices to influence creativity so it is not yet clear how these practices synergistically facilitate creativity and organisational performance. I address this significant but unanswered question in a three-part study. In Study 1, I develop a high performance work system (HPWS) for creativity scale. I use Study 2 sample to test the validity of the new scale. In Study 3, I test a multilevel model of the intervening processes through which branch HPWS for creativity influences creativity and branch performance. Specifically, at the branch level, I draw on social context theory and hypothesise that branch HPWS for creativity relates to climate for creativity which, in turn, leads to creativity, and ultimately, to profit. Furthermore, I hypothesise environmental dynamism as a boundary condition of the creativity-profit relationship. At the individual level, I hypothesise a cross-level effect of branch HPWS for creativity on employee-perceived HPWS. I draw on self-determination theory and argue that perceived HPWS for creativity relate to need satisfaction and the psychological pathways of intrinsic motivation and creative process engagement to predict creativity. I also hypothesise climate for creativity as a cross-level moderator of the intrinsic motivation-creativity and creative process engagement-creativity relationships. Results of hierarchical linear modeling (HLM) indicate that ten out of the fifteen hypotheses were supported. The findings of this study respond to calls for HPWS to be designed around a strategic focus by developing and providing initial validity evidence of an HPWS for creativity scale. The results reveal the underlying mechanisms through which HPWS for creativity simultaneously influences individual and branch creativity leading to profit. Lastly, results indicate environmental dynamism to be an important boundary condition of the creativity-profit relationship and climate for creativity as a cross-level moderator of the creative process engagement-creativity.

Prevalência e fatores associados à alteração vocal em idosos institucionalizados com capacidade cognitiva preservada

Voice disorders (VD) in the elderly can interfere negatively in communication, emotional well-being and quality of life, conditions that correspond to greater exposure to illness and social isolation bringing consequent economic impact for the health system. It is assumed that institutionalized confinement, weakness and morbidity associated to nursing home (NH) contribute to transform VD an especially prevalent condition in institutionalized elderly, including those without cognitive impairment. Thus, the aim of this study was to determine the prevalence and associated factors of VD in NH elderly residents without cognitive impairment. There is no epidemiological diagnostic instruments of VD for elderly populations, so the first step of this study was dedicated to prepare and analyze the psychometric properties of a short, inexpensive and easy to use questionnaire named Screening for Voice Disorders in Older Adults (Rastreamento de Alterações Vocais em Idosos—RAVI). The methodological procedures of this step followed the guidelines of the Standards for Educational and Psychological Testing and contemplated validity evidence based on test content, based on response processes, based on internal structure and based on relations with other variables, as well as reliability analysis and clinical consistency. The result of the validation process showed that the RAVI final score generate valid and reliable interpretations for the epidemiological diagnosis of VD in the elderly, which endorsed the use of the questionnaire in the second stage of the study, performed in ten NH located in the city of Natal, Rio Grande do Norte. At this stage, data from socioeconomic and demographic variables, lifestyle, general health conditions and characterization of the institution were collected. It was performed a bivariate analysis and it was calculated the prevalence ratio as a magnitude association measure, with a confidence interval of 95%. The variables with p-value less than 0.20 were included in the multiple logistic regression model that followed the Forward selection method. The odds ratio found in the multivariate model was converted into prevalence ratio and the level of significance was 5%. The sample consisted of 117 subjects with predominance of females and average of 79.68 (± 7.92) years old. The prevalence of VD was 39.3% (95% CI: 30.4-48.1%). The multivariate model showed statistically significant association between VD and depressive symptoms, smoking for a year or more and selfreported hearing loss. In conclusion, VD is a prevalent health condition in NH elderly residents without cognitive impairment and is associated with factors involving psychosocial, lifestyle and communicative disability that require attention of managers and professionals involved with NH environment. Strategies to encourage communication and social integration, actions to combat smoking and minimizing the effects of hearing loss could stimulate the physical well-being, emotional and mental health of institutionalized elderly population, contributing to the vocal and communicative maintenance, a more effective social inclusion and better overall health condition.

Indicadores e fatores preditivos de uso abusivo de álcool em adultos jovens

The abusive use of alcohol is closely related to dependence and to social and work damages. The main focus of this thesis is to create an instrument about alcohol abuse, in order to differentiate the degree of commitment of the symptomatology, considering its psychosocial factors of prediction. As specific goals: I) characterize the state of the art about assessment related to the abuse and dependence to alcohol; II) investigate and systematize aspects related to the predictive psychosocial factors for alcohol dependence; III) build an instrument for the assessment of alcohol abuse and protection and risk factors for the development of an alcohol dependence; and IV) verify validity evidence of the instrument built for the Brazilian population. In Study I, it was possible to observe the prevalence of articles related to the use of alcohol in a problematic way, without a classification dependence, it is lower than the one of articles that investigate the disease when it is already manifested, not to mention a few systematic studies about the theme of alcohol abuse in the scientific environment. In Study II, focus groups (FGs) were conducted, the analysis about the discourses of the focus groups were made through the ALCESTE software and it was possible to observe a response pattern that existed among the participants in different groups, with the generation of five classes. In Study III, we developed an instrument that contemplated aspects of the Alcohol Dependence Syndrome of the Millon Clinical Multiaxial Inventory-III, in addition to the characteristics defined in Study I and in Study II. The final version of the instrument had 59 items assessed through the likert scale of five points. In Study IV, the administration of the instrument was performed in an online format with university students ranging from 18 to 24 years old, residents in Brazilian metropolitan cities. The results evidenced that the internal consistency of the instrument is considered satisfactory (α = 0,882) and in what it refers to classes, the most significant data was the one related to financial loss and criteria for the diagnosis of alcohol abuse. It is important to consider the evaluative potential of risk and protective factors for the development of alcohol dependence of the instrument as a whole. Once the indicators of abuse and the profile of the abusers has been modified, the patient may have his/her treatment/intervention focused on the trouble and/or specific syndrome, thus having a clear and fast improvement.

Locus of Control revisited: development of a new bi-dimensional measure

Locus of control (LOC) has a long tradition in Psychology, and various instruments have been designed for its measurement. However, the dimensionality of the construct is unclear, and still gives rise to considerable controversy. The aim of the present work is to present new evidence of validity in relation to the dimensionality of LOC. To this end, we developed a new measurement instrument with 23 items. The sample was made up of 697 Spanish participants, of whom 57.5% were women (M=22.43; SD= 9.19). The results support the bi-dimensionality of LOC: internal (α=.87) and external (α=.85). Furthermore, both subscales have shown adequate validity evidence in relation to self-efficacy, achievement motivation and optimism (r xy> .21). Statistically significant differences were found by sex (p < .05): men scored higher in external LOC and women in internal LOC. The validity evidence supports a two-dimensional structure for the LOC, and the measurement instrument developed showed adequate psychometric properties.

Validation de l’échelle de mesure de la conscience de l’environnement d’affaires

La conscience de l’environnement d’affaires est définie comme l’ensemble des savoirs détenus par les employés non-cadres par rapport à l’environnement d’affaires interne et externe de leur organisation. Elle se manifeste lorsque l’employé est impliqué dans la prise de décision ou résolution de problème affectant l’entreprise. Ce travail a comme objectifs la validation d’une échelle de mesure du concept à l’étude, ainsi que la vérification de sa validité de construit et nomologique. L’ensemble initial d’items de l’échelle de mesure a été déterminé suite à des entrevues avec les employés d’organisations participantes (Gauvreau-Jean, 2008). L’ensemble initial de 40 items a été testé avec un échantillon de 508 employés d’une manufacture. Suite à des ajustements, l’échelle modifiée couvre quatre dimensions du construit. L’adéquation de son contenu a ensuite été mise à l’épreuve par deux groupes cibles et deux évaluations expertes (Lessard, 2014). Finalement, dans le cadre de notre étude, la validité du contenu et nomologique de l’échelle modifiée (contenant 24 items) a été testée avec 330 employés de première ligne de quatre entreprises. Les quatre dimensions du construit furent supportées, la fiabilité de l’échelle était haute, puis les hypothèses reliant la conscience de l’environnement avec les caractéristiques au travail, l’évaluation de la performance par un superviseur et l’engagement affectif furent supportées également. Un autre sondage complété par les gestionnaires (n=43) avait comme but une évaluation de la performance contextuelle des employés supervisés. Le développement de l’échelle à 24 items sur la conscience de l’environnement d’affaires pourrait répondre aux questions portant sur la participation de l’employé et l’efficacité organisationnelle.