Development of an instrument measuring the quality of residential care facilities for older people

Aims. To validate the Swedish version of the Sheffield Care Environment Assessment Matrix (S-SCEAM). The instrument’s items measure environmental elements important for supporting the needs of older people, and conceptualized within eight domains. Methods. Item relevance was assessed by a group of experts and measured using content validity index (CVI). Test-retest and inter-rater reliability tests were performed.  The domain structure was assessed by the inter-rater agreement of a second group of experts, and measured using Fleiss kappa. Results. All items attained a CVI above 0.78, the suggested criteria for excellent content validity. Test-retest reliability showed high stability (96% and 95% for two independent raters respectively), and inter-rater reliability demonstrated high levels of agreement (95% and 94% on two separate rating occasions). Kappa values were very good for test-retest (κ = 0.903 and 0.869) and inter-rater reliability (κ = 0.851 and 0.832). Domain structure was good,  Fleiss’ kappa was 0.63 (range 0.45 to 0.75).    Conclusion. The S-SCEAM of 210 items and eight domains showed good content validity and construct validity. The instrument is suggested for use in measuring of the quality of the physical environment in residential care facilities for older persons.

Validade e fiabilidade do protocolo de avaliação da qualidade vocal da Universidade de Aveiro

O presente estudo teve como principal objetivo rever a versão original do Protocolo de Avaliação da Qualidade Vocal da Universidade de Aveiro (PAQVUA), criar o respetivo manual, comprovar a validade de conteúdo do PAQVUA assim como a sua fiabilidade inter e intra avaliador. Para rever a versão original do PAQVUA, foi efetuada uma revisão narrativa da literatura, com o objetivo de encontrar informação que revelasse a pertinência, da versão original do protocolo, para assim se efetuarem as alterações necessárias. Para analisar a validade de conteúdo recorreu-se a um painel de peritos, num total de seis, especialistas na área da voz que avaliaram as provas do PAQVUA, folha de registo e manual. Os peritos avaliaram todos os componentes do PAQVUA através do preenchimento de um questionário com o objetivo de se comprovar a relevância, clareza e precisão de cada uma das provas de avaliação e também o conteúdo do seu manual. Para além disso pretendeu-se também comprovar a fiabilidade inter e intra avaliador do PAQVUA. Para isso, o mesmo protocolo foi aplicado duas vezes por duas avaliadoras diferentes e também foi aplicado em dois momentos temporais diferentes por uma só avaliadora. A amostra teve um total de doze participantes. Para analisar estatisticamente os resultados da validade de conteúdo utilizou-se o método gráfico Bland e Altman modificado e o Coeficiente de Correlação Intraclasses (CCI). Para a fiabilidade inter e intra avaliador recorreu-se ao Coeficiente de Correlação de Spearman (ρ) e ao Kappa de Cohen (k). Relativamente aos resultados obtidos com o método Bland e Altman modificado, verificou-se a existência de concordância entre os peritos através da análise dos gráficos, sendo que a maioria dos pontos se encontrou dentro dos limites esperados de concordância. Quanto aos valores do CCI (entre 0.379 e 0.479), estes revelaram uma correlação regular. No que concerne à fiabilidade inter avaliador, os resultados de correlação e concordância apresentaram-se relevantes, com valores de ρ (de Spearman) superiores a 0.700 e valores de k (de Cohen) superiores a 0.600, na maioria dos casos. O mesmo foi constatado para a fiabilidade intra avaliador. Desta forma pode concluir-se que a nova versão do PAQVUA apresenta validade de conteúdo sendo tal comprovado pelo método de Bland-Altman modificado e pelos valores do CCI. Quanto à fiabilidade inter e intra avaliador, pode afirmar-se que o PAQVUA é fiável, porém como a amostra em estudo é pequena, o que a torna pouco representativa, esta conclusão carece de fundamento com base num estudo mais alargado. Apesar da limitação referida, acredita-se que este protocolo é uma mais-valia para o estudo da patologia vocal em ambiente clínico, assim como para a investigação científica nesta área, pois através do PAQVUA podem-se recolher diversas informações relacionadas com a voz, úteis para uma intervenção terapêutica baseada em evidências científicas.

Development and preliminary psychometric properties of the Care Experience Feedback Improvement Tool (CEFIT).

Objective To develop a structurally valid and reliable, yet brief measure of patient experience of hospital quality of care, the Care Experience Feedback Improvement Tool (CEFIT). Also, to examine aspects of utility of CEFIT. Background Measuring quality improvement at the clinical interface has become a necessary component of healthcare measurement and improvement plans, but the effectiveness of measuring such complexity is dependent on the purpose and utility of the instrument used. Methods CEFIT was designed from a theoretical model, derived from the literature and a content validity index (CVI) procedure. A telephone population surveyed 802 eligible participants (healthcare experience within the previous 12 months) to complete CEFIT. Internal consistency reliability was tested using Cronbach's α. Principal component analysis was conducted to examine the factor structure and determine structural validity. Quality criteria were applied to judge aspects of utility. Results CVI found a statistically significant proportion of agreement between patient and practitioner experts for CEFIT construction. 802 eligible participants answered the CEFIT questions. Cronbach's α coefficient for internal consistency indicated high reliability (0.78). Interitem (question) total correlations (0.28–0.73) were used to establish the final instrument. Principal component analysis identified one factor accounting for 57.3% variance. Quality critique rated CEFIT as fair for content validity, excellent for structural validity, good for cost, poor for acceptability and good for educational impact. Conclusions CEFIT offers a brief yet structurally sound measure of patient experience of quality of care. The briefness of the 5-item instrument arguably offers high utility in practice. Further studies are needed to explore the utility of CEFIT to provide a robust basis for feedback to local clinical teams and drive quality improvement in the provision of care experience for patients. Further development of aspects of utility is also required.

Glasgow Psychosis Screening tool for use in adults with Intellectual Disabilities (GPS-ID): development and psychometric properties

Background: Prevalence of psychosis is known to be higher in adults with intellectual disabilities (ID) than in the general adult population. However, there have been no attempts to develop a psychosis screening tool specifically for the adult ID population. The present study describes the development and preliminary evaluation of a new measure, the Glasgow Psychosis Screening tool for use in Adults with Intellectual Disabilities (GPS-ID). Method: An item pool was generated following: 1) focus groups with adults with ID and psychosis, and their carers and/or workers; 2) expert input from clinicians. A draft scale was compiled and refined following expert feedback. The new scale, along with the Psychotic Symptom Rating Scales was administered to 20 adults with ID (10 with and 10 without psychosis) and their relative or carers. Results: The GPS-ID total score, self-report subscale and informant rating-subscale differentiated psychosis and non-psychosis groups. The tool had good internal consistency (Cronbach’s α=0.91), and a cut-off score ≥4 yielded high sensitivity (90%) and specificity (100%). The method of tool development supports face and content validity. Criterion validity was not supported. Conclusions: Preliminary investigation of the tool’s psychometric properties is positive, although further investigation is required. The tool is accessible to adults with mild to moderate ID and can be completed in 15-30 minutes. The GPS-ID is not a diagnostic tool, therefore any adult exceeding the cut-off score of ≥4 should receive further assessment.

The Warwick-Edinburgh Mental Well-Being Scale

Background There is increasing international interest in the concept of mental well-being and its contribution to all aspects of human life. Demand for instruments to monitor mental well-being at a population level and evaluate mental health promotion initiatives is growing. This article describes the development and validation of a new scale, comprised only of positively worded items relating to different aspects of positive mental health: the Warwick-Edinburgh Mental Well-Being Scale (WEMWBS). Methods WEMWBS was developed by an expert panel drawing on current academic literature, qualitative research with focus groups, and psychometric testing of an existing scale. It was validated on a student and representative population sample. Content validity was assessed by reviewing the frequency of complete responses and the distribution of responses to each item. Confirmatory factor analysis was used to test the hypothesis that the scale measured a single construct. Internal consistency was assessed using Cronbach’s alpha. Criterion validity was explored in terms of correlations between WEMWBS and other scales and by testing whether the scale discriminated between population groups in line with pre-specified hypotheses. Test-retest reliability was assessed at one week using intra-class correlation coefficients. Susceptibility to bias was measured using the Balanced Inventory of Desired Responding. Results WEMWBS showed good content validity. Confirmatory factor analysis supported the single factor hypothesis. A Cronbach’s alpha score of 0.89 (student sample) and 0.91 (population sample) suggests some item redundancy in the scale. WEMWBS showed high correlations with other mental health and well-being scales and lower correlations with scales measuring overall health. Its distribution was near normal and the scale did not show ceiling effects in a population sample. It discriminated between population groups in a way that is largely consistent with the results of other population surveys. Test–retest reliability at one week was high (0.83). Social desirability bias was lower or similar to that of other comparable scales. Conclusions WEMWBS is a measure of mental well-being focusing entirely on positive aspects of mental health. As a short and psychometrically robust scale, with no ceiling effects in a population sample, it offers promise as a tool for monitoring mental well-being at a population level. Whilst WEMWBS should appeal to those evaluating mental health promotion initiatives, it is important that the scale’s sensitivity to change is established before it is recommended in this context.

Development and psychometric evaluation of the basic electrocardiogram interpretation self-efficacy scale

Purpose: Research suggests that nurses and nursing students lack competence in basic electrocardiogram (ECG) interpretation. Self-efficacy is considered to be paramount in the development of one's competence. The aim of this study was to develop and psychometrically evaluate a scale to assess self-efficacy of nursing students in basic ECG interpretation. Materials and methods: Observational cross-sectional study with a convenience sample of 293 nursing students. The basic ECG interpretation self-efficacy scale (ECG-SES) was developed and psychometrically tested in terms of reliability (internal consistency and temporal stability) and validity (content, criterion and construct). The ECG-SES’ internal consistency was explored by calculating the Cronbach's alpha coefficient (α); its temporal stability was investigated by calculating the Pearson correlation coefficient (r) between the participants’ results on a test–retest separated by a 4-week interval. The content validity index of the items (I-CVI) and the scale (S-CVI) was calculated based on the reviews of a panel of 16 experts. Criterion validity was explored by correlating the participants’ results on the ECG-SES with their results on the New General Self-Efficacy Scale (NGSE). 1 Construct validity was investigated by performing Principal Component Analysis (PCA) and known-group analysis. Results: The excellent reliability of the ECG-SES was evidenced by its internal consistency (α = 0.98) and its temporal stability at the 4-week re-test (r = 0.81; p < 0.01). The ECG-SES’ content validity was also excellent (all items’ I-CVI = 0.94–1; S-CVI = 0.99). A strong, significant correlation between the NGSE and the ECG-SES (r = 0.70; p < 0.01) showed its criterion validity. Corroborating the ECG-SES’ construct validity, PCA revealed that all its items loaded on a single factor that explained 74.6% of the total variance found. Furthermore, known-groups analysis showed the ECG-SES’ ability to detect expected differences in self-efficacy between groups with different training experiences (p < 0.01). Conclusion: The ECG-SES showed excellent psychometric properties for measuring the self-efficacy of nursing students in basic ECG interpretation.

Regional Diversification and Performance: Conceptualization and Competing Strategic Objectives

This dissertation comprised of three essays provides justification for the need to pursue research on multinationality and performance with a more fine-grained approach. Essay one is a conceptual response to an article written by Jean-Francois Hennart in 2011 which questions the need and approach toward future research in this domain. I argue that internalization theory does not render multinationality and performance research meaningless and identify key areas where methodological enhancements can be made to strengthen our research findings with regard to Hennart’s call for more content validity. Essay two responds to the need for more-fine grained research on the consequences of multinationality by introducing non-traditional measures of performance such as social and environmental performance and adopting a more theoretically relevant construct of regionalization to capture international diversification levels of the firm. Using data from the world’s largest 600 firms (based on sales) derived from Bloomberg and the Directory of Corporate Affiliates; I employ general estimating equation analysis to account for the auto-correlated nature of the panel data alongside multivariate regression techniques. Results indicate that regionalization has a positive relationship with economic performance while it has a negative relationship with environmental and social performance outcomes, often referred to as the “Triple Bottom-Line” performance. Essay three builds upon the work in the previous essays by linking the aforementioned performance variables and sample to corporate reputation which has been shown to be a beneficial strategic asset. Using Structural Equation Modeling I explore economic, environmental and social signals as mediators on relationship between regionalization and firm reputation. Results indicate that these variables partially mediate a positive relationship between regionalization and firm reputation. While regionalization positively affects the reputation building signal of economic performance, it aids in reputation building by reducing environmental and social disclosure effects which interestingly impact reputation negatively. In conclusion, the dissertation submits opportunities for future research and contributes to research by demonstrating that regionalization affects performance, but the effect varies in accordance with the performance criterion and context. In some cases, regional diversification may produce competing or conflicting outcomes among the potential strategic objectives of the firm.

Propiedades psicométricas de los instrumentos para la medición de la actividad física en adultos con discapacidad física relacionada con lesión de médula espinal: una revisión sistemática

Objetivo: Evaluar las propiedades psicométricas de los instrumentos para la medición de la actividad física en adultos de 18-65 años con discapacidad física por lesión de médula espinal. Materiales y métodos: Revisión sistemática. Las bases de datos de Medline, Scopus, Web of Science y 19 revistas especializadas fueron consultadas durante once días entre abril de 2015 y febrero de 2016 para identificar estudios originales de validación, sin límite de tiempo y que estuvieran publicados en español, francés y/o inglés. La calidad metodológica de los instrumentos de medición se evaluó usando las diferentes cajas de propiedades de la lista COSMIN. Resultados: Se identificaron 9229 referencias, de las cuales sólo 12 cumplieron los criterios de inclusión, dando como resultado 13 instrumentos de medición. Se evaluaron seis propiedades psicométricas. La propiedad más común fue la confiabilidad, además se observó que la calidad metodológica de los estudios incluidos no representa los resultados de las propiedades psicométricas de los instrumentos de medición. La calidad metodológica de los instrumentos para la evaluación de la actividad física en población con lesión medular espinal es “baja” para propiedades como consistencia interna, error de medición, sensibilidad, validez de criterio (con excepción del WISCI II que tiene buena validez) y excelente para validez de contenido y fiabilidad. Conclusión: Se ha encontrado que instrumentos empleados hasta el presente en la medición de la actividad física en población con discapacidad física relacionada con lesión de médula espinal han sido creados para otros tipos de discapacidad y otros instrumentos deben ser validados en futuros estudios.

Morbilidade evitável relacionada com medicamentos: Desenvolvimento de novos indicadores e validação preliminar para cuidados primários em Portugal

Sumário: A morbilidade evitável relacionada com medicamentos (MERM) é um problema de saúde pública com considerável impacto negativo para os doentes e os sistemas de saúde. Os indicadores de MERM constituem medidas operacionais de gestão do risco terapêutico, uma vez que permitem identificar processos de cuidados de saúde que predizem um resultado clínico negativo potencialmente evitável. O uso destes indicadores pode contribuir para reduzir a morbilidade evitável relacionada com medicamentos, evitando danos desnecessários e desperdício de recursos. Objectivo: Desenvolver novos indicadores de MERM para cuidados primários em Portugal a partir da análise de resumos das características de medicamentos (RCM's) e determinar a validade preliminar de face e de conteúdo dos indicadores derivados para este setting. Métodos: A primeira fase deste estudo consistiu num ensaio piloto, com o intuito de testar uma metodologia de inclusão dos RCM's com base na frequência de dispensa da especialidade farmacêutica no ambulatório. Seguidamente procedeu-se ao desenvolvimento de mais indicadores de MERM utilizando a metodologia testada no ensaio piloto. Os indicadores obtidos foram alvo de uma primeira análise com base em aspectos como duplicações e relevância para cuidados primários, de modo a seleccionar os que poderiam passar à fase de validade preliminar de face e de conteúdo. Procedeu­ se então à pesquisa de evidência clínica em fontes de referência para estes indicadores. Na última fase do estudo estes indicadores, bem como a respectiva evidência clínica, foram analisados por um painel de peritos constituído por quatro académicos (dois médicos de família e dois farmacêuticos), sendo aprovados ou eliminados com base num critério de consenso. Resultados: Obteve-se um total de 64 indicadores de MERM, a partir da análise de 35 RCM's. Sujeitaram-se à determinação da validade preliminar de face e de conteúdo 44 indicadores. Foram aprovados por consenso 28 indicadores, tendo sido excluídos 17 (4 por consenso e 13 sem obtenção de consenso). Conclusão: É exequível derivar novos indicadores de MERM para cuidados primários a partir da análise de RCM's. A validade formal de face e de conteúdo destes indicadores obtidos será objecto de estudo de ulterior investigação. /ABSTRACT: Backgroud: Preventable drug-relate morbidity (PDRM) is a public health problem with significant negative impact at a patient and system level. PDRM indicators are operational measures of therapeutic risk management; they identifying processes of care leading to preventable adverse outcomes. The use of these indicators may contribute to tackle preventable drug related morbidity, avoiding unnecessary harm and waste of resources. Objective: To develop new PDRM indicators to Portuguese primary care based on the analysis of summaries of product characteristics (SPC’s) and to determine their preliminary face and content validity to this setting. Methods: Firstly a pilot study was conducted to test a methodology for including SPC's based on the most frequently sold medicines in the ambulatory. Then more indicators were developed using the previously tested methodology. The indicators obtained were analyzed in respect to aspects such as duplications and relevance for primary care to select those that could proceed to the next stage. Clinical evidence was searched for each of these indicators in gold-standard information sources. Finally, this set of indicators and the respective clinical evidence were analysised by a panel of four experts (two academic general practitioners and two academic pharmacists). Preliminary face and content validity was established by means of consensus. Results: A total of 64 indicators was obtained, based on the analysis of 35 SPCs. Forty­ four indicators were subjected to a preliminary assessment of face and content validity, resulting in 28 consensus-approved indicators. Seventeen indicators were excluded (4 rejected by consensus and 13 that did not reach consensus). Conclusion: lt is feasible to derive new PDRM indicators for primary care based on SPC's. The formal face and content validity of the indicators will be determined in a further study.

Validity evidence based on test content

Resumen tomado de la publicación

Face, content and construct validity of a novel robotic surgery simulator

We evaluated the face, content and construct validity of the novel da Vinci® Skills Simulator™ using the da Vinci Si™ Surgeon Console as the surgeon interface.

Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons

Background The aim of this study is to present face, content, and constructs validity of the endoscopic orthogonal video system (EndoViS) training system and determines its efficiency as a training and objective assessment tool of the surgeons’ psychomotor skills. Methods Thirty-five surgeons and medical students participated in this study: 11 medical students, 19 residents, and 5 experts. All participants performed four basic skill tasks using conventional laparoscopic instruments and EndoViS training system. Subsequently, participants filled out a questionnaire regarding the design, realism, overall functionality, and its capabilities to train hand–eye coordination and depth perception, rated on a 5-point Likert scale. Motion data of the instruments were obtained by means of two webcams built into a laparoscopic physical trainer. To identify the surgical instruments in the images, colored markers were placed in each instrument. Thirteen motion-related metrics were used to assess laparoscopic performance of the participants. Statistical analysis of performance was made between novice, intermediate, and expert groups. Internal consistency of all metrics was analyzed with Cronbach’s α test. Results Overall scores about features of the EndoViS system were positives. Participants agreed with the usefulness of tasks and the training capacities of EndoViS system (score >4). Results presented significant differences in the execution of three skill tasks performed by participants. Seven metrics showed construct validity for assessment of performance with high consistency levels. Conclusions EndoViS training system has been successfully validated. Results showed that EndoViS was able to differentiate between participants of varying laparoscopic experience. This simulator is a useful and effective tool to objectively assess laparoscopic psychomotor skills of the surgeons.

Self-reported calcium intake and bone mineral content in children and adolescents

Objective: We examined the relationship between self-reported calcium (Cal intake and bone mineral content (BMC) in children and adolescents. We hypothesized that an expression of Ca adjusted for energy intake (El), i.e., Ca density, would be a better predictor of BMC than unadjusted Ca because of underreporting of EI. Methods: Data were obtained on dietary intakes (repeated 24-hour recalls) and BMC (by DEXA) in a cross-section of 227 children aged 8 to 17 years. Bivariate and multivariate analyses were used to examine die relationship between Ca, Ca density, and the dependent variables total body BMC and lumbar spine BMC. Covariates included were height, weight, bone area, maturity age, activity score and El. Results: Reported El compared to estimated basal metabolic rate suggested underreporting of El. Total body and lumbar spine BMC were significantly associated with El, but not Ca or Ca density, in bivariate analyses. After controlling for size and maturity, multiple linear regression analysis revealed unadjusted Ca to be a predictor of BMC in males in the total body (p = 0.08) and lumbar spine (p = 0.01). Unadjusted Ca was not a predictor of BMC at either site in females. Ca density was not a better predictor of BMC at either site in males or females. Conclusions: The relationship observed in male adolescents in this study between Ca intake and BMC is similar to that seen in clinical trials. Ca density did not enable us to see a relationship between Ca intake and BMC in females, which may reflect systematic reporting errors or that diet is not a limiting factor in this group of healthy adolescents.

Validity and reliability of the Spanish version of the DN4 (Douleur Neuropathique 4 questions) questionnaire for differential diagnosis of pain syndromes associated to a neuropathic or somatic component

BACKGROUND This study assesses the validity and reliability of the Spanish version of DN4 questionnaire as a tool for differential diagnosis of pain syndromes associated to a neuropathic (NP) or somatic component (non-neuropathic pain, NNP). METHODS A study was conducted consisting of two phases: cultural adaptation into the Spanish language by means of conceptual equivalence, including forward and backward translations in duplicate and cognitive debriefing, and testing of psychometric properties in patients with NP (peripheral, central and mixed) and NNP. The analysis of psychometric properties included reliability (internal consistency, inter-rater agreement and test-retest reliability) and validity (ROC curve analysis, agreement with the reference diagnosis and determination of sensitivity, specificity, and positive and negative predictive values in different subsamples according to type of NP). RESULTS A sample of 164 subjects (99 women, 60.4%; age: 60.4 +/- 16.0 years), 94 (57.3%) with NP (36 with peripheral, 32 with central, and 26 with mixed pain) and 70 with NNP was enrolled. The questionnaire was reliable [Cronbach's alpha coefficient: 0.71, inter-rater agreement coefficient: 0.80 (0.71-0.89), and test-retest intra-class correlation coefficient: 0.95 (0.92-0.97)] and valid for a cut-off value > or = 4 points, which was the best value to discriminate between NP and NNP subjects. DISCUSSION This study, representing the first validation of the DN4 questionnaire into another language different than the original, not only supported its high discriminatory value for identification of neuropathic pain, but also provided supplemental psychometric validation (i.e. test-retest reliability, influence of educational level and pain intensity) and showed its validity in mixed pain syndromes.