The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content

Background: The COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) was developed in an international Delphi study to evaluate the methodological quality of studies on measurement properties of health-related patient reported outcomes (HR-PROs). In this paper, we explain our choices for the design requirements and preferred statistical methods for which no evidence is available in the literature or on which the Delphi panel members had substantial discussion. Methods: The issues described in this paper are a reflection of the Delphi process in which 43 panel members participated. Results: The topics discussed are internal consistency (relevance for reflective and formative models, and distinction with unidimensionality), content validity (judging relevance and comprehensiveness), hypotheses testing as an aspect of construct validity (specificity of hypotheses), criterion validity (relevance for PROs), and responsiveness (concept and relation to validity, and (in) appropriate measures).Conclusions: We expect that this paper will contribute to a better understanding of the rationale behind the items, thereby enhancing the acceptance and use of the COSMIN checklist.

Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) Checklist

Background: The COSMIN checklist is a tool for evaluating the methodological quality of studies on measurement properties of health-related patient-reported outcomes. The aim of this study is to determine the inter-rater agreement and reliability of each item score of the COSMIN checklist (n = 114). Methods: 75 articles evaluating measurement properties were randomly selected from the bibliographic database compiled by the Patient-Reported Outcome Measurement Group, Oxford, UK. Raters were asked to assess the methodological quality of three articles, using the COSMIN checklist. In a one-way design, percentage agreement and intraclass kappa coefficients or quadratic-weighted kappa coefficients were calculated for each item. Results: 88 raters participated. Of the 75 selected articles, 26 articles were rated by four to six participants, and 49 by two or three participants. Overall, percentage agreement was appropriate (68% was above 80% agreement), and the kappa coefficients for the COSMIN items were low (61% was below 0.40, 6% was above 0.75). Reasons for low inter-rater agreement were need for subjective judgement, and accustom to different standards, terminology and definitions.Conclusions: Results indicated that raters often choose the same response option, but that it is difficult on item level to distinguish between articles. When using the COSMIN checklist in a systematic review, we recommend getting some training and experience, completing it by two independent raters, and reaching consensus on one final rating. Instructions for using the checklist are improved.

Protocol of the COSMIN study: consensus-based standards for the selection of health measurement instruments

Background: Choosing an adequate measurement instrument depends on the proposed use of the instrument, the concept to be measured, the measurement properties (e.g. internal consistency, reproducibility, content and construct validity, responsiveness, and interpretability), the requirements, the burden for subjects, and costs of the available instruments. As far as measurement properties are concerned, there are no sufficiently specific standards for the evaluation of measurement properties of instruments to measure health status, and also no explicit criteria for what constitutes good measurement properties. In this paper we describe the protocol for the COSMIN study, the objective of which is to develop a checklist that contains COnsensus-based Standards for the selection of health Measurement INstruments, including explicit criteria for satisfying these standards. We will focus on evaluative health related patient-reported outcomes (HR-PROs), i.e. patient-reported health measurement instruments used in a longitudinal design as an outcome measure, excluding health care related PROs, such as satisfaction with care or adherence. The COSMIN standards will be made available in the form of an easily applicable checklist.Method: An international Delphi study will be performed to reach consensus on which and how measurement properties should be assessed, and on criteria for good measurement properties. Two sources of input will be used for the Delphi study: (1) a systematic review of properties, standards and criteria of measurement properties found in systematic reviews of measurement instruments, and (2) an additional literature search of methodological articles presenting a comprehensive checklist of standards and criteria. The Delphi study will consist of four (written) Delphi rounds, with approximately 30 expert panel members with different backgrounds in clinical medicine, biostatistics, psychology, and epidemiology. The final checklist will subsequently be field-tested by assessing the inter-rater reproducibility of the checklist.Discussion: Since the study will mainly be anonymous, problems that are commonly encountered in face-to-face group meetings, such as the dominance of certain persons in the communication process, will be avoided. By performing a Delphi study and involving many experts, the likelihood that the checklist will have sufficient credibility to be accepted and implemented will increase.

Methodological note about the determination of key sectors in final energy consumption: a preliminary approach to the Spanish case

In this paper we analyze the determination of "key" sectors in the final energy consumption. We approach this issue from an input-output perspective and we design a methodology based on the elasticities of the demands of final energy consumption. As an exercise, we apply the proposed methodology to the Spanish economy. The analysis allows us to indicate the greater or lesser relevance of the different sectors in the consumption of final energy, pointing out which sectors deserve greater attention in the Spanish case and showing the implications for energy policy.

Aprender investigando. Una experiencia formativa en el ámbito de la enfermería = Research teaching processes used as a methodological strategy in the education of healthcare professionals

Este trabajo tiene como propósito presentar y valorar, desde la perspectiva del alumnado participante, un proyecto de investigación-formación puesto en marcha durante el curso 2003-2004 en la elaboración del trabajo de tesina, fin de carrera, en la Escuela de Enfermería de Vitoria, dentro del programa de Licenciatura Europea de Enfermería. Constituye el punto de partida de un proyecto a largo plazo, iniciado con la intención de desarrollar principios teóricos y procedimientos prácticos que nos permitan sistematizar procesos formativos que, centrados en la investigación, articulen la teoría y la práctica e integren una perspectiva comunicativa y cooperativa.

Open to the users' needs : combining user-centered design, standards and open source software

This case study introduces our continuous work to enhance the virtual classroom in order to provide faculty and students with an environment open to their needs, compliant with learning standards and, therefore compatible with other e-learning environments, and based on open source software. The result is a modulable, sustainable and interoperable learning environment that can be adapted to different teaching and learning situations by incorporating the LMS integrated tools as well as wikis, blogs, forums and Moodle activities among others.

OER in Portugal as agent of curriculum innovation and technological change. Inducing practices of "new" teaching standards

This research project aimed the following goal: promote the creation, use and disclosure of OER in a Group of Schools, involving schools and teachers from different learning levels, expecting to test and validate the use of OER, in a learning-teaching model towards curricular innovation. Defining as a starting point different subjects and teachers from distinct academic areas, we have implemented a set of activities leading to the creation of OER supported, when possible, in FLOSS tools. We adopted an action research methodology with a dual purpose: to act within a community of teachers and students, while increasing at the same time their knowledge, as well as the researcher's. The activity was developed cooperatively in order to process a certain reality of the teaching-learning process, through practical/reflective action towards it and inducing its implementation by others in the Portuguese School System, based on the production and sharing OER.

Ecological Footprint Inequality: A methodological review and some results

Scarcities of environmental services are no longer merely a remote hypothesis. Consequently, analysis of their inequalities between nations becomes of paramount importance for the achievement of sustainability in terms either of international policy, or of Universalist ethical principles of equity. This paper aims, on the one hand, at revising methodological aspects of the inequality measurement of certain environmental data and, on the other, at extending the scarce empirical evidence relating to the international distribution of Ecological Footprint (EF), by using a longer EF time series. Most of the techniques currently important in the literature are revised and then tested on EF data with interesting results. We look in depth at Lorenz dominance analyses and consider the underlying properties of different inequality indices. Those indices which fit best with environmental inequality measurements are CV2 and GE(2) because of their neutrality property, however a trade-off may occur when subgroup decompositions are performed. A weighting factor decomposition method is proposed in order to isolate weighting factor changes in inequality growth rates. Finally, the only non-ambiguous way of decomposing inequality by source is the natural decomposition of CV2, which additionally allows the interpretation of marginal term contributions. Empirically, this paper contributes to the environmental inequality measurement of EF: this inequality has been quite stable and its change over time is due to per capita vector changes rather than population changes. Almost the entirety of the EF inequality is explainable by differences in the means between the countries of the World Bank group. This finding suggests that international environmental agreements should be attempted on a regional basis in an attempt to achieve greater consensus between the parties involved. Additionally, source decomposition warns of the dangers of confining CO2 emissions reduction to crop-based energies because of the implications for basic needs satisfaction.

Ecological Footprint Inequality: A methodological review and some results

Scarcities of environmental services are no longer merely a remote hypothesis. Consequently, analysis of their inequalities between nations becomes of paramount importance for the achievement of sustainability in terms either of international policy, or of Universalist ethical principles of equity. This paper aims, on the one hand, at revising methodological aspects of the inequality measurement of certain environmental data and, on the other, at extending the scarce empirical evidence relating to the international distribution of Ecological Footprint (EF), by using a longer EF time series. Most of the techniques currently important in the literature are revised and then tested on EF data with interesting results. We look in depth at Lorenz dominance analyses and consider the underlying properties of different inequality indices. Those indices which fit best with environmental inequality measurements are CV2 and GE(2) because of their neutrality property, however a trade-off may occur when subgroup decompositions are performed. A weighting factor decomposition method is proposed in order to isolate weighting factor changes in inequality growth rates. Finally, the only non-ambiguous way of decomposing inequality by source is the natural decomposition of CV2, which additionally allows the interpretation of marginal term contributions. Empirically, this paper contributes to the environmental inequality measurement of EF: this inequality has been quite stable and its change over time is due to per capita vector changes rather than population changes. Almost the entirety of the EF inequality is explainable by differences in the means between the countries of the World Bank group. This finding suggests that international environmental agreements should be attempted on a regional basis in an attempt to achieve greater consensus between the parties involved. Additionally, source decomposition warns of the dangers of confining CO2 emissions reduction to crop-based energies because of the implications for basic needs satisfaction. Keywords: ecological footprint; ecological inequality measurement, inequality decomposition.

Beyond GDP: Methodological and measurement issues in redefining “wealth”

In the last decades; a growing stock of literature has been devoted to the criticism of GDP as an indicator of societal wealth. A relevant question is: what are the perspectives to build, on the existing knowledge and consensus, alternative measures of prosperity? A starting point may be to connect well-being research agenda with the sustainability one. However, there is no doubt that there is a lot of complexity and fuzziness inherent in multidimensional concepts such as sustainability and well-being. This article analyses the theoretical foundations and the empirical validity of some multidimensional technical tools that can be used for well-being evaluation and assessment. Of course one should not forget that policy conclusions derived through any mathematical model depend also on the conceptual framework used, i.e. which representation of reality (and thus which societal values and interests) has been considered.

Marco metodológico para la definición de un perfil profesional en educación superior = Methodological framework for the definition of a professional profile in higher education

En el marco del Espacio Europeo de Educación Superior 1 se inscriben innumerables acciones que están desarrollando todas las universidades europeas con la intención de llegar al 2010 con un bagaje lo suficientemente importante como para afrontar este cambio con entereza y sobretodo conservando (o incrementando) la calidad de los procesos de enseñanza- aprendizaje que hasta el momento han predominado. Dentro de este amplio proceso de transformación se encuentra el diseño de los nuevos Grados que brinda la oportunidad de replantear los planes de estudios, por tanto, la organización de las asignaturas, estructura de los contenidos, metodologías y sistemas de evaluación. Toda esta reflexión debe girar, a nuestro entender, alrededor de tres núcleos que se encuentran en estado deinterdependencia: los escenarios profesionales, los perfiles profesionales y las competencias que en ellos se inscriben.Para poder plantear los nuevos Grados en coherencia con la era del EEES debe hacerse un análisis minucioso de los perfil profesionales demandados por el mercado de trabajo que, al fin y al cabo será el destino de los profesionales que se forman en nuestras universidades, es por ello, que la tarea de definir el perfil profesional a priori del diseño de los nuevos Grados resulta una máxima para garantizar la calidad de estos. En este trabajo se presenta un ejemplo de metodología a seguir para la definición de un perfil profesional en Educación Superior, concretamente, el del Ingeniero TIC mediante el Análisis Funcional.

Construyendo las bases para una comparación fiable: la Encuesta Social Europea

Surveys are a valuable instrument to find out about the social and politicalreality of our context. However, the work of researchers is often limitedby a number of handicaps that are mainly two. On one hand, the samples areusually low technical quality ones and the fieldwork is not carried out inthe finest conditions. On the other hand, many surveys are not especiallydesigned to allow their comparison, a precisely appreciated operation inpolitical research. The article presents the European Social Survey andjustifies its methodological bases. The survey, promoted by the EuropeanScience Foundation and the European Commission, is born from the collectiveeffort of the scientific community with the explicit aim to establishcertain quality standards in the sample design and in the carrying out ofthe fieldwork so as to guarantee the quality of the data and allow eachcomparison between countries.

The European Social Survey. Methodological aspects

From a scientific point of view, surveys are undoubtedly a valuable tool for the knowledge of the social and political reality. They are widely used in the social sciences research. However, the researcher's task is often disturbed by a series of deficiencies related to some technical aspects that make difficult both the inference and the comparison. The main aim of the present paper is to report and justify the European Social Survey's technical specifications addressed to avoid and/or minimize such deficiencies. The article also gives a characterization of the non-respondents in Spain obtained from the analysis of the 2002 fieldwork data file.

Sweet diversity: Colonial goods and the rise of European living standards after 1492

When did overseas trade start to matter for living standards? Traditional real-wage indices suggest that living standards in Europe stagnated before 1800. In this paper, we argue thatwelfare rose substantially, but surreptitiously, because of an influx of new goods as a result ofoverseas trade. Colonial luxuries such as tea, coffee, and sugar transformed European diets afterthe discovery of America and the rounding of the Cape of Good Hope. These goods became household items in many countries by the end of the 18th century. We use three different methodsto calculate welfare gains based on price data and the rate of adoption of these new colonialgoods. Our results suggest that by 1800, the average Englishman would have been willing to forego 10% or more of his income in order to maintain access to sugar and tea alone. These findings are robust to a wide range of alternative assumptions, data series, and valuation methods.

Setting standards: Information accumulation in development

We propose a model in which economic relations and institutions in advancedand less developed economies differ as these societies have access to different amounts of information. This lack of information makes it hard to give the right incentives to managers and entrepreneurs. We argue that differences in the amount of information arise because of the differences in the scale of activities in rich and poor economies; namely, there is too little repetition of similar activities in pooreconomies, thus insufficient information to set the appropriate standards for firm performance. Our model predicts a number of institutional and structural transformations as the economy accumulates capital and information.