Design of an assessment methodology for Educational Innovation Groups in the EHEA.

This document explains the process of designing a methodology to evaluate Educational Innovation Groups, which are structures created within universities in the context of adaptation to the European Higher Education Area. These groups are committed to introduce innovation in educational processes as a means to improve educational quality. The assessment design is based on a participatory model of planning called Working With People, that tries to integrate the perspectives of all stakeholders. The aim of the methodology is to be a useful tool for the university to evaluate the work done by the groups, encourage the members to continue improving the quality of teaching and reorient the activities to fulfill the emergent needs that the university faces.

GLMP for automatic assessment of DFS algorithm learning

We describe how to use a Granular Linguistic Model of a Phenomenon (GLMP) to assess e-learning processes. We apply this technique to evaluate algorithm learning using the GRAPHs learning environment.

Relevance of motion-related assessment metrics in laparoscopic surgery

INTRODUCTION: Motion metrics have become an important source of information when addressing the assessment of surgical expertise. However, their direct relationship with the different surgical skills has not been fully explored. The purpose of this study is to investigate the relevance of motion-related metrics in the evaluation processes of basic psychomotor laparoscopic skills, as well as their correlation with the different abilities sought to measure. METHODS: A framework for task definition and metric analysis is proposed. An explorative survey was first conducted with a board of experts to identify metrics to assess basic psychomotor skills. Based on the output of that survey, three novel tasks for surgical assessment were designed. Face and construct validation study was performed, with focus on motion-related metrics. Tasks were performed by 42 participants (16 novices, 22 residents and 4 experts). Movements of the laparoscopic instruments were registered with the TrEndo tracking system and analyzed. RESULTS: Time, path length and depth showed construct validity for all three tasks. Motion smoothness and idle time also showed validity for tasks involving bi-manual coordination and tasks requiring a more tactical approach respectively. Additionally, motion smoothness and average speed showed a high internal consistency, proving them to be the most task-independent of all the metrics analyzed. CONCLUSION: Motion metrics are complementary and valid for assessing basic psychomotor skills, and their relevance depends on the skill being evaluated. A larger clinical implementation, combined with quality performance information, will give more insight on the relevance of the results shown in this study.

Specific skills assessment learned by students of building engineering and technical architecture in topics relating to durability and control of concrete structures

In many university courses such as Building Engineering or Technical Architectural, the high density of the contents included in the curriculum, make the student, after graduation, unable to develop the skills already acquired and evaluated in the disciplines of the first courses. From the Group of Educational Innovation at the Polytechnic University of Madrid (UPM) "Teaching of Structural Concrete" (GIEHE) we have conducted a study in which are valued specific skills acquired by students after the first courses of career. We have worked with students from UPM fourth-year career and with Technical Architecture students who have completed their studies and also have completed the Adaptation Course of Technical Architecture to the Building Engineer. The work is part of the Educational Innovation Project funded by the UPM "Integration of training and assessment of generic and specific skills in structural concrete" We have evaluated specific skills learned in the areas of durability and control of structural concrete structures. The results show that overall, students are not able to fully develop the skills already acquired earlier, even being these essential to their professional development. Possibly, the large amount of content taught in these degrees together with a teaching and assessment of "flat profile", ie, which are presented and evaluated with the same intensity as the fundamental and the accessory, are causes enough to cause these results.

Competence assessment of final year projects for undergraduate telecommunication students

The value of project-based learning has lead to the inclusion of project development activities in engineering courses, being the Final Year Project (FYP) the most remarkable one. Several approaches have been proposed for assessing and grading FYPs but, among them, rubrics are becoming a standard for such type of assessment. However, due to the different characteristics and orientations of the projects (some are more practically oriented, some more theoretically), and the high amount of different competences to be evaluated (knowledge, working capability, communication skills, etc.), the definition of one unique rubric suitable for the evaluation of all FYPs presented in different degree programs, is a big challenge. In a former work, the educational outcomes expected from the FYP were defined and resulted in a proposal for their assessment. Afterwards, the proposal has been tested during one year within an educational innovation-project at the Universidad Politécnica de Madrid which involved the follow-up of 8 undergraduate telecommunication students elaborating their FYPs. In this publication, our experience will be described, based on the emerging work taking place through the formalisation of the process which consisted in the following steps: i) establishment of a schedule for the whole process (publication of FYPs topics, selection of applying students and their enrolment, assignation of a jury to each FYP, elaboration and follow-up of FYPs, final report submission, oral presentation, etc.); ii) design of rubrics for each of three assessment parts: working process, final report and oral presentation; and iii) follow-up and evaluation of the involved FYPs. Finally, problems that appeared during this experience (e.g. administrative aspects, criticisms and suggestions from the students, tutors and juries involved) are discussed and some modifications in the assessment system will be proposed in order to solve or minimize these problems.

The Innovation Sources in Software Engineering: A Systematic Literature Review

During the last century many researches on the business, marketing and technology fields have developed the innovation research line and large amount of knowledge can be found in the literature. Currently, the importance of systematic and openness approaches to manage the available innovation sources is well established in many knowledge fields. Also in the software engineering sector, where the organizations need to absorb and to exploit as much innovative ideas as possible to get success in the current competitive environment. This Master Thesis presents an study related with the innovation sources in the software engineering eld. The main research goals of this work are the identication and the relevance assessment of the available innovation sources and the understanding of the trends on the innovation sources usage. Firstly, a general review of the literature have been conducted in order to define the research area and to identify research gaps. Secondly, the Systematic Literature Review (SLR) has been proposed as the research method in this work to report reliable conclusions collecting systematically quality evidences about the innovation sources in software engineering field. This contribution provides resources, built-on empirical studies included in the SLR, to support a systematic identication and an adequate exploitation of the innovation sources most suitable in the software engineering field. Several artefacts such as lists, taxonomies and relevance assessments of the innovation sources most suitable for software engineering have been built, and their usage trends in the last decades and their particularities on some countries and knowledge fields, especially on the software engineering, have been researched. This work can facilitate to researchers, managers and practitioners of innovative software organizations the systematization of critical activities on innovation processes like the identication and exploitation of the most suitable opportunities. Innovation researchers can use the results of this work to conduct research studies involving the innovation sources research area. Whereas, organization managers and software practitioners can use the provided outcomes in a systematic way to improve their innovation capability, increasing consequently the value creation in the processes that they run to provide products and services useful to their environment. In summary, this Master Thesis research the innovation sources in the software engineering field, providing useful resources to support an effective innovation sources management. Moreover, several aspects should be deeply study to increase the accuracy of the presented results and to obtain more resources built-on empirical knowledge. It can be supported by the INno- vation SOurces MAnagement (InSoMa) framework, which is introduced in this work in order to encourage openness and systematic approaches to identify and to exploit the innovation sources in the software engineering field.

Caracterización de Sistemas intensivos en Software desde un punto de vista de innovación=Characterizing Software-intensive Systems from the innovation point of view

La innovación en Sistemas Intesivos en Software está alcanzando relevancia por múltiples razones: el software está presente en sectores como automóvil, teléfonos móviles o salud. Las empresas necesitan conocer aquellos factores que afectan a la innovación para incrementar las probabilidades de éxito en el desarrollo de sus productos y, la evaluación de productos sofware es un mecanismo potente para capturar este conocimiento. En consecuencia, las empresas necesitan evaluar sus productos desde la perpectiva de innovación para reducir la distancia entre los productos desarrollados y el mercado. Esto es incluso más relevante en el caso de los productos intensivos en software, donde el tiempo real, la oportunidad, complejidad, interoperabilidad, capacidad de respuesta y compartción de recursos son características críticas de los nuevos sistemas. La evaluación de la innovación de productos ya ha sido estudiada y se han definido algunos esquemas de evaluación pero no son específicos para Sistemas intensivos en Sofwtare; además, no se ha alcanzado consenso en los factores ni el procedimiento de evaluación. Por lo tanto, tiene sentido trabajar en la definición de un marco de evaluación de innovación enfocado a Sistemas intesivos en Software. Esta tesis identifica los elementos necesarios para construir in marco para la evaluación de de Sistemas intensivos en Software desde el punto de vista de la innovación. Se han identificado dos componentes como partes del marco de evaluación: un modelo de referencia y una herramienta adaptativa y personalizable para la realización de la evaluación y posicionamiento de la innovación. El modelo de referencia está compuesto por cuatro elementos principales que caracterizan la evaluación de innovación de productos: los conceptos, modelos de innovación, cuestionarios de evaluación y la evaluación de productos. El modelo de referencia aporta las bases para definir instancias de los modelos de evaluación de innovación de productos que pueden se evaluados y posicionados en la herramienta a través de cuestionarios y que de forma automatizada aporta los resultados de la evaluación y el posicionamiento respecto a la innovación de producto. El modelo de referencia ha sido rigurosamente construido aplicando modelado conceptual e integración de vistas junto con la aplicación de métodos cualitativos de investigación. La herramienta ha sido utilizada para evaluar productos como Skype a través de la instanciación del modelo de referencia. ABSTRACT Innovation in Software intensive Systems is becoming relevant for several reasons: software is present embedded in many sectors like automotive, robotics, mobile phones or heath care. Firms need to have knowledge about factors affecting the innovation to increase the probability of success in their product development and the assessment of innovation in software products is a powerful mechanism to capture this knowledge. Therefore, companies need to assess products from an innovation perspective to reduce the gap between their developed products and the market. This is even more relevant in the case of SiSs, where real time, timeliness, complexity, interoperability, reactivity, and resource sharing are critical features of a new system. Many authors have analysed product innovation assessment and some schemas have been developed but they are not specific to SiSs; in addition, there is no consensus about the factors or the procedures for performing an assessment. Therefore, it has sense to work in the definition of a customized software product innovation evaluation framework. This thesis identifies the elements needed to build a framework to assess software products from the innovation perspective. Two components have been identified as part of the framework to assess Software intensive Systems from the innovation perspective: a reference-model and an adaptive and customizable tool to perform the assessment and to position product innovation. The reference-model is composed by four main elements characterizing product innovation assessment: concepts, innovation models, assessment questionnaires and product assessment. The reference model provides the umbrella to define instances of product innovation assessment models that can be assessed and positioned through questionnaires in the proposed tool that also provides automation in the assessment and positioning of innovation. The reference-model has been rigorously built by applying conceptual modelling and view integration integrated with qualitative research methods. The tool has been used to assess products like Skype through models instantiated from the reference-model.

A language exchange program: sustainability innovation in language and culture engagement

Spanish Educational Laws have been promoting the widespread use of English; as a result, Spanish Uni versities are looking for ways to give students more international training in order to prepare them for a future that will increasingly involve global problems and partnerships. Therefore, the Polytechnic University of Madrid, Spain (UPM), and the University of British Columbia, Okanagan, Canada (UBCO) have come together to offer opportunities for international collaboration and learning, thus facilitating virtual encounters among Spanish and Canadian students. The Language Exchange Program between the UPM and UBCO acts as a model for sustainability innovation in language and culture engagement as the students can interact with native speakers in communication tasks. This interdisciplinary initiative supports the latest methodological principles observed in the Common European Framework for Languages, such as autonomous and life-long learning, self-assessment and peer-assessment as well as the incorporation of new technologies to the learning process. Additionally the ‘virtual’ mobility is provided at no extra cost. This article presents the preliminary results of two virtual exchange programs that have been offering varied forms of study which are venue-independent, and have clearly expanded the range of scenarios for the students on both sides by promoting collaborative work and cultural exchange.

A language exchange program: sustainability innovation in language and culture engagement

The Language Exchange Program between the UPM and UBCO acts as a model for sustainability innovation in language and culture engagement as the students can interact with native speakers in communication tasks. This interdisciplinary initiative supports the latest methodological principles observed in the Common European Framework for Languages [1], such as autonomous and lifelong learning, self-assessment and peer-assessment as well as the incorporation of new technologies to the learning process

Novel educational assessment for building structures: Automatic evaluation of on-line graphics

This article presents a new automatic evaluation for on-line graphics, its application and the numerous advantages achieved applying this developed correcting method. The software application developed by the Innovation in Education Group “E4”, from the Technical University of Madrid, is oriented for the online self-assessment of the graphic drawings that students carry out as continuous training. The adaptation to the European Higher Educational Area is an important opportunity to research about the possibilities of on-line education assessment. In this way, a new software tool has been developed for continuous self-testing by undergraduates. Using this software it is possible to evaluate the graphical answer of the students. Thus, the drawings made on-line by students are automatically corrected according to the geometry (straight lines, sloping lines or second order curves) and by sizes (depending on the specific values which define the graphics).

A tool assisting teachers in automating the assessment of programming assignments

Automating the assessment of programming assignments brings benefits for both students and teachers, since it helps the formers to gain a timely feedback and releases the latter from tedious tasks. The related literature in the domain has usually focused on the assessment process and the tools required for it, proposing libraries and systems that teachers can use in this process. However, few of them have work rowards reducing the effort and time teacher require to properly set up new assessente processes. This paper describes our experience with the analysis and design of a new tool to support teachers in visually developing automatic grades of programming assignments, introducing the underlying concepts and technologies and presenting the system architecture.

A multidisciplinary reference framework to support innovative design, implementation and assessment of chronic care models

Hoy en día, por primera vez en la historia, la mayor parte de la población podrá vivir hasta los sesenta años y más (United Nations, 2015). Sin embargo, todavía existe poca evidencia que demuestre que las personas mayores, estén viviendo con mejor salud que sus padres, a la misma edad, ya que la mayoría de los problemas de salud en edades avanzadas están asociados a las enfermedades crónicas (WHO, 2015). Los sistemas sanitarios de los países desarrollados funcionan adecuadamente cuando se trata del cuidado de enfermedades agudas, pero no son lo suficientemente eficaces en la gestión de las enfermedades crónicas. Durante la última década, se han realizado esfuerzos para mejorar esta gestión, por medio de la utilización de estrategias de prevención y de reenfoque de la provisión de los servicios de atención para la salud (Kane et al. 2005). Según una revisión sistemática de modelos de cuidado de salud, comisionada por el sistema nacional de salud Británico, pocos modelos han conceptualizado cuáles son los componentes que hay que utilizar para proporcionar un cuidado crónico efectivo, y estos componentes no han sido suficientemente estructurados y articulados. Por lo tanto, no hay suficiente evidencia sobre el impacto real de cualquier modelo existente en la actualidad (Ham, 2006). Las innovaciones podrían ayudar a conseguir mejores diagnósticos, tratamientos y gestión de pacientes crónicos, así como a dar soporte a los profesionales y a los pacientes en el cuidado. Sin embargo, la forma en las que estas innovaciones se proporcionan no es lo suficientemente eficiente, efectiva y amigable para el usuario. Para mejorar esto, hace falta crear equipos de trabajo y estrategias multidisciplinares. En conclusión, hacen falta actividades que permitan conseguir que las innovaciones sean utilizadas en los sistemas de salud que quieren mejorar la gestión del cuidado crónico, para que sea posible: 1) traducir la “atención sanitaria basada en la evidencia” en “conocimiento factible”; 2) hacer frente a la complejidad de la atención sanitaria a través de una investigación multidisciplinaria; 3) identificar una aproximación sistemática para que se establezcan intervenciones innovadoras en el cuidado de salud. El marco de referencia desarrollado en este trabajo de investigación es un intento de aportar estas mejoras. Las siguientes hipótesis han sido propuestas: Hipótesis 1: es posible definir un proceso de traducción que convierta un modelo de cuidado crónico en una descripción estructurada de objetivos, requisitos e indicadores clave de rendimiento. Hipótesis 2: el proceso de traducción, si se ejecuta a través de elementos basados en la evidencia, multidisciplinares y de orientación económica, puede convertir un modelo de cuidado crónico en un marco descriptivo, que define el ciclo de vida de soluciones innovadoras para el cuidado de enfermedades crónicas. Hipótesis 3: es posible definir un método para evaluar procesos, resultados y capacidad de desarrollar habilidades, y asistir equipos multidisciplinares en la creación de soluciones innovadoras para el cuidado crónico. Hipótesis 4: es posible dar soporte al desarrollo de soluciones innovadoras para el cuidado crónico a través de un marco de referencia y conseguir efectos positivos, medidos en indicadores clave de rendimiento. Para verificar las hipótesis, se ha definido una aproximación metodológica compuesta de cuatro Fases, cada una asociada a una hipótesis. Antes de esto, se ha llevado a cabo una “Fase 0”, donde se han analizado los antecedentes sobre el problema (i.e. adopción sistemática de la innovación en el cuidado crónico) desde una perspectiva multi-dominio y multi-disciplinar. Durante la fase 1, se ha desarrollado un Proceso de Traducción del Conocimiento, elaborado a partir del JBI Joanna Briggs Institute (JBI) model of evidence-based healthcare (Pearson, 2005), y sobre el cual se han definido cuatro Bloques de Innovación. Estos bloques consisten en una descripción de elementos innovadores, definidos en la fase 0, que han sido añadidos a los cuatros elementos que componen el modelo JBI. El trabajo llevado a cabo en esta fase ha servido también para definir los materiales que el proceso de traducción tiene que ejecutar. La traducción que se ha llevado a cabo en la fase 2, y que traduce la mejor evidencia disponible de cuidado crónico en acción: resultado de este proceso de traducción es la parte descriptiva del marco de referencia, que consiste en una descripción de un modelo de cuidado crónico (se ha elegido el Chronic Care Model, Wagner, 1996) en términos de objetivos, especificaciones e indicadores clave de rendimiento y organizada en tres ciclos de innovación (diseño, implementación y evaluación). Este resultado ha permitido verificar la segunda hipótesis. Durante la fase 3, para demostrar la tercera hipótesis, se ha desarrollado un método-mixto de evaluación de equipos multidisciplinares que trabajan en innovaciones para el cuidado crónico. Este método se ha creado a partir del método mixto usado para la evaluación de equipo multidisciplinares translacionales (Wooden, 2013). El método creado añade una dimensión procedural al marco. El resultado de esta fase consiste, por lo tanto, en una primera versión del marco de referencia, lista para ser experimentada. En la fase 4, se ha validado el marco a través de un caso de estudio multinivel y con técnicas de observación-participante como método de recolección de datos. Como caso de estudio se han elegido las actividades de investigación que el grupo de investigación LifeStech ha desarrollado desde el 2008 para mejorar la gestión de la diabetes, actividades realizadas en un contexto internacional. Los resultados demuestran que el marco ha permitido mejorar las actividades de trabajo en distintos niveles: 1) la calidad y cantidad de las publicaciones; 2) se han conseguido dos contratos de investigación sobre diabetes: el primero es un proyecto de investigación aplicada, el segundo es un proyecto financiado para acelerar las innovaciones en el mercado; 3) a través de los indicadores claves de rendimiento propuestos en el marco, una prueba de concepto de un prototipo desarrollado en un proyecto de investigación ha sido transformada en una evaluación temprana de una intervención eHealth para el manejo de la diabetes, que ha sido recientemente incluida en Repositorio de prácticas innovadoras del Partenariado de Innovación Europeo en Envejecimiento saludable y activo. La verificación de las 4 hipótesis ha permitido demonstrar la hipótesis principal de este trabajo de investigación: es posible contribuir a crear un puente entre la atención sanitaria y la innovación y, por lo tanto, mejorar la manera en que el cuidado crónico sea procurado en los sistemas sanitarios. ABSTRACT Nowadays, for the first time in history, most people can expect to live into their sixties and beyond (United Nations, 2015). However, little evidence suggests that older people are experiencing better health than their parents, and most of the health problems of older age are linked to Chronic Diseases (WHO, 2015). The established health care systems in developed countries are well suited to the treatment of acute diseases but are mostly inadequate for dealing with CDs. Healthcare systems are challenging the burden of chronic diseases by putting more emphasis on the prevention of disease and by looking for new ways to reorient the provision of care (Kane et al., 2005). According to an evidence-based review commissioned by the British NHS Institute, few models have conceptualized effective components of care for CDs and these components have been not structured and articulated. “Consequently, there is limited evidence about the real impact of any of the existing models” (Ham, 2006). Innovations could support to achieve better diagnosis, treatment and management for patients across the continuum of care, by supporting health professionals and empowering patients to take responsibility. However, the way they are delivered is not sufficiently efficient, effective and consumer friendly. The improvement of innovation delivery, involves the creation of multidisciplinary research teams and taskforces, rather than just working teams. There are several actions to improve the adoption of innovations from healthcare systems that are tackling the epidemics of CDs: 1) Translate Evidence-Based Healthcare (EBH) into actionable knowledge; 2) Face the complexity of healthcare through multidisciplinary research; 3) Identify a systematic approach to support effective implementation of healthcare interventions through innovation. The framework proposed in this research work is an attempt to provide these improvements. The following hypotheses have been drafted: Hypothesis 1: it is possible to define a translation process to convert a model of chronic care into a structured description of goals, requirements and key performance indicators. Hypothesis 2: a translation process, if executed through evidence-based, multidisciplinary, holistic and business-oriented elements, can convert a model of chronic care in a descriptive framework, which defines the whole development cycle of innovative solutions for chronic disease management. Hypothesis 3: it is possible to design a method to evaluate processes, outcomes and skill acquisition capacities, and assist multidisciplinary research teams in the creation of innovative solutions for chronic disease management. Hypothesis 4: it is possible to assist the development of innovative solutions for chronic disease management through a reference framework and produce positive effects, measured through key performance indicators. In order to verify the hypotheses, a methodological approach, composed of four Phases that correspond to each one of the stated hypothesis, was defined. Prior to this, a “Phase 0”, consisting in a multi-domain and multi-disciplinary background analysis of the problem (i.e.: systematic adoption of innovation to chronic care), was carried out. During phase 1, in order to verify the first hypothesis, a Knowledge Translation Process (KTP) was developed, starting from the JBI Joanna Briggs Institute (JBI) model of evidence-based healthcare was used (Pearson, 2005) and adding Four Innovation Blocks. These blocks represent an enriched description, added to the JBI model, to accelerate the transformation of evidence-healthcare through innovation; the innovation blocks are built on top of the conclusions drawn after Phase 0. The background analysis gave also indication on the materials and methods to be used for the execution of the KTP, carried out during phase 2, that translates the actual best available evidence for chronic care into action: this resulted in a descriptive Framework, which is a description of a model of chronic care (the Chronic Care Model was chosen, Wagner, 1996) in terms of goals, specified requirements and Key Performance Indicators, and articulated in the three development cycles of innovation (i.e. design, implementation and evaluation). Thanks to this result the second hypothesis was verified. During phase 3, in order to verify the third hypothesis, a mixed-method to evaluate multidisciplinary teams working on innovations for chronic care, was created, based on a mixed-method used for the evaluation of Multidisciplinary Translational Teams (Wooden, 2013). This method adds a procedural dimension to the descriptive component of the Framework, The result of this phase consisted in a draft version of the framework, ready to be tested in a real scenario. During phase 4, a single and multilevel case study, with participant-observation data collection, was carried out, in order to have a complete but at the same time multi-sectorial evaluation of the framework. The activities that the LifeStech research group carried out since 2008 to improve the management of diabetes have been selected as case study. The results achieved showed that the framework allowed to improve the research activities in different directions: the quality and quantity of the research publications that LifeStech has issued, have increased substantially; 2 project grants to improve the management of diabetes, have been assigned: the first is a grant funding applied research while the second is about accelerating innovations into the market; by using the assessment KPIs of the framework, the proof of concept validation of a prototype developed in a research project was transformed into an early stage assessment of innovative eHealth intervention for Diabetes Management, which has been recently included in the repository of innovative practice of the European Innovation Partnership on Active and Health Ageing initiative. The verification of the 4 hypotheses lead to verify the main hypothesis of this research work: it is possible to contribute to bridge the gap between healthcare and innovation and, in turn, improve the way chronic care is delivered by healthcare systems.