Assessment of learning outcomes in computing studies

The assessment of learning outcomes is a key concept in the European Credit Transfer and Accumulation System (ECTS) since credits are awarded when the assessment shows the competences which were aimed at have been developed at an appropriate level. This paper describes a study which was first part of the project of the Bologna Experts Team-Spain and then developed as an independent study. It was carried out with the overall goal to gain experience in the assessment of learning outcomes. More specifically it aimed at 1) designing procedures for the assessment of learning outcomes related to these compulsory generic competences; 2) testing some basic psychometric features that an assessment device with some consequences for the subjects being evaluated needs to prove; 3) testing different procedures of standard setting, and 4) using assessment results as orienting feedback to students and their tutors. The process of development of tests to carry out the assessment of learning outcomes related to these competences, as well as some basic features regarding their reliability and validity is described and first results on the comparison of results achieved at two academic levels, will also be described at a later stage.

Learning Analytics and Interactions in Virtual Learning Environments. A Comparative Study of Typologies and their Relationship with Academic Performance

Analysis of learning data (learning analytics) is a new research field with high growth potential. The main objective of Learning analytics is the analysis of data (interactions being the basic data unit) generated in virtual learning environments, in order to maximize the outcomes of the learning process; however, a consensus has not been reached yet on which interactions must be measured and what is their influence on learning outcomes. This research is grounded on the study of e-learning interaction typologies and their relationship with students? academic performance, by means of a comparative study between different interaction typologies (based on the agents involved, frequency of use and participation mode). The main conclusions are a) that classifications based on agents offer a better explanation of academic performance; and b) that each of the three typologies are able to explain academic performance in terms of some of their components (student-teacher and student-student interactions, evaluating students interactions and active interactions, respectively), with the other components being nonrelevant.

Predicting academic performance with learning analytics in virtual learning environments: A comparative study of three interaction classifications

Learning analytics is the analysis of static and dynamic data extracted from virtual learning environments, in order to understand and optimize the learning process. Generally, this dynamic data is generated by the interactions which take place in the virtual learning environment. At the present time, many implementations for grouping of data have been proposed, but there is no consensus yet on which interactions and groups must be measured and analyzed. There is also no agreement on what is the influence of these interactions, if any, on learning outcomes, academic performance or student success. This study presents three different extant interaction typologies in e-learning and analyzes the relation of their components with students? academic performance. The three different classifications are based on the agents involved in the learning process, the frequency of use and the participation mode, respectively. The main findings from the research are: a) that agent-based classifications offer a better explanation of student academic performance; b) that at least one component in each typology predicts academic performance; and c) that student-teacher and student-student, evaluating students, and active interactions, respectively, have a significant impact on academic performance, while the other interaction types are not significantly related to academic performance.

Plataformas experimentales para la formación en el área de la electrónica en entornos de e-learning

El campo de estudio relacionado con los laboratorios remotos en el ámbito educativo de las ciencias y la ingeniería está sufriendo una notable expansión ante la necesidad de adaptar los procesos de aprendizaje en dichas áreas a las características y posibilidades de la formación online. Muchos de los recursos educativos basados en esta tecnología, existentes en la actualidad, presentan ciertas limitaciones que impiden alcanzar las competencias que se deben adquirir en los laboratorios de ingeniería. Estas limitaciones están relacionadas con diferentes aspectos de carácter técnico y formativo. A nivel técnico las limitaciones principales se centran en el grado de versatilidad que son capaces de proporcionar comparado con el que se dispone en un laboratorio tradicional y en el modo de interacción del usuario, que provoca que el estudiante no distinga claramente si está realizando acciones sobre sistemas reales o simulaciones. A nivel formativo las limitaciones detectadas son relevantes para poder alcanzar un aprendizaje significativo. En concreto están relacionadas principalmente con un escaso sentimiento de inmersión, una reducida sensación de realismo respecto a las operaciones que se realizan o la limitada posibilidad de realizar actividades de forma colaborativa. La aparición de nuevas tecnologías basadas en entornos inmersivos, unida a los avances producidos relacionados con el aumento de la capacidad gráfica de los ordenadores y del ancho de banda de acceso a Internet, han hecho factible que las limitaciones comentadas anteriormente puedan ser superadas gracias al desarrollo de nuevos recursos de aprendizaje surgidos de la fusión de laboratorios remotos y mundos virtuales 3D. Esta tesis doctoral aborda un trabajo de investigación centrado en proponer un modelo de plataformas experimentales, basado en la fusión de las dos tecnologías mencionadas, que permita generar recursos educativos online que faciliten la adquisición de competencias prácticas similares a las que se consiguen en un laboratorio tradicional vinculado a la enseñanza de la electrónica. El campo de aplicación en el que se ha focalizado el trabajo realizado se ha centrado en el área de la electrónica aunque los resultados de la investigación realizada se podrían adaptar fácilmente a otras disciplinas de la ingeniería. Fruto del trabajo realizado en esta tesis es el desarrollo de la plataforma eLab3D, basada en el modelo de plataformas experimentales propuesto, y la realización de dos estudios empíricos llevados a cabo con estudiantes de grado en ingeniería, muy demandados por la comunidad investigadora. Por un lado, la plataforma eLab3D, que permite llevar a cabo de forma remota actividades prácticas relacionadas con el diseño, montaje y prueba de circuitos electrónicos analógicos, aporta como novedad un dispositivo hardware basado en un sistema de conmutación distribuido. Dicho sistema proporciona un nivel de versatilidad muy elevado, a nivel de configuración de circuitos y selección de puntos de medida, que hace posible la realización de acciones similares a las que se llevan a cabo en los laboratorios presenciales. Por otra parte, los estudios empíricos realizados, que comparaban la eficacia educativa de una metodología de aprendizaje online, basada en el uso de la plataforma eLab3D, con la conseguida siguiendo una metodología clásica en los laboratorios tradicionales, mostraron que no se detectaron diferencias significativas en el grado de adquisición de los resultados de aprendizaje entre los estudiantes que utilizaron la plataforma eLab3D y los que asistieron a los laboratorios presenciales. Por último, hay que destacar dos aspectos relevantes relacionados directamente con esta tesis. En primer lugar, los resultados obtenidos en las experiencias educativas llevadas a cabo junto a valoraciones obtenidas por el profesorado que ha colaborado en las mismas han sido decisivos para que la plataforma eLab3D se haya integrado como recurso complementario de aprendizaje en titulaciones de grado de ingeniería de la Universidad Politécnica de Madrid. En segundo lugar, el modelo de plataformas experimentales que se ha propuesto en esta tesis, analizado por investigadores vinculados a proyectos en el ámbito de la fusión nuclear, ha sido tomado como referencia para generar nuevas herramientas de formación en dicho campo. ABSTRACT The field of study of remote laboratories in sciences and engineering educational disciplines is undergoing a remarkable expansion given the need to adapt the learning processes in the aforementioned areas to the characteristics and possibilities of online education. Several of the current educational resources based on this technology have certain limitations that prevent from reaching the required competencies in engineering laboratories. These limitations are related to different aspects of technical and educational nature. At the technical level, they are centered on the degree of versatility they are able to provide compared to a traditional laboratory and in the way the user interacts with them, which causes the student to not clearly distinguish if actions are being performed over real systems or over simulations. At the educational level, the detected limitations are relevant in order to reach a meaningful learning. In particular, they are mainly related to a scarce immersion feeling, a reduced realism sense regarding the operations performed or the limited possibility to carry out activities in a collaborative way. The appearance of new technologies based on immersive environments, together with the advances in graphical computer capabilities and Internet bandwidth access, have made the previous limitations feasible to be overcome thanks to the development of new learning resources that arise from merging remote laboratories and 3D virtual worlds. This PhD thesis tackles a research work focused on the proposal of an experimental platform model, based on the fusion of both mentioned technologies, which allows for generating online educational resources that facilitate the acquisition of practical competencies similar to those obtained in a traditional electronics laboratory. The application field, in which this work is focused, is electronics, although the research results could be easily adapted to other engineering disciplines. A result of this work is the development of eLab3D platform, based on the experimental platform model proposed, and the realization of two empirical studies with undergraduate students, highly demanded by research community. On one side, eLab3D platform, which allows to accomplish remote practical activities related to the design, assembling and test of analog electronic circuits, provides, as an original contribution, a hardware device based on a distributed switching system. This system offers a high level of versatility, both at the circuit configuration level and at the selection of measurement points, which allows for doing similar actions to those conducted in hands-on laboratories. On the other side, the empirical studies carried out, which compare the educational efficiency of an online learning methodology based on the use of eLab3D platform with that obtained following a classical methodology in traditional laboratories, shows that no significant differences in the acquired degree of learning outcomes among the students that used eLab3D platform and those that attended hands-on laboratories were detected. Finally, it is important to highlight two relevant aspects directly related with this thesis work. First of all, the results obtained in the educational experiences conducted, along with the assessment from the faculty that has collaborated in them, have been decisive to integrate eLab3D platform as a supplementary learning resource in engineering degrees at Universidad Politecnica de Madrid. Secondly, the experimental platform model originally proposed in this thesis, which has been analysed by nuclear fusion researchers, has been taken as a reference to generate new educational tools in that field.

Aprendizaje integrado en arquitectura con modelos virtuales : implementación de metodología BIM en la docencia universitaria

Los procesos de diseño y construcción en Arquitectura han mostrado un desarrollo de optimización históricamente muy deficiente cuando se compara con las restantes actividades típicamente industriales. La aspiración constante a una industrialización efectiva, tanto en aras de alcanzar mayores cotas de calidad así como de ahorro de recursos, recibe hoy una oportunidad inmejorable desde el ámbito informático: el Building Information Modelling o BIM. Lo que en un inicio puede parecer meramente un determinado tipo de programa informático, en realidad supone un concepto de “proceso” que subvierte muchas rutinas hoy habituales en el desarrollo de proyectos y construcciones arquitectónicas. La inclusión y desarrollo de datos ligados al proyecto, desde su inicio hasta el fin de su ciclo de vida, conlleva la oportunidad de crear una realidad virtual dinámica y actualizable, que por añadidura posibilita su ensayo y optimización en todos sus aspectos: antes y durante su ejecución, así como vida útil. A ello se suma la oportunidad de transmitir eficientemente los datos completos de proyecto, sin apenas pérdidas o reelaboración, a la cadena de fabricación, lo que facilita el paso a una industrialización verdaderamente significativa en edificación. Ante una llamada mundial a la optimización de recursos y el interés indudable de aumentar beneficios económicos por medio de la reducción del factor de incertidumbre de los procesos, BIM supone un opción de mejora indudable, y así ha sido reconocido a través de la inminente implantación obligatoria por parte de los gobiernos (p. ej. Gran Bretaña en 2016 y España en 2018). La modificación de procesos y roles profesionales que conlleva la incorporación de BIM resulta muy significativa y marcará el ejercicio profesional de los futuros graduados en las disciplinas de Arquitectura, Ingeniería y Construcción (AEC por sus siglas en inglés). La universidad debe responder ágilmente a estas nuevas necesidades incorporando esta metodología en la enseñanza reglada y aportando una visión sinérgica que permita extraer los beneficios formativos subyacentes en el propio marco BIM. En este sentido BIM, al aglutinar el conjunto de datos sobre un único modelo virtual, ofrece un potencial singularmente interesante. La realidad tridimensional del modelo, desarrollada y actualizada continuamente, ofrece al estudiante una gestión radicalmente distinta de la representación gráfica, en la que las vistas parciales de secciones y plantas, tan complejas de asimilar en los inicios de la formación universitaria, resultan en una mera petición a posteriori, para ser extraída según necesidad del modelo virtual. El diseño se realiza siempre sobre el propio modelo único, independientemente de la vista de trabajo elegida en cada momento, permaneciendo los datos y sus relaciones constructivas siempre actualizados y plenamente coherentes. Esta descripción condensada de características de BIM preconfiguran gran parte de las beneficios formativos que ofrecen los procesos BIM, en especial, en referencia al desarrollo del diseño integrado y la gestión de la información (incluyendo TIC). Destacan a su vez las facilidades en comprensión visual de elementos arquitectónicos, sistemas técnicos, sus relaciones intrínsecas así como procesos constructivos. A ello se une el desarrollo experimental que la plataforma BIM ofrece a través de sus software colaborativos: la simulación del comportamiento estructural, energético, económico, entre otros muchos, del modelo virtual en base a los datos inherentes del proyecto. En la presente tesis se describe un estudio de conjunto para explicitar tanto las cualidades como posibles reservas en el uso de procesos BIM, en el marco de una disciplina concreta: la docencia de la Arquitectura. Para ello se ha realizado una revisión bibliográfica general sobre BIM y específica sobre docencia en Arquitectura, así como analizado las experiencias de distintos grupos de interés en el marco concreto de la enseñanza de la en Arquitectura en la Universidad Europea de Madrid. El análisis de beneficios o reservas respecto al uso de BIM se ha enfocado a través de la encuesta a estudiantes y la entrevista a profesionales AEC relacionados o no con BIM. Las conclusiones del estudio permiten sintetizar una implantación de metodología BIM que para mayor claridad y facilidad de comunicación y manejo, se ha volcado en un Marco de Implantación eminentemente gráfico. En él se orienta sobre las acciones docentes para el desarrollo de competencias concretas, valiéndose de la flexibilidad conceptual de los Planes de Estudio en el contexto del Espacio Europeo de Educación Superior (Declaración de Bolonia) para incorporar con naturalidad la nueva herramienta docente al servicio de los objetivos formativo legalmente establecidos. El enfoque global del Marco de Implementación propuesto facilita la planificación de acciones formativas con perspectiva de conjunto: combinar los formatos puntuales o vehiculares BIM, establecer sinergias transversales y armonizar recursos, de modo que la metodología pueda beneficiar tanto la asimilación de conocimientos y habilidades establecidas para el título, como el propio flujo de aprendizaje o learn flow BIM. Del mismo modo reserva, incluso visualmente, aquellas áreas de conocimiento en las que, al menos en la planificación actual, la inclusión de procesos BIM no se considera ventajosa respecto a otras metodologías, o incluso inadecuadas para los objetivos docentes establecidos. Y es esta última categorización la que caracteriza el conjunto de conclusiones de esta investigación, centrada en: 1. la incuestionable necesidad de formar en conceptos y procesos BIM desde etapas muy iniciales de la formación universitaria en Arquitectura, 2. los beneficios formativos adicionales que aporta BIM en el desarrollo de competencias muy diversas contempladas en el currículum académico y 3. la especificidad del rol profesional del arquitecto que exigirá una implantación cuidadosa y ponderada de BIM que respete las metodologías de desarrollo creativo tradicionalmente efectivas, y aporte valor en una reorientación simbiótica con el diseño paramétrico y fabricación digital que permita un diseño finalmente generativo. ABSTRACT The traditional architectural design and construction procedures have proven to be deficient where process optimization is concerned, particularly when compared to other common industrial activities. The ever‐growing strife to achieve effective industrialization, both in favor of reaching greater quality levels as well as sustainable management of resources, has a better chance today than ever through a mean out of the realm of information technology, the Building Information Modelling o BIM. What may initially seem to be merely another computer program, in reality turns out to be a “process” concept that subverts many of today’s routines in architectural design and construction. Including and working with project data from the very beginning to the end of its full life cycle allows for creating a dynamic and updatable virtual reality, enabling data testing and optimizing throughout: before and during execution, all the way to the end of its lifespan. In addition, there is an opportunity to transmit complete project data efficiently, with hardly any loss or redeveloping of the manufacture chain required, which facilitates attaining a truly significant industrialization within the construction industry. In the presence of a world‐wide call for optimizing resources, along with an undeniable interest in increasing economic benefits through reducing uncertainty factors in its processes, BIM undoubtedly offers a chance for improvement as acknowledged by its imminent and mandatory implementation on the part of governments (for example United Kingdom in 2016 and Spain in 2018). The changes involved in professional roles and procedures upon incorporating BIM are highly significant and will set the course for future graduates of Architecture, Engineering and Construction disciplines (AEC) within their professions. Higher Education must respond to such needs with swiftness by incorporating this methodology into their educational standards and providing a synergetic vision that focuses on the underlying educational benefits inherent in the BIM framework. In this respect, BIM, in gathering data set under one single virtual model, offers a uniquely interesting potential. The three‐dimensional reality of the model, under continuous development and updating, provides students with a radically different graphic environment, in which partial views of elevation, section or plan that tend characteristically to be difficult to assimilate at the beginning of their studies, become mere post hoc requests to be ordered when needed directly out the virtual model. The design is always carried out on the sole model itself, independently of the working view chosen at any particular moment, with all data and data relations within construction permanently updated and fully coherent. This condensed description of the features of BIM begin to shape an important part of the educational benefits posed by BIM processes, particularly in reference to integrated design development and information management (including ITC). At the same time, it highlights the ease with which visual understanding is achieved regarding architectural elements, technology systems, their intrinsic relationships, and construction processes. In addition to this, there is the experimental development the BIM platform grants through its collaborative software: simulation of structural, energetic, and economic behavior, among others, of the virtual model according to the data inherent to the project. This doctoral dissertation presents a broad study including a wide array of research methods and issues in order to specify both the virtues and possible reservations in the use of BIM processes within the framework of a specific discipline: teaching Architecture. To do so, a literature review on BIM has been carried out, specifically concerning teaching in the discipline of Architecture, as well as an analysis of the experience of different groups of interest delimited to Universidad Europea de Madrid. The analysis of the benefits and/or limitations of using BIM has been approached through student surveys and interviews with professionals from the AEC sector, associated or not, with BIM. Various diverse educational experiences are described and academic management for experimental implementation has been analyzed. The conclusions of this study offer a synthesis for a Framework of Implementation of BIM methodology, which in order to reach greater clarity, communication ease and user‐friendliness, have been posed in an eminently graphic manner. The proposed framework proffers guidance on teaching methods conducive to the development of specific skills, taking advantage of the conceptual flexibility of the European Higher Education Area guidelines based on competencies, which naturally facilitate for the incorporation of this new teaching tool to achieve the educational objectives established by law. The global approach of the Implementation Framework put forth in this study facilitates the planning of educational actions within a common perspective: combining exceptional or vehicular BIM formats, establishing cross‐disciplinary synergies, and sharing resources, so as to purport a methodology that contributes to the assimilation of knowledge and pre‐defined competencies within the degree program, and to the flow of learning itself. At the same time, it reserves, even visually, those areas of knowledge in which the use of BIM processes is not considered necessarily an advantage over other methodologies, or even inadequate for the learning outcomes established, at least where current planning is concerned. It is this last category which characterizes the research conclusions as a whole, centering on: 1. The unquestionable need for teaching BIM concepts and processes in Architecture very early on, in the initial stages of higher education; 2. The additional educational benefits that BIM offers in a varied array of competency development within the academic curriculum; and 3. The specific nature of the professional role of the Architect, which demands a careful and balanced implementation of BIM that respects the traditional teaching methodologies that have proven effective and creative, and adds value by a symbiotic reorientation merged with parametric design and digital manufacturing so to enable for a finally generative design.

La visita a campo como metodología para el proceso de enseñanza-aprendizaje

Dentro de la enseñanza de la geotecnia los viajes a campo son una herramienta útil para superar las limitaciones asociadas a la enseñanza en el aula así como para promover el autoaprendizaje del alumno, el cual se enfrenta en primera persona a la información en estado bruto. Mediante esta comunicación compartimos la experiencia de la visita a las obras de construcción de los Túneles de Sorbas y El Almendral dentro del Máster de "Geología Aplicada a la Obra Civil y los Recursos Hídricos" ofertado por la Universidad de Granada, comentando, con un enfoque docente, la planificación de la actividad en función de los resultados de aprendizaje deseados. Fieldtrips are a good tool to overcome the inherent difficulties associated to teaching engineering geology at the classroom and to encourage student self-learning, when they face raw data. In this paper, we share our recent experience with the organization of a fieldtrip to two tunneling construction site (Sorbas Tunnel and El Almendral Tunnel) for the MSc program of “Applied Geology in Civil Engineering and Water Resources” offered by the University of Granada, discussing, with a educational point of view, the planning and learning outcomes.

Learning and Assessing Competencies: New challenges for Mathematics in Engineering Degrees in Spain.

The introduction of new degrees adapted to the European Area of Higher Education (EAHE) has involved a radically different approach to the curriculum. The new programs are structured around competencies that should be acquired. Considering the competencies, teachers must define and develop learning objectives, design teaching methods and establish appropriate evaluation systems. While most Spanish universities have incorporated methodological innovations and evaluation systems different from traditional exams, there is enough confusion about how to teach and assess competencies and learning outcomes, as traditionally the teaching and assessment have focused on knowledge. In this paper we analyze the state-of-the-art in the mathematical courses of the new engineering degrees in some Spanish universities.

Using multimedia and peer assessment to promote collaborative e-learning

Collaborative e-learning is increasingly appealing as a pedagogical approach that can positively affect student learning. We propose a didactical model that integrates multimedia with collaborative tools and peer assessment to foster collaborative e-learning. In this paper, we explain it and present the results of its application to the “International Seminars on Materials Science” online course. The proposed didactical model consists of five educational activities. In the first three, students review the multimedia resources proposed by the teacher in collaboration with their classmates. Then, in the last two activities, they create their own multimedia resources and assess those created by their classmates. These activities foster communication and collaboration among students and their ability to use and create multimedia resources. Our purpose is to encourage the creativity, motivation, and dynamism of the learning process for both teachers and students.

Do Classes in Cooperative Classrooms Have a Positive Influence on Creativity and Teamwork Skills for Engineering Students?

Contributing to the acquisition of professional creativity and teamwork skills has been a special challenge for some of the subjects taught at the Technical University of Madrid (UPM), and this has been a starting point for the work described in this paper. Some professors have intuited that the use of cooperative classrooms could facilitate the acquisition of these skills. We describe the new methodologies applied within cooperative classrooms by some professors, and present the procedure for measuring students’ perception of their own learning outcomes, skill improvements, and overall satisfaction with the use of this kind of classroom. For this project, 250 students enrolled in several subjects answered a questionnaire. The featuresof thesubjectsinvolved intheproject arewidely disparate. We present the results of the statistical analysis with special emphasis on creativity and teamwork skills, and we conclude that the use of cooperative classroom has a positive influence on the acquisition of these skills. This work has the added value of being the first analysis of student perception of the use of cooperative classroom in the acquisition of creativity and teamwork skills.