Assessment of learning outcomes in computing studies

The assessment of learning outcomes is a key concept in the European Credit Transfer and Accumulation System (ECTS) since credits are awarded when the assessment shows the competences which were aimed at have been developed at an appropriate level. This paper describes a study which was first part of the project of the Bologna Experts Team-Spain and then developed as an independent study. It was carried out with the overall goal to gain experience in the assessment of learning outcomes. More specifically it aimed at 1) designing procedures for the assessment of learning outcomes related to these compulsory generic competences; 2) testing some basic psychometric features that an assessment device with some consequences for the subjects being evaluated needs to prove; 3) testing different procedures of standard setting, and 4) using assessment results as orienting feedback to students and their tutors. The process of development of tests to carry out the assessment of learning outcomes related to these competences, as well as some basic features regarding their reliability and validity is described and first results on the comparison of results achieved at two academic levels, will also be described at a later stage.

An example of Learning based on competences: Use of Maxima in Linear Algebra for Engineers

This paper analyzes the role of Computer Algebra Systems (CAS) in a model of learning based on competences. The proposal is an e-learning model Linear Algebra course for Engineering, which includes the use of a CAS (Maxima) and focuses on problem solving. A reference model has been taken from the Spanish Open University. The proper use of CAS is defined as an indicator of the generic ompetence: Use of Technology. Additionally, we show that using CAS could help to enhance the following generic competences: Self Learning, Planning and Organization, Communication and Writing, Mathematical and Technical Writing, Information Management and Critical Thinking.

An analysis of the moderating effects of age in the acceptance of learning management systems

El presente estudio analiza las intenciones de los usuarios acerca del uso de sistemas de tele-enseñanza LMS (Learning Management Systems, basándose en un modelo que integra el Modelo de Aceptación Tecnológica (TAM, Technology Acceptance Model, la Teoría del Comportamiento Percibido (TPB, Theory of Planned Behavior) y la Teoría Unificada de la Aceptación y Uso de la Tecnología (UTAUT, Unified Theory of Acceptance and Use of Technology), tomando la edad como variable moderadora. Así, este artículo estudia la influencia de la intención conductual, la actitud hacia el uso, la facilidad de uso percibida, la utilidad percibida, la norma subjetiva y la influencia social en la intención de utilizar sistemas e-learning LMS. Como antecedentes de estos factores de influencia se plantean las características del sistema y del usuario. El resultado de la revisión teórica es un modelo unificado que ha sido validado con datos recogidos de 94 estudiantes a través de un cuestionario en línea. Estos datos han sido analizados utilizando la técnica de mínimos cuadrados parciales, y los principales resultados confirman la relevancia predictiva del modelo para usuarios de entre 26 y 35 años y de entre 36 y 45 años.

Validity and reliability of AMPET Greek versión: a first examination of learning motivation in Greek PE settings

Validity and reliability of AMPET Greek versión: a first examination of learning motivation in Greek PE settings

Una propuesta de modelado del estudiante basada en ontologías y diagnóstico pedagógico-cognitivo no monótono

Los recientes avances tecnológicos han encontrado un potencial campo de explotación en la educación asistida por computador. A finales de los años 90 surgió un nuevo campo de investigación denominado Entornos Virtuales Inteligentes para el Entrenamiento y/o Enseñanza (EVIEs), que combinan dos áreas de gran complejidad: Los Entornos Virtuales (EVs) y los Sistemas de Tutoría Inteligente (STIs). De este modo, los beneficios de los entornos 3D (simulación de entornos de alto riesgo o entornos de difícil uso, etc.) pueden combinarse con aquéllos de un STIs (personalización de materias y presentaciones, adaptación de la estrategia de tutoría a las necesidades del estudiante, etc.) para proporcionar soluciones educativas/de entrenamiento con valores añadidos. El Modelo del Estudiante, núcleo de un SIT, representa el conocimiento y características del estudiante, y refleja el proceso de razonamiento del estudiante. Su complejidad es incluso superior cuando los STIs se aplican a EVs porque las nuevas posibilidades de interacción proporcionadas por estos entornos deben considerarse como nuevos elementos de información clave para el modelado del estudiante, incidiendo en todo el proceso educativo: el camino seguido por el estudiante durante su navegación a través de escenarios 3D; el comportamiento no verbal tal como la dirección de la mirada; nuevos tipos de pistas e instrucciones que el módulo de tutoría puede proporcionar al estudiante; nuevos tipos de preguntas que el estudiante puede formular, etc. Por consiguiente, es necesario que la estructura de los STIs, embebida en el EVIE, se enriquezca con estos aspectos, mientras mantiene una estructura clara, estructurada, y bien definida. La mayoría de las aproximaciones al Modelo del Estudiante en STIs y en IVETs no consideran una taxonomía de posibles conocimientos acerca del estudiante suficientemente completa. Además, la mayoría de ellas sólo tienen validez en ciertos dominios o es difícil su adaptación a diferentes STIs. Para vencer estas limitaciones, hemos propuesto, en el marco de esta tesis doctoral, un nuevo mecanismo de Modelado del Estudiante basado en la Ingeniería Ontológica e inspirado en principios pedagógicos, con un modelo de datos sobre el estudiante amplio y flexible que facilita su adaptación y extensión para diferentes STIs y aplicaciones de aprendizaje, además de un método de diagnóstico con capacidades de razonamiento no monótono. El método de diagnóstico es capaz de inferir el estado de los objetivos de aprendizaje contenidos en el SIT y, a partir de él, el estado de los conocimientos del estudiante durante su proceso de aprendizaje. La aproximación almodelado del estudiante propuesta ha sido implementada e integrada en un agente software (el agente de modelado del estudiante) dentro de una plataforma software existente para el desarrollo de EVIEs denominadaMAEVIF. Esta plataforma ha sido diseñada para ser fácilmente configurable para diferentes aplicaciones de aprendizaje. El modelado del estudiante presentado ha sido implementado e instanciado para dos tipos de entornos de aprendizaje: uno para aprendizaje del uso de interfaces gráficas de usuario en una aplicación software y para un Entorno Virtual para entrenamiento procedimental. Además, se ha desarrollado una metodología para guiar en la aplicación del esta aproximación de modelado del estudiante a cada sistema concreto.---ABSTRACT---Recent technological advances have found a potential field of exploitation in computeraided education. At the end of the 90’s a new research field emerged, the so-called Intelligent Virtual Environments for Training and/or Education (IVETs), which combines two areas of great complexity: Virtual Environments (VE) and Intelligent Tutoring Systems (ITS). In this way, the benefits of 3D environments (simulation of high risk or difficult-to-use environments, etc.) may be combined with those of an ITS (content and presentation customization, adaptation of the tutoring strategy to the student requirements, etc.) in order to provide added value educational/training solutions. The StudentModel, core of an ITS, represents the student’s knowledge and characteristics, and reflects the student’s reasoning process. Its complexity is even higher when the ITSs are applied on VEs because the new interaction possibilities offered by these environments must be considered as new key information pieces for student modelling, impacting all the educational process: the path followed by the student during their navigation through 3D scenarios; non-verbal behavior such as gaze direction; new types of hints or instructions that the tutoring module can provide to the student; new question types that the student can ask, etc. Thus, it is necessary for the ITS structure, which is embedded in the IVET, to be enriched by these aspects, while keeping a clear, structured and well defined architecture. Most approaches to SM on ITSs and IVETs don’t consider a complete enough taxonomy of possible knowledge about the student. In addition, most of them have validity only in certain domains or they are hard to be adapted for different ITSs. In order to overcome these limitations, we have proposed, in the framework of this doctoral research project, a newStudentModeling mechanism that is based onOntological Engineering and inspired on pedagogical principles, with a wide and flexible data model about the student that facilitates its adaptation and extension to different ITSs and learning applications, as well as a rich diagnosis method with non-monotonic reasoning capacities. The diagnosis method is able to infer the state of the learning objectives encompassed by the ITS and, fromit, the student’s knowledge state during the student’s process of learning. The proposed student modelling approach has been implemented and integrated in a software agent (the student modeling agent) within an existing software platform for the development of IVETs called MAEVIF. This platform was designed to be easily configurable for different learning applications. The proposed student modeling has been implemented and it has been instantiated for two types of learning environments: one for learning to use the graphical user interface of a software application and a Virtual Environment for procedural training. In addition, a methodology to guide on the application of this student modeling approach to each specific system has been developed.

Development of Interactive Materials by Engineering Students Using the Descartes Applets.

In a degree course such as Forestry Engineering, the general teaching objectives consist of explaining and helping students to understand the principles of Mechanics. For some time now we have encountered significant difficulties in teaching this subject due to the students' lack of motivation and to their insufficient prior preparation for the topic. If we add to this the discipline's inherent complexity and the students' preconceptions about the subject, these teaching difficulties become considerable. For this reason a series of didactic activities have been introduced sequentially in the teaching of this subject. This work describes the methodology, procedure and results for the action of developing a work project in groups using Descartes software. The results of this experiment can be considered very positive. Some of the critical preconceptions for learning the subject can be corrected, and the tutoring process in the classroom contributes to an improvement in teacherstudent communication. Since this scheme was established, the number of students taking part each academic year has increased, and this is the group with the greatest percentage of passing scores.

Assessment of work performed by paired students: a pedagogical experience

“Teamwork” is one of the abilities most valued by employers. In [16] we describe the process of adapting to the ECTS methodologies (for ongoing assessment), a course in computer programming for students in a technical degree (Marine Engineering, UPM) not specifically dedicated to computing. As a further step in this process we have emphasized cooperative learning. For this, the students were paired and the work of each pair was evaluated via surprise tests taken and graded jointly, and constituting a substantial part of the final grade. Here we document this experience, discussing methodological aspects, describing indicators for measuring the impact of these methodologies on the educational experience, and reporting on the students’ opinion of it.

Web-based education in Spanish Universities. A Comparison of Open Source E-Learning Platforms

Web-based education or „e-learning‟ has become a critical component in higher education for the last decade, replacing other distance learning methods, such as traditional computer training or correspondence learning. The number of university students who take on-line courses is continuously increasing all over the world. In Spain, nearly a 90% of the universities have an institutional e-learning platform and over 60% of the traditional on-site courses use this technology as a supplement to the traditional face-to-face classes. This new form of learning allows the disappearance of geographical barriers and enables students to schedule their own learning process, among some other advantages. On-line education is developed through specific software called „e-learning platform‟ or „virtual learning environment‟ (VLE). A considerable number of web-based tools to deliver distance courses are currently available. Open source software packages such as Moodle, Sakai, dotLRN or Dokeos are the most commonly used in the virtual campuses of Spanish universities. This paper analyzes the possibilities that virtual learning environments provide university teachers and learners and offers a technical comparison among some of the most popular e-learning learning platforms.

Learning Analytics and Interactions in Virtual Learning Environments. A Comparative Study of Typologies and their Relationship with Academic Performance

Analysis of learning data (learning analytics) is a new research field with high growth potential. The main objective of Learning analytics is the analysis of data (interactions being the basic data unit) generated in virtual learning environments, in order to maximize the outcomes of the learning process; however, a consensus has not been reached yet on which interactions must be measured and what is their influence on learning outcomes. This research is grounded on the study of e-learning interaction typologies and their relationship with students? academic performance, by means of a comparative study between different interaction typologies (based on the agents involved, frequency of use and participation mode). The main conclusions are a) that classifications based on agents offer a better explanation of academic performance; and b) that each of the three typologies are able to explain academic performance in terms of some of their components (student-teacher and student-student interactions, evaluating students interactions and active interactions, respectively), with the other components being nonrelevant.

Diseño y desarrollo de un sistema basado en Learning Analytics para evaluar la competencia de trabajo en equipo.Design and development of a Learning Analytics System to evaluate group work competence

La adquisición de la competencia grupal es algo básico en la docencia universitaria. Esta tarea va a suponer evaluar diferentes factores en un número elevado de alumnos, lo que puede supone gran complejidad y un esfuerzo elevado. De cara a evitar este esfuerzo se puede pensar en emplear los registros de la interacción de los usuarios almacenados en las plataformas de aprendizaje. Para ello el presente trabajo se basa en el desarrollo de un sistema de Learning Analytics que es utilizado como herramienta para analizar las evidencias individuales de los distintos miembros de un equipo de trabajo. El trabajo desarrolla un modelo teórico apoyado en la herramienta, que permite relacionar las evidencias observadas de forma empírica para cada alumno, con indicadores obtenidos tanto de la acción individual como cooperativo de los miembros de un equipo realizadas a través de los foros de trabajo. Abstract — The development of the group work competence is something basic in university teaching. It should be evaluated, but this means to analyze different issues about the participation of a high number of students which is very complex and implies a lot of effort. In order to facilitate this evaluation it is possible to analyze the logs of students’ interaction in Learning Management Systems. The present work describes the development of a Learning Analytics system that analyzes the interaction of each of the members of working group. This tool is supported by a theoretical model, which allows establishing links between the empirical evidences of each student and the indicators of their action in working forums.

An interactive web-based learning assistant for structural analysis of continuous beams

There are significant levels of concern about the relevance and the difficulty of learning some issues on Strength of Materials and Structural Analysis. Most students of Continuum Mechanics and Structural Analysis in Civil Engineering usually point out some key learning aspects as especially difficult for acquiring specific skills. These key concepts entail comprehension difficulties but ease access and applicability to structural analysis in more advanced subjects. Likewise, some elusive but basic structural concepts, such as flexibility, stiffness or influence lines, are paramount for developing further skills required for advanced structural design: tall buildings, arch-type structures as well as bridges. As new curricular itineraries are currently being implemented, it appears appropriate to devise a repository of interactive web-based applications for training in those basic concepts. That will hopefully train the student to understand the complexity of such concepts, to develop intuitive knowledge on actual structural response and to improve their preparation for exams. In this work, a web-based learning assistant system for influence lines on continuous beams is presented. It consists of a collection of interactive user-friendly applications accessible via Web. It is performed in both Spanish and English languages. Rather than a “black box” system, the procedure involves open interaction with the student, who can simulate and virtually envisage the structural response. Thus, the student is enabled to set the geometric, topologic and mechanic layout of a continuous beam and to change or shift the loading and the support conditions. Simultaneously, the changes in the beam response prompt on the screen, so that the effects of the several issues involved in structural analysis become apparent. The system is performed through a set of web pages which encompasses interactive exercises and problems, written in JavaScript under JQuery and DyGraphs frameworks, given that their efficiency and graphic capabilities are renowned. Students can freely boost their self-study on this subject in order to face their exams more confidently. Besides, this collection is expected to be added to the "Virtual Lab of Continuum Mechanics" of the UPM, launched in 2013 (http://serviciosgate.upm.es/laboratoriosvirtuales/laboratorios/medios-continuos-en-construcci%C3%B3n)