Assessment of learning outcomes in computing studies

The assessment of learning outcomes is a key concept in the European Credit Transfer and Accumulation System (ECTS) since credits are awarded when the assessment shows the competences which were aimed at have been developed at an appropriate level. This paper describes a study which was first part of the project of the Bologna Experts Team-Spain and then developed as an independent study. It was carried out with the overall goal to gain experience in the assessment of learning outcomes. More specifically it aimed at 1) designing procedures for the assessment of learning outcomes related to these compulsory generic competences; 2) testing some basic psychometric features that an assessment device with some consequences for the subjects being evaluated needs to prove; 3) testing different procedures of standard setting, and 4) using assessment results as orienting feedback to students and their tutors. The process of development of tests to carry out the assessment of learning outcomes related to these competences, as well as some basic features regarding their reliability and validity is described and first results on the comparison of results achieved at two academic levels, will also be described at a later stage.

Peer Assessment and Self-assessment: Effective Learning Tools in Higher Education.

When used appropriately, self- and peer-assessment are very effective learning tools. In the present work, instructor formative assessment and feedback, self-assessment (SA), and peer-assessment (PA) have been compared. During the first part of a semester, the students followed a continuous formative assessment. Subsequently, they were divided into two subgroups based on similar performances. One subgroup performed SAs, and the other followedPAduring the last part of the course. The performances of the two groups in solving problems were compared. Results suggest that PA is a more effective learning tool than SA, and both are more effective than instructor formative assessment. However, a survey that was conducted at the end of the experiment showed higher student confidence in instructor assessment than in PA. The students recognized the usefulness of acting as peer assessors, but believed that SA helped them more than PA.

Assessment of problem solving in computing studies

The assessment of learning outcomes is a key concept in the European Credit Transfer and Accumulation System (ECTS) since credits are awarded when the assessment shows the competences which were aimed at have been developed at an appropriate level. This paper describes a study which was first part of the Bologna Experts Team-Spain project and then developed as an independent study. It was carried out with the overall goal to gain experience in the assessment of learning outcomes. More specifically it aimed at 1) designing procedures for the assessment of learning outcomes related to these compulsory generic competences; 2) testing some basic psychometric features that an assessment device with some consequences for the subjects being evaluated needs to prove; 3) testing different procedures of standard setting, and 4) using assessment results as orienting feedback to students and their tutors. The process of development of tests to carry out the assessment of learning outcomes is described as well as some basic features regarding their reliability and validity. First conclusions on the comparison of the results achieved at two academic levels are also presented.

Evaluación para el aprendizaje en la asignatura sistemas operativos en modalidad online: un estudio cualitativo basado en la taxonomía de Bloom

La asignatura Sistemas Operativos presenta dificultades para su aprendizaje, pero poco se conoce acerca de las mismas, ya que no han sido determinadas ni estudiadas por la literatura. Asimismo, los trabajos existentes sobre la enseñanza y aprendizaje de Sistemas Operativos se limitan a proponer distintos enfoques para impartir la asignatura y en general no evalúan el aprendizaje de los estudiantes para comprobar la eficacia del método propuesto ni usan metodologías de investigación rigurosas. Por otra parte, la impartición de la asignatura Sistemas Operativos en modalidad online ha sido escasamente estudiada y podría tener dificultades adicionales a las de la modalidad presencial, ya que el contexto online impone una serie de restricciones tanto para el profesor como para el estudiante. En la presente tesis se ha llevado a cabo una evaluación formativa en la asignatura Sistemas Operativos, perteneciente al Grado de Ingeniería Informática de una universidad online. El objetivo inicial de la evaluación era descubrir las dificultades de los estudiantes para la comprensión de los conceptos de la asignatura. Posteriormente y, dada la buena aceptación de la evaluación por parte de los estudiantes, se ampliaron los objetivos del trabajo para explorar los efectos de la evaluación realizada sobre el aprendizaje. La evaluación formativa diseñada está basada en la taxonomía revisada de Bloom y sus principales objetivos son: (a) promover el aprendizaje significativo y (b) hacer a los estudiantes conscientes de su proceso de aprendizaje. La metodología de investigación utilizada es el estudio de caso cualitativo y la muestra está constituida por 9 estudiantes del total de 13 matriculados en la asignatura. Los datos cualitativos analizados proceden de las pruebas de evaluación formativa llevadas a cabo por los estudiantes durante la impartición de la asignatura. Los conceptos de sistemas operativos que han resultado más difíciles de comprender en el curso online estudiado han sido las interrupciones y los semáforos. Además, alrededor de estos conceptos se han identificado las dificultades específicas y sus posibles causas. Las dificultades descubiertas acerca de los semáforos corroboran las investigaciones existentes en el área de programación concurrente. El resto de las dificultades identificadas no habían sido determinadas por la literatura existente. En cuanto a los efectos de la evaluación formativa sobre el aprendizaje, la evidencia empírica muestra que ésta ha provocado en los estudiantes una reflexión profunda sobre los conceptos de la asignatura y sobre su propio proceso de aprendizaje. El estudio de caso presentado puede ayudar a los profesores del área de ingeniería a crear evaluaciones formativas en cursos online. La tesis, por tanto, realiza aportaciones relevantes en las áreas de enseñanza y aprendizaje de sistemas operativos, evaluación formativa, metodologías cualitativas y educación online. ABSTRACT Operating Systems is a difficult subject to learn; however little is known about said difficulties, as they have not been studied nor determined by the relevant literature. Existing studies on teaching and learning the subject of operating systems are limited to presenting different approaches for teaching the subject and generally do not evaluate students’ learning to verify the effectiveness of the proposed methods, nor do they use rigorous research methodologies. On the other hand, there are very few studies on teaching operating systems online, which may inherently present more difficulties than the in-person format, since an online context imposes a series of restrictions on both professors and students, such as not having face-to-face interaction for communications. This thesis studies a formative assessment of the subject of operating systems, as part of the Degree in Information Technology Engineering for an online university. The initial objective of this assessment was to determine the students’ difficulties in conceptual comprehension for this subject. Once students had accepted the assessment, the study’s objectives were expanded to include an investigation of the effects of the assessment on learning. The designed formative assessment was based on Revised Bloom’s Taxonomy with the following main objectives: (a) to promote meaningful learning and (b) (b) to make students aware of their learning process. The research methodology involves a qualitative case study with a sample consisting of 9 of the total 13 students registered for this course. The qualitative data analyzed comes from the formative assessment tests taken by these students during the course. The most difficult operating systems concepts for students in the online course were interrupts and semaphores. Additionally, the specific difficulties and their possible causes have been identified. The students’ comprehension difficulties with semaphores corroborate the existing research in the area of concurrent programming. The other identified difficulties were not discussed in the existing literature. Regarding the effects of the formative assessment on learning, the empirical evidence shows that it causes students to reflect carefully on the subject’s concepts as well as their own learning process. The presented case study can help professors in the area of engineering to create formative assessments for online courses. This thesis, therefore, makes relevant contributions to the areas of teaching and learning operating systems, formative assessment, qualitative methodologies, and online education.

Learning and Assessing Competencies: New challenges for Mathematics in Engineering Degrees in Spain.

The introduction of new degrees adapted to the European Area of Higher Education (EAHE) has involved a radically different approach to the curriculum. The new programs are structured around competencies that should be acquired. Considering the competencies, teachers must define and develop learning objectives, design teaching methods and establish appropriate evaluation systems. While most Spanish universities have incorporated methodological innovations and evaluation systems different from traditional exams, there is enough confusion about how to teach and assess competencies and learning outcomes, as traditionally the teaching and assessment have focused on knowledge. In this paper we analyze the state-of-the-art in the mathematical courses of the new engineering degrees in some Spanish universities.

GLMP for automatic assessment of DFS algorithm learning

We describe how to use a Granular Linguistic Model of a Phenomenon (GLMP) to assess e-learning processes. We apply this technique to evaluate algorithm learning using the GRAPHs learning environment.

Statistical Machine Learning for Automatic Assessment of Physical Activity Intensity Using Multi-axial Accelerometry and Heart Rate

This work explores the automatic recognition of physical activity intensity patterns from multi-axial accelerometry and heart rate signals. Data collection was carried out in free-living conditions and in three controlled gymnasium circuits, for a total amount of 179.80 h of data divided into: sedentary situations (65.5%), light-to-moderate activity (17.6%) and vigorous exercise (16.9%). The proposed machine learning algorithms comprise the following steps: time-domain feature definition, standardization and PCA projection, unsupervised clustering (by k-means and GMM) and a HMM to account for long-term temporal trends. Performance was evaluated by 30 runs of a 10-fold cross-validation. Both k-means and GMM-based approaches yielded high overall accuracy (86.97% and 85.03%, respectively) and, given the imbalance of the dataset, meritorious F-measures (up to 77.88%) for non-sedentary cases. Classification errors tended to be concentrated around transients, what constrains their practical impact. Hence, we consider our proposal to be suitable for 24 h-based monitoring of physical activity in ambulatory scenarios and a first step towards intensity-specific energy expenditure estimators

New ICT Literacies and Holistic Approaches for the Effective Learning of Languages in Tertiary Education

Abstract: This paper summarizes the evolution of different subjects of English for Specific Purposes and English for Academic and Professional Purposes. The aim here is to show a continuum of changes that have not started and nished in one subject alone but affect the whole curriculum. After the discussion section where advantages and drawbacks of the changes introduced are analyzed, we arrive at some conclusions regarding this ve year period of development in the approach to the teaching and learning of the specific or academic English language in the Escuela Universitaria de Ingeniería Técnica de Telecomunicación, Universidad Politécnica de Madrid. Resumen: Este trabajo resume la evolución que han experimentado distintas asignaturas de Inglés para Fines Especí cos e Inglés para Fines Académicos y Profesionales. El objetivo principal es mostrar cómo el esfuerzo por mejorar las asignaturas afecta al currículo como un todo y no sólo a cada una de las asignaturas. Tras el análisis de algunas de las ventajas e inconvenientes de los cambios introducidos, se alcanzan algunas conclusiones sobre la evolución que han sufrido este tipo de asignaturas durante los últimos cinco años en la Escuela Universitaria de Ingeniería Técnica de Telecomunicación, Universidad Politécnica de Madrid.

An approach to automatic learning assessment based on the computational theory of perceptions

E-learning systems output a huge quantity of data on a learning process. However, it takes a lot of specialist human resources to manually process these data and generate an assessment report. Additionally, for formative assessment, the report should state the attainment level of the learning goals defined by the instructor. This paper describes the use of the granular linguistic model of a phenomenon (GLMP) to model the assessment of the learning process and implement the automated generation of an assessment report. GLMP is based on fuzzy logic and the computational theory of perceptions. This technique is useful for implementing complex assessment criteria using inference systems based on linguistic rules. Apart from the grade, the model also generates a detailed natural language progress report on the achieved proficiency level, based exclusively on the objective data gathered from correct and incorrect responses. This is illustrated by applying the model to the assessment of Dijkstra’s algorithm learning using a visual simulation-based graph algorithm learning environment, called GRAPHs

Online games for the teaching of mathematics

In this article we present a didactic experience developed by the GIE (Group of Educational Innovation) “Pensamiento Matemático” of the Polytechnics University of Madrid (UPM), in order to bring secondary students and university students closer to Mathematics. It deals with the development of a virtual board game called Mate-trivial. The mechanics of the game is to win points by going around the board which consists of four types of squares identified by colours: “Statistics and Probability”, “Calculus and Analysis”, “Algebra and Geometry” and “Arithmetic and Number Theory ”. When landing on a square, a question of its category is set out: a correct answer wins 200 points, if wrong it loses 100 points, and not answering causes no effect on the points, but all the same, two minutes out of the 20 minutes that each game lasts are lost. For the game to be over it is necessary, before those 20 minutes run out, to reach the central square and succeed in the final task: four chained questions, one of each type, which must be all answered correctly. It is possible to choose between two levels to play: Level 1, for pre-university students and Level 2 for university students. A prototype of the game is available at the website “Aula de Pensamiento Matemático” developed by the GIE: http://innovacioneducativa.upm.es/pensamientomatematico/. This activity lies within a set of didactic actions which the GIE is developing in the framework of the project “Collaborative Strategies between University and Secondary School Education for the teaching and learning of Mathematics: An Application to solve problems while playing”, a transversal project financed by the UPM.

Using multimedia and peer assessment to promote collaborative e-learning

Collaborative e-learning is increasingly appealing as a pedagogical approach that can positively affect student learning. We propose a didactical model that integrates multimedia with collaborative tools and peer assessment to foster collaborative e-learning. In this paper, we explain it and present the results of its application to the “International Seminars on Materials Science” online course. The proposed didactical model consists of five educational activities. In the first three, students review the multimedia resources proposed by the teacher in collaboration with their classmates. Then, in the last two activities, they create their own multimedia resources and assess those created by their classmates. These activities foster communication and collaboration among students and their ability to use and create multimedia resources. Our purpose is to encourage the creativity, motivation, and dynamism of the learning process for both teachers and students.

Changing assessment methods: New rules, new roles

Over the past 20 years,theuse of Computer Algebra Systems(CAS) has helped with the teaching of mathematics inengineer-ingschools. However the traditional use of CAS only in math labs has led to a narrow view by the student: the CAS is an additional work, not included in the learning process. The didactic guidelines of the European Higher Education Area(EHEA) propose a new teaching–learning model based on competencies. We suggest the use of the CAS be adapted to the new rules. In this paper,we present a model for the integrated use of the CAS,and we describe and analyze two experiments carried out in the academic year2011–2012. Our analysis suggests that the use of CAS in all learning and assessment activities has the potential to positively influence the development of competencies.

Large-scale assessment of language proficiency: theoretical and pedagogical reflections on the use of multiple-choice tests

The new requirement placed on students in tertiary settings in Spain to demonstrate a B1 or a B2 proficiency level of English, in accordance with the Common European Framework of Reference for Languages (CEFRL), has led most Spanish universities to develop a program of certification or accreditation of the required level. The first part of this paper aims to provide a rationale for the type of test that has been developed at the Universidad Politécnica de Madrid for the accreditation of a B2 level, a multiple choice version, and to describe how it was constructed and validated. Then, in the second part of the paper, the results from its application to 924 students enrolled in different degree courses at a variety of schools and faculties at the university are analyzed based on a final test version item analysis. To conclude, some theoretical as well as practical conclusions about testing grammar that affect the teaching and learning process are drawn. RESUMEN. Las nuevas exigencias sobre niveles de competencia B1 y B2 en inglés según el Marco Común Europeo de Referencia para las Lenguas (MCERL) que se imponen sobre los estudiantes de grado y posgrado han llevado a la mayoría de las universidades españolas a desarrollar programas de acreditación o de certificación de estos niveles. La primera parte de este trabajo trata sobre las razones que fundamentan la elección de un tipo concreto de examen para la acreditación del nivel B2 de lengua inglesa en la Universidad Politécnica de Madrid. Se trata de un test de opción múltiple y en esta parte del trabajo se describe cómo fue diseñado y validado. En la segunda parte, se analizan los resultados de la aplicación del test a gran escala a un total de 924 estudiantes matriculados en varias escuelas y Facultades de la Universidad. Para terminar, se apuntan una serie de conclusiones teóricas y prácticas sobre la evaluación de la gramática y de qué modo influye en los procesos de enseñanza y aprendizaje.