A model for evaluation of generic competences in engineering: application to the problem-solving competence at UPM.

The competence evaluation promoted by the European High Education Area entails a very important methodological change that requires guiding support to help teachers carry out this new and complex task. In this regard, the Technical University of Madrid (UPM, by its Spanish acronym) has financed a series of coordinated projects with a two-fold objective: a) To develop a model for teaching and evaluating core competences that is useful and easily applicable to its different degrees, and b) to provide support to teachers by creating an area within the Website for Educational Innovation where they can search for information on the model corresponding to each core competence approved by UPM. Information available on each competence includes its definition, the formulation of indicators providing evidence on the level of acquisition, the recommended teaching and evaluation methodology, examples of evaluation rules for the different levels of competence acquisition, and descriptions of best practices. These best practices correspond to pilot tests applied to several of the academic subjects conducted at UPM in order to validate the model. This work describes the general procedure that was used and presents the model developed specifically for the problem-solving competence. Some of the pilot experiences are also summarised and their results analysed

Final year project: students and instructors perceptions as a competence-strengthening tool for engineering students

This work indicates the importance of the Final Year Project (FYP) in the strengthening of competences of engineering students. The study also shows which personal competences of students are reinforced most during the FYP process,including the preparation, elaboration, presentation and defence stages. In order to gather information on this subject, a survey was conducted at two different Spanish technical universities—one public and one private—and a comparative analysis was performed of the questionnaires collected. The competence model considered is that used by the Accreditation Board for Engineering and Technology (ABET), since the official title of the public university has been accredited by this model. The results indicate which personal and professional competences of students are reinforced well by undertaking the FYP. Any significant differences in response by university are explained in the study. For validation purposes, the results were contrasted with the instructor’s perspective using the triangulation methodology. Finally, the conclusions drawn will permit the design of new study plans to cope more effectively with the challenges of the FYP in the new Bologna framework.

Analysing written production competence descriptors for academic and professional purposes and their calibration to the CEFR

The Common European Framework of Reference for Languages (CEFR) "describes in a comprehensive way what language learners have to learn to do in order to use a language for communication and what knowledge and skills they have to develop so as to be able to act effectively" (Council of Europe, 2001: 1). This paper reports on the findings of two studies whose purpose was to assess written production competence descriptors meant for their inclusion into the Academic and Professional English Language Portfolio KELP) for students of engineering and architecture. The main objective of these studies was to establish whether the language competence descriptors were a satisfactory valid tool in their language programmes from the point of view of clarity, relevance and reliability, as perceived by the students and fellow English for Academic Purposes (RAP) / English for Science and Technology (EST) instructors. The studies shed light on how to improve unsatisfactory descriptors. Results show that the final descriptor lists were on the whole well calibrated and fairly well written: the great majority was considered valid for both teachers and students involved.

Identifying the core competence as a key requirement for business model innovation. The case of Airlines as a service industry

Core competencies form the basis of an organization’s skills and the basic element of a successful strategic execution. Identifying and strengthening the core competencies enhances flexibility thereby strategically positioning a firm for responding to competition in the dynamic marketplace and can be the difference in quality among firms that follow the same business model. A correct understanding of the concept of business models, employing the right core competencies, organizing them effectively and building the business model around the competencies that are constantly gained and assimilated can result in enhanced business performance and thus having implications for firms that want to innovate their business models. Flexibility can be the firm’s agility to shift focus in response to external factors such as changing markets, new technologies or competition and a firm’s success can be gauged by the ability it displays in this transition. Although industry transformations generally emanate from technological changes, recent examples suggests they may also be due to the introduction of new business models and nowhere is it more relevant than in the airline industry. An analysis of the business model flexibility of 17 Airlines from Asia, Europe and Oceania, that is done with core competence as the indicator reveals a picture of inconsistencies in the core competence strategy of certain airlines and the corresponding reduction in business performance. The performance variations are explained from a service oriented core competence strategy employed by airlines that ultimately enables them in having a flexible business model that not only increases business performance but also helps in reducing the uncertainties in the internal and external operating environments. This is more relevant in the case of airline industry, as the product (the air transportation of passengers) minus the service competence is all the same.

Overview of international research in generic competence assessment

The competency assessment is a key issue for improving the quality of teaching and learning within the current European Higher Education Area (EHEA). The aim of this paper is to review the current research on assessment of generic competences, especially through online tools. It has conducted a search of the Web of Knowledge (Thomson Reuters) from keywords. It have been reviewed the abstracts and the results have been classified by time periods, countries and research area. It has selected a set of articles of the period 2010?2012, in which we have analyzed future trends. It is concluded that the research of assessment generic competences is been developing nowadays in educational area, although is still more important in the professional one. Additionally, it is surprising that appears most often used in preuniversity education levels. The academic context has increased research activities over the past five years, with different developments in the Anglo?Saxon countries over that those countries attached to the Bologna Process. The latest reports indicate that the learning competences must meet the individual reality of each person, so that the use of ICTs in their development and evaluation are essential given its immediacy and motivational ability. There is a clear trend towards an evaluation model that includes a 360º specific and generic competences analysis.

Adaptive neuro-fuzzy inference systems (ANFIS) for supervised classification of competence in MIS based on psychomotor skills

INTRODUCTION: Objective assessment of motor skills has become an important challenge in minimally invasive surgery (MIS) training.Currently, there is no gold standard defining and determining the residents' surgical competence.To aid in the decision process, we analyze the validity of a supervised classifier to determine the degree of MIS competence based on assessment of psychomotor skills METHODOLOGY: The ANFIS is trained to classify performance in a box trainer peg transfer task performed by two groups (expert/non expert). There were 42 participants included in the study: the non-expert group consisted of 16 medical students and 8 residents (< 10 MIS procedures performed), whereas the expert group consisted of 14 residents (> 10 MIS procedures performed) and 4 experienced surgeons. Instrument movements were captured by means of the Endoscopic Video Analysis (EVA) tracking system. Nine motion analysis parameters (MAPs) were analyzed, including time, path length, depth, average speed, average acceleration, economy of area, economy of volume, idle time and motion smoothness. Data reduction was performed by means of principal component analysis, and then used to train the ANFIS net. Performance was measured by leave one out cross validation. RESULTS: The ANFIS presented an accuracy of 80.95%, where 13 experts and 21 non-experts were correctly classified. Total root mean square error was 0.88, while the area under the classifiers' ROC curve (AUC) was measured at 0.81. DISCUSSION: We have shown the usefulness of ANFIS for classification of MIS competence in a simple box trainer exercise. The main advantage of using ANFIS resides in its continuous output, which allows fine discrimination of surgical competence. There are, however, challenges that must be taken into account when considering use of ANFIS (e.g. training time, architecture modeling). Despite this, we have shown discriminative power of ANFIS for a low-difficulty box trainer task, regardless of the individual significances between MAPs. Future studies are required to confirm the findings, inclusion of new tasks, conditions and sample population.

Monitoring the communication competence in an innovative context of project management learning

This document presents an innovative, formal educational initiative that is aimed at enhancing the development of engineering students' specific competences. The subject of project management is the common theoretical and practical framework that articulates an experience that is carried out by multidisciplinary groups. Full utilization of Web 2.0 platforms and Project Based Learning constitutes the applied methodology. More specifically, this study focuses on monitoring communication competence when working in virtual environments, providing an ad-hoc rubric as a final result.

Competence assessment of final year projects for undergraduate telecommunication students

The value of project-based learning has lead to the inclusion of project development activities in engineering courses, being the Final Year Project (FYP) the most remarkable one. Several approaches have been proposed for assessing and grading FYPs but, among them, rubrics are becoming a standard for such type of assessment. However, due to the different characteristics and orientations of the projects (some are more practically oriented, some more theoretically), and the high amount of different competences to be evaluated (knowledge, working capability, communication skills, etc.), the definition of one unique rubric suitable for the evaluation of all FYPs presented in different degree programs, is a big challenge. In a former work, the educational outcomes expected from the FYP were defined and resulted in a proposal for their assessment. Afterwards, the proposal has been tested during one year within an educational innovation-project at the Universidad Politécnica de Madrid which involved the follow-up of 8 undergraduate telecommunication students elaborating their FYPs. In this publication, our experience will be described, based on the emerging work taking place through the formalisation of the process which consisted in the following steps: i) establishment of a schedule for the whole process (publication of FYPs topics, selection of applying students and their enrolment, assignation of a jury to each FYP, elaboration and follow-up of FYPs, final report submission, oral presentation, etc.); ii) design of rubrics for each of three assessment parts: working process, final report and oral presentation; and iii) follow-up and evaluation of the involved FYPs. Finally, problems that appeared during this experience (e.g. administrative aspects, criticisms and suggestions from the students, tutors and juries involved) are discussed and some modifications in the assessment system will be proposed in order to solve or minimize these problems.

Supervised classification of minimally invasive surgery psychomotor competence based on instruments motion analysis

Background Objective assessment of psychomotor skills has become an important challenge in the training of minimally invasive surgical (MIS) techniques. Currently, no gold standard defining surgical competence exists for classifying residents according to their surgical skills. Supervised classification has been proposed as a means for objectively establishing competence thresholds in psychomotor skills evaluation. This report presents a study comparing three classification methods for establishing their validity in a set of tasks for basic skills’ assessment. Methods Linear discriminant analysis (LDA), support vector machines (SVM), and adaptive neuro-fuzzy inference systems (ANFIS) were used. A total of 42 participants, divided into an experienced group (4 expert surgeons and 14 residents with >10 laparoscopic surgeries performed) and a nonexperienced group (16 students and 8 residents with <10 laparoscopic surgeries performed), performed three box trainer tasks validated for assessment of MIS psychomotor skills. Instrument movements were captured using the TrEndo tracking system, and nine motion analysis parameters (MAPs) were analyzed. The performance of the classifiers was measured by leave-one-out cross-validation using the scores obtained by the participants. Results The mean accuracy performances of the classifiers were 71 % (LDA), 78.2 % (SVM), and 71.7 % (ANFIS). No statistically significant differences in the performance were identified between the classifiers. Conclusions The three proposed classifiers showed good performance in the discrimination of skills, especially when information from all MAPs and tasks combined were considered. A correlation between the surgeons’ previous experience and their execution of the tasks could be ascertained from results. However, misclassifications across all the classifiers could imply the existence of other factors influencing psychomotor competence.

Development of a method of assessment of the problem-solving competence at the technical University of Madrid.

The competence evaluation promoted by the European High Education Area entails a very important methodological change that requires guiding support to help lecturers carry out this new and complex task. In this regard, the Technical University of Madrid (UPM, by its Spanish acronym) has financed a series of coordinated projects with the objective of developing a model for teaching and evaluating core competences and providing support to lecturers. This paper deals with the problem-solving competence. The first step has been to elaborate a guide for teachers to provide a homogeneous way to asses this competence. This guide considers several levels of acquisition of the competence and provides the rubrics to be applied for each one. The guide has been subsequently validated with several pilot experiences. In this paper we will explain the problem-solving assessment guide for teachers and will show the pilot experiences that has been carried out. We will finally justify the validity of the method to assess the problem-solving competence.

Teamwork competence and academic motivation in computer science engineering studies.

The present work is focused on studying two issues: the “teamwork” generic competence and the “academic motivation”. Currently the professional profile of engineers has a strong component of teamwork. On the other hand, motivational profile of students determines their tendencies when they come to work in team, as well as their performance at work. In this context we suggest four hypotheses: (H1) students improve their teamwork capacity by specific training and carrying out a set of activities integrated into an active learning process; (H2) students with higher mastery motivation have better attitude towards team working; (H3) students with higher mastery motivation obtain better results in academic performance; and (H4) students show different motivation profiles in different circumstances: type of courses, teaching methodologies, different times of the learning process. This study was carried out with computer science engineering students from two Spanish universities. The first results point to an improvement in teamwork competence of students if they have previously received specific training in facets of that competence. Other results indicate that there is a correlation between the motivational profiles of students and their perception about teamwork competence. Finally, and contrary to the initial hypothesis, these profiles appear to not influence significantly the academic performance of students.

Best practices for school leaders: Implementation of a profesional competence model for school management in Chile

In Chile, during the last three decades there has been a strong decentralization process whose main objective has been to improve the management of schools through the transfer of responsibilities and resources of education and thus improve the outcome of learning, reducing equity gaps between schools and territories. This is how, there has been an evolution of school principals¿ professional profile from an administrative to a management approach, in which principals have become project managers of educational projects. From a competence model for school leaders, based on IPMA guidelines, the present article presents an analysis of best practices for school management, allowing to generate a link between competencies and school management, from the perspective of project management. Results showed that the different competence elements, have relative weights according to the different practice fields, and that this analysis can be considered as a strategic element in educational project planning and development.

Self perception of decision making competence in spanish football players

Background: It is known that competence to make decisions is a fundamental aspect of sport competition. Objective: This study has analyzed the decision profile of a sample of Spanish football players of different levels of expertise. Methods: 690 Spanish football players of national and international level completed the decision making questionnaire, which cover three dimensions ? perceived decision competence, decision anxiety and commitment with decision learning. MANCOVA and ANOVA analysis were carried out to analyse the differences in each dimension based on the level of expertise. Results: Results showed that perception of decision making competence increased and the anxiety decreased with the level of expertise. Conclusions: This study confirmed the usefulness of this questionnaire in the process of training for coaches and sport psychologists.

Teamwork Competence Professor s Self-Assessment, a Study Case in the Universidad Politecnica de Madrid

Teamwork Competence Professors Self-Assessment, a Study Case in the Universidad Politecnica de Madrid

Diseño y desarrollo de un sistema basado en Learning Analytics para evaluar la competencia de trabajo en equipo.Design and development of a Learning Analytics System to evaluate group work competence

La adquisición de la competencia grupal es algo básico en la docencia universitaria. Esta tarea va a suponer evaluar diferentes factores en un número elevado de alumnos, lo que puede supone gran complejidad y un esfuerzo elevado. De cara a evitar este esfuerzo se puede pensar en emplear los registros de la interacción de los usuarios almacenados en las plataformas de aprendizaje. Para ello el presente trabajo se basa en el desarrollo de un sistema de Learning Analytics que es utilizado como herramienta para analizar las evidencias individuales de los distintos miembros de un equipo de trabajo. El trabajo desarrolla un modelo teórico apoyado en la herramienta, que permite relacionar las evidencias observadas de forma empírica para cada alumno, con indicadores obtenidos tanto de la acción individual como cooperativo de los miembros de un equipo realizadas a través de los foros de trabajo. Abstract — The development of the group work competence is something basic in university teaching. It should be evaluated, but this means to analyze different issues about the participation of a high number of students which is very complex and implies a lot of effort. In order to facilitate this evaluation it is possible to analyze the logs of students’ interaction in Learning Management Systems. The present work describes the development of a Learning Analytics system that analyzes the interaction of each of the members of working group. This tool is supported by a theoretical model, which allows establishing links between the empirical evidences of each student and the indicators of their action in working forums.