Large-scale assessment of language proficiency: theoretical and pedagogical reflections on the use of multiple-choice tests

The new requirement placed on students in tertiary settings in Spain to demonstrate a B1 or a B2 proficiency level of English, in accordance with the Common European Framework of Reference for Languages (CEFRL), has led most Spanish universities to develop a program of certification or accreditation of the required level. The first part of this paper aims to provide a rationale for the type of test that has been developed at the Universidad Politécnica de Madrid for the accreditation of a B2 level, a multiple choice version, and to describe how it was constructed and validated. Then, in the second part of the paper, the results from its application to 924 students enrolled in different degree courses at a variety of schools and faculties at the university are analyzed based on a final test version item analysis. To conclude, some theoretical as well as practical conclusions about testing grammar that affect the teaching and learning process are drawn. RESUMEN. Las nuevas exigencias sobre niveles de competencia B1 y B2 en inglés según el Marco Común Europeo de Referencia para las Lenguas (MCERL) que se imponen sobre los estudiantes de grado y posgrado han llevado a la mayoría de las universidades españolas a desarrollar programas de acreditación o de certificación de estos niveles. La primera parte de este trabajo trata sobre las razones que fundamentan la elección de un tipo concreto de examen para la acreditación del nivel B2 de lengua inglesa en la Universidad Politécnica de Madrid. Se trata de un test de opción múltiple y en esta parte del trabajo se describe cómo fue diseñado y validado. En la segunda parte, se analizan los resultados de la aplicación del test a gran escala a un total de 924 estudiantes matriculados en varias escuelas y Facultades de la Universidad. Para terminar, se apuntan una serie de conclusiones teóricas y prácticas sobre la evaluación de la gramática y de qué modo influye en los procesos de enseñanza y aprendizaje.

A language exchange program: sustainability innovation in language and culture engagement

Spanish Educational Laws have been promoting the widespread use of English; as a result, Spanish Uni versities are looking for ways to give students more international training in order to prepare them for a future that will increasingly involve global problems and partnerships. Therefore, the Polytechnic University of Madrid, Spain (UPM), and the University of British Columbia, Okanagan, Canada (UBCO) have come together to offer opportunities for international collaboration and learning, thus facilitating virtual encounters among Spanish and Canadian students. The Language Exchange Program between the UPM and UBCO acts as a model for sustainability innovation in language and culture engagement as the students can interact with native speakers in communication tasks. This interdisciplinary initiative supports the latest methodological principles observed in the Common European Framework for Languages, such as autonomous and life-long learning, self-assessment and peer-assessment as well as the incorporation of new technologies to the learning process. Additionally the ‘virtual’ mobility is provided at no extra cost. This article presents the preliminary results of two virtual exchange programs that have been offering varied forms of study which are venue-independent, and have clearly expanded the range of scenarios for the students on both sides by promoting collaborative work and cultural exchange.

A language exchange program: sustainability innovation in language and culture engagement

The Language Exchange Program between the UPM and UBCO acts as a model for sustainability innovation in language and culture engagement as the students can interact with native speakers in communication tasks. This interdisciplinary initiative supports the latest methodological principles observed in the Common European Framework for Languages [1], such as autonomous and lifelong learning, self-assessment and peer-assessment as well as the incorporation of new technologies to the learning process

Exploring cognition processes in second language acquisition: the case of cognates and false-friends in EST

This article explores one aspect of the processing perspective in L2 learning in an EST context: the processing of new content words, in English, of the type ‘cognates’ and ‘false friends’, by Spanish speaking engineering students. The paper does not try to offer a comprehensive overview of language acquisition mechanisms, but rather it is intended to review more narrowly how our conceptual systems, governed by intricately linked networks of neural connections in the brain, make language development possible, creating, at the same time, some L2 processing problems. The case of ‘cognates and false friends’ in specialised contexts is brought here to illustrate some of the processing problems that the L2 learner has to confront, and how mappings in the visual, phonological and semantic (conceptual) brain structures function in second language processing of new vocabulary. Resumen Este artículo pretende reflexionar sobre un aspecto de la perspectiva del procesamiento de segundas lenguas (L2) en el contexto del ICT: el procesamiento de palabras nuevas, en inglés, conocidas como “cognados” y “falsos amigos”, por parte de estudiantes de ingeniería españoles. No se pretende ofrecer una visión completa de los mecanismos de adquisición del lenguaje, más bien se intenta mostrar cómo nuestro sistema conceptual, gobernado por una complicada red de conexiones neuronales en el cerebro, hace posible el desarrollo del lenguaje, aunque ello conlleve ciertas dificultades en el procesamiento de segundas lenguas. El caso de los “cognados” y los “falsos amigos”, en los lenguajes de especialidad, se trae para ilustrar algunos de los problemas de procesamiento que el estudiante de una lengua extranjera tiene que afrontar y el funcionamiento de las correspondencias entre las estructuras visuales, fonológicas y semánticas (conceptuales) del cerebro en el procesamiento de nuevo vocabulario.

Evaluating a web-based clinical decision support system for language disorders screening in a nursery school

Background: Early and effective identification of developmental disorders during childhood remains a critical task for the international community. The second highest prevalence of common developmental disorders in children are language delays, which are frequently the first symptoms of a possible disorder. Objective: This paper evaluates a Web-based Clinical Decision Support System (CDSS) whose aim is to enhance the screening of language disorders at a nursery school. The common lack of early diagnosis of language disorders led us to deploy an easy-to-use CDSS in order to evaluate its accuracy in early detection of language pathologies. This CDSS can be used by pediatricians to support the screening of language disorders in primary care. Methods: This paper details the evaluation results of the ?Gades? CDSS at a nursery school with 146 children, 12 educators, and 1 language therapist. The methodology embraces two consecutive phases. The first stage involves the observation of each child?s language abilities, carried out by the educators, to facilitate the evaluation of language acquisition level performed by a language therapist. Next, the same language therapist evaluates the reliability of the observed results. Results: The Gades CDSS was integrated to provide the language therapist with the required clinical information. The validation process showed a global 83.6% (122/146) success rate in language evaluation and a 7% (7/94) rate of non-accepted system decisions within the range of children from 0 to 3 years old. The system helped language therapists to identify new children with potential disorders who required further evaluation. This process will revalidate the CDSS output and allow the enhancement of early detection of language disorders in children. The system does need minor refinement, since the therapists disagreed with some questions from the CDSS knowledge base (KB) and suggested adding a few questions about speech production and pragmatic abilities. The refinement of the KB will address these issues and include the requested improvements, with the support of the experts who took part in the original KB development. Conclusions: This research demonstrated the benefit of a Web-based CDSS to monitor children?s neurodevelopment via the early detection of language delays at a nursery school. Current next steps focus on the design of a model that includes pseudo auto-learning capacity, supervised by experts.

Improving engineering students' communication competence: designing innovative learning strategies

This research presents an innovative and formal educational initiative that is aimed at enhancing the development of engineering students’ specific competencies when studying Engineering Project Management subject. The framework of the experience combines theoretical concepts, the development of a real-case project carried out by multidisciplinary groups of three different universities, the use of software web 2.0 tools, and group and individual assignments of students that play different roles (project managers and team members). Under this scenario, this paper focuses on monitoring the communication competence in the ever growing Project Management virtual environment. Factors such as corporal language, technical means, stage, and management specific vocabulary among others have been considered in order to assess the students’ performance on this issue. As a main contribution, the paper introduces an ad-hoc rubric that, based on previous investigations, has been adapted and tested to this specific context. Additionally, the research conducted has provided some interesting findings that suggest further actions to improve and better define future rubrics, oriented to communication or even other competencies. As specific Project Management subject concerns, it has been detected that students playing the role of Project Managers strengthen their competencies more than those ones that play the role of Team Members. It has also been detected that students have more difficulty assimilating concepts related to risk and quality management. However those concepts related with areas of knowledge like scope, time or cost have been better assimilated by the students.

Integrated approach to foreign language in the building engineering degree at the Universidad Politécnica Madrid

The European Union has been promoting linguistic diversity for many years as one of its main educational goals. This is an element that facilitates student mobility and student exchanges between different universities and countries and enriches the education of young undergraduates. In particular,a higher degree of competence in the English language is becoming essential for engineers, architects and researchers in general, as English has become the lingua franca that opens up horizons to internationalisation and the transfer of knowledge in today’s world. Many experts point to the Integrated Approach to Contents and Foreign Languages System as being an option that has certain benefits over the traditional method of teaching a second language that is exclusively based on specific subjects. This system advocates teaching the different subjects in the syllabus in a language other than one’s mother tongue, without prioritising knowledge of the language over the subject. This was the idea that in the 2009/10 academic year gave rise to the Second Language Integration Programme (SLI Programme) at the Escuela Arquitectura Tecnica in the Universidad Politecnica Madrid (EUATM-UPM), just at the beginning of the tuition of the new Building Engineering Degree, which had been adapted to the European Higher Education Area (EHEA) model. This programme is an interdisciplinary initiative for the set of subjects taught during the semester and is coordinated through the Assistant Director Office for Educational Innovation. The SLI Programme has a dual goal; to familiarise students with the specific English terminology of the subject being taught, and at the same time improve their communication skills in English. A total of thirty lecturers are taking part in the teaching of eleven first year subjects and twelve in the second year, with around 120 students who have voluntarily enrolled in a special group in each semester. During the 2010/2011 academic year the degree of acceptance and the results of the SLI Programme are being monitored. Tools have been designed to aid interdisciplinary coordination and to analyse satisfaction, such as coordination records and surveys. The results currently available refer to the first semester of the year and are divided into specific aspects of the different subjects involved and into general aspects of the ongoing experience.

Improving flexible databases searches using machine learning

El objetivo principal de este proyecto ha sido introducir aprendizaje automático en la aplicación FleSe. FleSe es una aplicación web que permite realizar consultas borrosas sobre bases de datos nítidos. Para llevar a cabo esta función la aplicación utiliza unos criterios para definir los conceptos borrosos usados para llevar a cabo las consultas. FleSe además permite que el usuario cambie estas personalizaciones. Es aquí donde introduciremos el aprendizaje automático, de tal manera que los criterios por defecto cambien y aprendan en función de las personalizaciones que van realizando los usuarios. Los objetivos secundarios han sido familiarizarse con el desarrollo y diseño web, al igual que recordar y ampliar el conocimiento sobre lógica borrosa y el lenguaje de programación lógica Ciao-Prolog. A lo largo de la realización del proyecto y sobre todo después del estudio de los resultados se demuestra que la agrupación de los usuarios marca la diferencia con la última versión de la aplicación. Esto se basa en la siguiente idea, podemos usar un algoritmo de aprendizaje automático sobre las personalizaciones de los criterios de todos los usuarios, pero la gran diversidad de opiniones de los usuarios puede llevar al algoritmo a concluir criterios erróneos o no representativos. Para solucionar este problema agrupamos a los usuarios intentando que cada grupo tengan la misma opinión o mismo criterio sobre el concepto. Y después de haber realizado las agrupaciones usar el algoritmo de aprendizaje automático para precisar el criterio por defecto de cada grupo de usuarios. Como posibles mejoras para futuras versiones de la aplicación FleSe sería un mejor control y manejo del ejecutable plserver. Este archivo se encarga de permitir a la aplicación web usar el lenguaje de programación lógica Ciao-Prolog para llevar a cabo la lógica borrosa relacionada con las consultas. Uno de los problemas más importantes que ofrece plserver es que bloquea el hilo de ejecución al intentar cargar un archivo con errores y en caso de ocurrir repetidas veces bloquea todas las peticiones siguientes bloqueando la aplicación. Pensando en los usuarios y posibles clientes, sería también importante permitir que FleSe trabajase con bases de datos de SQL en vez de almacenar la base de datos en los archivos de Prolog. Otra posible mejora basarse en distintas características a la hora de agrupar los usuarios dependiendo de los conceptos borrosos que se van ha utilizar en las consultas. Con esto se conseguiría que para cada concepto borroso, se generasen distintos grupos de usuarios, los cuales tendrían opiniones distintas sobre el concepto en cuestión. Así se generarían criterios por defecto más precisos para cada usuario y cada concepto borroso.---ABSTRACT---The main objective of this project has been to introduce machine learning in the application FleSe. FleSe is a web application that makes fuzzy queries over databases with precise information, using defined criteria to define the fuzzy concepts used by the queries. The application allows the users to change and custom these criteria. On this point is where the machine learning would be introduced, so FleSe learn from every new user customization of the criteria in order to generate a new default value of it. The secondary objectives of this project were get familiar with web development and web design in order to understand the how the application works, as well as refresh and improve the knowledge about fuzzy logic and logic programing. During the realization of the project and after the study of the results, I realized that clustering the users in different groups makes the difference between this new version of the application and the previous. This conclusion follows the next idea, we can use an algorithm to introduce machine learning over the criteria that people have, but the problem is the diversity of opinions and judgements that exists, making impossible to generate a unique correct criteria for all the users. In order to solve this problem, before using the machine learning methods, we cluster the users in order to make groups that have the same opinion, and afterwards, use the machine learning methods to precise the default criteria of each users group. The future improvements that could be important for the next versions of FleSe will be to control better the behaviour of the plserver file, that cost many troubles at the beginning of this project and it also generate important errors in the previous version. The file plserver allows the web application to use Ciao-Prolog, a logic programming language that control and manage all the fuzzy logic. One of the main problems with plserver is that when the user uploads a file with errors, it will block the thread and when this happens multiple times it will start blocking all the requests. Oriented to the customer, would be important as well to allow FleSe to manage and work with SQL databases instead of store the data in the Prolog files. Another possible improvement would that the cluster algorithm would be based on different criteria depending on the fuzzy concepts that the selected Prolog file have. This will generate more meaningful clusters, and therefore, the default criteria offered to the users will be more precise.

Técnicas de machine learning para la detección de la negación en textos clínicos en español

En los últimos años han surgido nuevos campos de las tecnologías de la información que exploran el tratamiento de la gran cantidad de datos digitales existentes y cómo transformarlos en conocimiento explícito. Las técnicas de Procesamiento del Lenguaje Natural (NLP) son capaces de extraer información de los textos digitales presentados en forma narrativa. Además, las técnicas de machine learning clasifican instancias o ejemplos en función de sus atributos, en distintas categorías, aprendiendo de otros previamente clasificados. Los textos clínicos son una gran fuente de información no estructurada; en consecuencia, información no explotada en su totalidad. Algunos términos usados en textos clínicos se encuentran en una situación de afirmación, negación, hipótesis o histórica. La detección de esta situación es necesaria para la estructuración de información, pero a su vez tiene una gran complejidad. Extrayendo características lingüísticas de los elementos, o tokens, de los textos mediante NLP; transformando estos tokens en instancias y las características en atributos, podemos mediante técnicas de machine learning clasificarlos con el objetivo de detectar si se encuentran afirmados, negados, hipotéticos o históricos. La selección de los atributos que cada token debe tener para su clasificación, así como la selección del algoritmo de machine learning utilizado son elementos cruciales para la clasificación. Son, de hecho, los elementos que componen el modelo de clasificación. Consecuentemente, este trabajo aborda el proceso de extracción de características, selección de atributos y selección del algoritmo de machine learning para la detección de la negación en textos clínicos en español. Se expone un modelo para la clasificación que, mediante el algoritmo J48 y 35 atributos obtenidos de características lingüísticas (morfológicas y sintácticas) y disparadores de negación, detecta si un token está negado en 465 frases provenientes de textos clínicos con un F-Score del 73%, una exhaustividad del 66% y una precisión del 81% con una validación cruzada de 10 iteraciones. ---ABSTRACT--- New information technologies have emerged in the recent years which explore the processing of the huge amount of existing digital data and its transformation into knowledge. Natural Language Processing (NLP) techniques are able to extract certain features from digital texts. Additionally, through machine learning techniques it is feasible to classify instances according to different categories, learning from others previously classified. Clinical texts contain great amount of unstructured data, therefore information not fully exploited. Some terms (tokens) in clinical texts appear in different situations such as affirmed, negated, hypothetic or historic. Detecting this situation is necessary for the structuring of this data, however not simple. It is possible to detect whether if a token is negated, affirmed, hypothetic or historic by extracting its linguistic features by NLP; transforming these tokens into instances, the features into attributes, and classifying these instances through machine learning techniques. Selecting the attributes each instance must have, and choosing the machine learning algorithm are crucial issues for the classification. In fact, these elements set the classification model. Consequently, this work approaches the features retrieval as well as the attributes and algorithm selection process used by machine learning techniques for the detection of negation in clinical texts in Spanish. We present a classification model which, through J48 algorithm and 35 attributes from linguistic features (morphologic and syntactic) and negation triggers, detects whether if a token is negated in 465 sentences from historical records, with a result of 73% FScore, 66% recall and 81% precision using a 10-fold cross-validation.

Capacity-building through English language training to improve long-term resettlement success: A case study of the refugees along the Thailand-Burma border

Achieving long-term resettlement success is a challenge for many refugees seeking to restart their lives after displacement and being uprooted from their lives. Refugees must deal with finding employment, integrating into a society immensely different from what they have known their whole lives, and starting over from scratch. Learning a new language enables refugees to progress towards integration and long-term resettlement success, however, resettled refugees face a multitude of barriers in the U.S. to accessing language classes and attaining English proficiency. This study seeks to bridge this problem by exploring the possibilities of implementing a standardized language training program in the refugee camps to better prepare refugees for resettlement. A case study of the refugees along the Thai-Burma border demonstrated the significance of learning English in the camps on eventual English proficiency as well as the need for increased partnerships to overcome the barriers of lack of motivation and lack of funding. The author explores the possibilities of implementing a language training program in the camps by determining need, interest, barriers, and perceptions through the use of interviews, surveys, and focus groups of camp refugees, resettled refugees, and key organizational representatives. The significance of these results offers the possibility of leveraging and unlocking resettlement as a durable solution for more of the world's refugees in protracted situations.

What We Are Learning About the Diverse Backgrounds of Academic Library Users: An Overview of Research Designs and Methods in Information Behaviour Studies

Academic libraries increasingly serve a more diverse population of users not only in regard to race and ethnicity, but also to age, gender, language, sexual orientation, and national and cultural backgrounds. This papers reports the findings of the study that explored information behaviour research as a potential source of information about diversity of academic library users and examined the relationship between the use of different research designs and data collection methods and the information gathered about users’ diverse backgrounds. The study found that information behaviour research offers limited insight into the diversity of academic library users. The choice of a research design was not critical but the use of multiple data collection played a role in gathering information about culturally diverse users.

EmotiBlog: a finer-grained and more precise learning of subjectivity expression models

The exponential growth of the subjective information in the framework of the Web 2.0 has led to the need to create Natural Language Processing tools able to analyse and process such data for multiple practical applications. They require training on specifically annotated corpora, whose level of detail must be fine enough to capture the phenomena involved. This paper presents EmotiBlog – a fine-grained annotation scheme for subjectivity. We show the manner in which it is built and demonstrate the benefits it brings to the systems using it for training, through the experiments we carried out on opinion mining and emotion detection. We employ corpora of different textual genres –a set of annotated reported speech extracted from news articles, the set of news titles annotated with polarity and emotion from the SemEval 2007 (Task 14) and ISEAR, a corpus of real-life self-expressed emotion. We also show how the model built from the EmotiBlog annotations can be enhanced with external resources. The results demonstrate that EmotiBlog, through its structure and annotation paradigm, offers high quality training data for systems dealing both with opinion mining, as well as emotion detection.

A structural model of cognitive-motivational variables as explanatory factors of academic achievement in Spanish Language and Mathematics

In recent years, several explanatory models have been developed which attempt to analyse the predictive worth of various factors in relation to academic achievement, as well as the direct and indirect effects that they produce. The aim of this study was to examine a structural model incorporating various cognitive and motivational variables which influence student achievement in the two basic core skills in the Spanish curriculum: Spanish Language and Mathematics. These variables included differential aptitudes, specific self-concept, goal orientations, effort and learning strategies. The sample comprised 341 Spanish students in their first year of Compulsory Secondary Education. Various tests and questionnaires were used to assess each student, and Structural Equation Modelling (SEM) was employed to study the relationships in the initial model. The proposed model obtained a satisfactory fit for the two subjects studied, and all the relationships hypothesised were significant. The variable with the most explanatory power regarding academic achievement was mathematical and verbal aptitude. Also notable was the direct influence of specific self-concept on achievement, goal-orientation and effort, as was the mediatory effect that effort and learning strategies had between academic goals and final achievement.

Machine learning techniques for automatic opinion detection in non-traditional textual genres

This paper presents a preliminary study in which Machine Learning experiments applied to Opinion Mining in blogs have been carried out. We created and annotated a blog corpus in Spanish using EmotiBlog. We evaluated the utility of the features labelled firstly carrying out experiments with combinations of them and secondly using the feature selection techniques, we also deal with several problems, such as the noisy character of the input texts, the small size of the training set, the granularity of the annotation scheme and the language object of our study, Spanish, with less resource than English. We obtained promising results considering that it is a preliminary study.

Evaluation of hybrid and distance education learning environments in Spain

This article describes the adaptation and validation of the Distance Education Learning Environments Survey (DELES) for use in investigating the qualities found in distance and hybrid education psycho-social learning environments in Spain. As Europe moves toward post-secondary student mobility, equanimity in access to higher education, and more standardised degree programs across the European Higher Education Area (EHEA) the need for a high quality method for continually assessing the excellence of distance and hybrid learning environments has arisen. This study outlines how the English language DELES was adapted into the new Spanish-Distance Education Learning Environments Survey (S-DELES) for use with a Bachelor of Psychology and Criminology degree program offering both distance and hybrid education classes. We present the relationships between psycho-social learning environment perceptions and those of student affect. We also present the asynchronous aspects of the environment, scale means, and a comparison between the perceptions of distance education students and their hybrid education counterparts that inform the university about the baseline health of the information and communication technologies (ICT) environment within which the study was conducted.