Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

Cross-Linguistic Transfer (CLT) in Bilingual Speakers : Neural Correlates of Language Learning

Le but de cette thèse est d'étudier les corrélats comportementaux et neuronaux du transfert inter-linguistique (TIL) dans l'apprentissage d’une langue seconde (L2). Compte tenu de nos connaissances sur l'influence de la distance linguistique sur le TIL (Paradis, 1987, 2004; Odlin, 1989, 2004, 2005; Gollan, 2005; Ringbom, 2007), nous avons examiné l'effet de facilitation de la similarité phonologique à l’aide de la résonance magnétique fonctionnelle entre des langues linguistiquement proches (espagnol-français) et des langues linguistiquement éloignées (persan-français). L'étude I rapporte les résultats obtenus pour des langues linguistiquement proches (espagnol-français), alors que l'étude II porte sur des langues linguistiquement éloignées (persan-français). Puis, les changements de connectivité fonctionnelle dans le réseau langagier (Price, 2010) et dans le réseau de contrôle supplémentaire impliqué dans le traitement d’une langue seconde (Abutalebi & Green, 2007) lors de l’apprentissage d’une langue linguistiquement éloignée (persan-français) sont rapportés dans l’étude III. Les résultats des analyses d’IRMF suivant le modèle linéaire général chez les bilingues de langues linguistiquement proches (français-espagnol) montrent que le traitement des mots phonologiquement similaires dans les deux langues (cognates et clangs) compte sur un réseau neuronal partagé par la langue maternelle (L1) et la L2, tandis que le traitement des mots phonologiquement éloignés (non-clang-non-cognates) active des structures impliquées dans le traitement de la mémoire de travail et d'attention. Toutefois, chez les personnes bilingues de L1-L2 linguistiquement éloignées (français-persan), même les mots phonologiquement similaires à travers les langues (cognates et clangs) activent des régions connues pour être impliquées dans l'attention et le contrôle cognitif. Par ailleurs, les mots phonologiquement éloignés (non-clang-non-cognates) activent des régions usuellement associées à la mémoire de travail et aux fonctions exécutives. Ainsi, le facteur de distance inter-linguistique entre L1 et L2 module la charge cognitive sur la base du degré de similarité phonologiques entres les items en L1 et L2. Des structures soutenant les processus impliqués dans le traitement exécutif sont recrutées afin de compenser pour des demandes cognitives. Lorsque la compétence linguistique en L2 augmente et que les tâches linguistiques exigent ainsi moins d’effort, la demande pour les ressources cognitives diminue. Tel que déjà rapporté (Majerus, et al, 2008; Prat, et al, 2007; Veroude, et al, 2010; Dodel, et al, 2005; Coynel, et al ., 2009), les résultats des analyses de connectivité fonctionnelle montrent qu’après l’entraînement la valeur d'intégration (connectivité fonctionnelle) diminue puisqu’il y a moins de circulation du flux d'information. Les résultats de cette recherche contribuent à une meilleure compréhension des aspects neurocognitifs et de plasticité cérébrale du TIL ainsi que l'impact de la distance linguistique dans l'apprentissage des langues. Ces résultats ont des implications dans les stratégies d'apprentissage d’une L2, les méthodes d’enseignement d’une L2 ainsi que le développement d'approches thérapeutiques chez des patients bilingues qui souffrent de troubles langagiers.

Design And Development Of A Named Entity Based Question Answering System For Malayalam Language

This is a Named Entity Based Question Answering System for Malayalam Language. Although a vast amount of information is available today in digital form, no effective information access mechanism exists to provide humans with convenient information access. Information Retrieval and Question Answering systems are the two mechanisms available now for information access. Information systems typically return a long list of documents in response to a user’s query which are to be skimmed by the user to determine whether they contain an answer. But a Question Answering System allows the user to state his/her information need as a natural language question and receives most appropriate answer in a word or a sentence or a paragraph. This system is based on Named Entity Tagging and Question Classification. Document tagging extracts useful information from the documents which will be used in finding the answer to the question. Question Classification extracts useful information from the question to determine the type of the question and the way in which the question is to be answered. Various Machine Learning methods are used to tag the documents. Rule-Based Approach is used for Question Classification. Malayalam belongs to the Dravidian family of languages and is one of the four major languages of this family. It is one of the 22 Scheduled Languages of India with official language status in the state of Kerala. It is spoken by 40 million people. Malayalam is a morphologically rich agglutinative language and relatively of free word order. Also Malayalam has a productive morphology that allows the creation of complex words which are often highly ambiguous. Document tagging tools such as Parts-of-Speech Tagger, Phrase Chunker, Named Entity Tagger, and Compound Word Splitter are developed as a part of this research work. No such tools were available for Malayalam language. Finite State Transducer, High Order Conditional Random Field, Artificial Immunity System Principles, and Support Vector Machines are the techniques used for the design of these document preprocessing tools. This research work describes how the Named Entity is used to represent the documents. Single sentence questions are used to test the system. Overall Precision and Recall obtained are 88.5% and 85.9% respectively. This work can be extended in several directions. The coverage of non-factoid questions can be increased and also it can be extended to include open domain applications. Reference Resolution and Word Sense Disambiguation techniques are suggested as the future enhancements

Language learning in the public eye: an analysis of newspapers and official documents in England

This article considers the issue of low levels of motivation for foreign language learning in England by exploring how language learning is conceptualised by different key voices in that country through the examination of written data: policy documents and reports on the UK's language needs, curriculum documents, and press articles. The extent to which this conceptualisation has changed over time is explored, through the consideration of documents from two time points, before and after a change in government in the UK. The study uses corpus analysis methods in this exploration. The picture that emerges is a complex one regarding how the 'problems' and 'solutions' surrounding language learning in that context are presented in public discourse. This, we conclude, has implications for the likely success of measures adopted to increase language learning uptake in that context.

K-Surfer: a KNIME extension for the management and analysis of Human brain MRI FreeSurfer/FSL data

Human brain imaging techniques, such as Magnetic Resonance Imaging (MRI) or Diffusion Tensor Imaging (DTI), have been established as scientific and diagnostic tools and their adoption is growing in popularity. Statistical methods, machine learning and data mining algorithms have successfully been adopted to extract predictive and descriptive models from neuroimage data. However, the knowledge discovery process typically requires also the adoption of pre-processing, post-processing and visualisation techniques in complex data workflows. Currently, a main problem for the integrated preprocessing and mining of MRI data is the lack of comprehensive platforms able to avoid the manual invocation of preprocessing and mining tools, that yields to an error-prone and inefficient process. In this work we present K-Surfer, a novel plug-in of the Konstanz Information Miner (KNIME) workbench, that automatizes the preprocessing of brain images and leverages the mining capabilities of KNIME in an integrated way. K-Surfer supports the importing, filtering, merging and pre-processing of neuroimage data from FreeSurfer, a tool for human brain MRI feature extraction and interpretation. K-Surfer automatizes the steps for importing FreeSurfer data, reducing time costs, eliminating human errors and enabling the design of complex analytics workflow for neuroimage data by leveraging the rich functionalities available in the KNIME workbench.

Assessment for Learning in teaching English to Young Learners: teachers’ understanding, classroom practice and impact on interactions

It has been suggested that Assessment for Learning (AfL) plays a significant role in enhancing teaching and learning in mainstream educational contexts. However, little empirical evidence can support these claims. As AfL has been shown to be enacted predominantly through interactions in primary classes, there is a need to understand if it is appropriate, whether it can be efficiently used in teaching English to Young Learners (TEYL) and how it can facilitate learning in such a context. This emerging research focus gains currency especially in the light of SLA research, which suggests the important role of interactions in foreign language learning. This mixed-method, descriptive and exploratory study aims to investigate how teachers of learners aged 7-11 understand AfL; how they implement it; and the impact that such implementation could have on interactions which occur during lessons. The data were collected through lesson observations, scrutiny of school documents, semi-structured interviews and a focus group interview with teachers. The findings indicate that fitness for purpose guides the implementation of AfL in TEYL classrooms. Significantly, the study has revealed differences in the implementation of AfL between classes of 7-9 and 10-11 year olds within each of the three purposes (setting objectives and expectations; monitoring performance; and checking achievement) identified through the data. Another important finding of this study is the empirical evidence suggesting that the use of AfL could facilitate creating conditions conducive to learning in TEYL classes during collaborative and expert/novice interactions. The findings suggest that teachers’ understanding of AfL is largely aligned with the theoretical frameworks (Black & Wiliam, 2009; Swaffield, 2011) already available. However, they also demonstrate that there are TEYL specific characteristics. This research has important pedagogical implications and indicates a number of areas for further research.

Discourse strategies and power roles in student-led distance learning

The assertion of identity and power via computer-mediated communication in the context of distance or web-based learning presents challenges to both teachers and students. When regular, face-to-face classroom interaction is replaced by online chat or group discussion forums, participants must avail themselves of new techniques and tactics for contributing to and furthering interaction, discussion, and learning. During student-only chat sessions, the absence of teacher-led, face-to-face classroom activities requires the students to assume leadership roles and responsibilities normally associated with the teacher. This situation raises the questions of who teaches and who learns; how students discursively negotiate power roles; and whether power emerges as a function of displayed expertise and knowledge or rather the use of authoritative language. This descriptive study represents an examination of a corpus of task-based discussion logs among Vietnamese students of distance learning courses in English linguistics. The data reveal recurring discourse strategies for 1) negotiating the progression of the discussion sessions, 2) asserting and questioning knowledge, and 3) assuming or delegating responsibility. Power is defined ad hoc as the ability to successfully perform these strategies. The data analysis contributes to a better understanding of how working methods and materials can be tailored to students in distance learning courses, and how such students can be empowered by being afforded opportunities and effectively encouraged to assert their knowledge and authority.

How to make active interactions in Japanese as a second language.

This paper analyzes Japanese language classes at Dalarna University in Sweden that are held through a web conferencing system. It discusses how students’ learning and language acquisition can be supported by making better use of the available features of using a web conferencing system for language lessons. Of particular interest is the existence of an “information gap” among students, created because of the limits posed by distance communication. Students who take Japanese courses at Dalarna University usually access classes from their home, which are located all over Sweden or even abroad. This fact can be utilized in language classes because the “information gap” can lead to interactions that are essential for language learning. In order to make use of this natural “information gap” and turn it into an opportunity for communication, our classes used a teaching method called “personalization” [Kawaguchi, 2004].  “Personalization” aims to persuade students to express their own ideas, opinions, feelings and preferences. The present analysis suggests that “personalization” in web-based language classes is a surprisingly effective teaching method. By making students explain about things at home (why they have them, what they use them for, or why they are important), students become motivated to express themselves in Japanese. This makes communication meaningful and enhances students’ interest in improving their vocabulary. Furthermore, by knowing each other, it becomes easier to create a ”supportive classroom environment” [Nuibe, 2001] in which students feel able to express themselves. The analysis suggests that that web-based education can be seen not simply as a supplement to traditional face-to face classroom education, but as a unique and effective educational platform in itself.

Mobile Assisted Language Learning : A Literature Review

Mobile assisted language learning (MALL) is a subarea of the growing field of mobile learning (mLearning) research which increasingly attracts the attention of scholars. This study provides a systematic review of MALL research within the specific area of second language acquisition during the period 2007 - 2012 in terms of research approaches, methods, theories and models, as well as results in the form of linguistic knowledge and skills. The findings show that studies of mobile technology use in different aspects of language learning support the hypothesis that mobile technology can enhance learners’ second language acquisition. However, most of the reviewed studies are experimental, small-scale, and conducted within a short period of time. There is also a lack of cumulative research; most theories and concepts are used only in one or a few papers. This raises the issue of the reliability of findings over time, across changing technologies, and in terms of scalability. In terms of gained linguistic knowledge and skills, attention is primarily on learners’ vocabulary acquisition, listening and speaking skills, and language acquisition in more general terms.

Evidence Algorithm and Processing Formalized Mathematical Texts

The goal of a research programme Evidence Algorithm is a development of an open system of automated proving that is able to accumulate mathematical knowledge and to prove theorems in a context of a self-contained mathematical text. By now, the first version of such a system called a System for Automated Deduction, SAD, is implemented in software. The system SAD possesses the following main features: mathematical texts are formalized using a specific formal language that is close to a natural language of mathematical publications; a proof search is based on special sequent-type calculi formalizing natural reasoning style, such as application of definitions and auxiliary propositions. These calculi also admit a separation of equality handling from deduction that gives an opportunity to integrate logical reasoning with symbolic calculation.

Particle Competition and Cooperation in Networks for Semi-Supervised Learning with Concept Drift

Concept drift is a problem of increasing importance in machine learning and data mining. Data sets under analysis are no longer only static databases, but also data streams in which concepts and data distributions may not be stable over time. However, most learning algorithms produced so far are based on the assumption that data comes from a fixed distribution, so they are not suitable to handle concept drifts. Moreover, some concept drifts applications requires fast response, which means an algorithm must always be (re) trained with the latest available data. But the process of labeling data is usually expensive and/or time consuming when compared to unlabeled data acquisition, thus only a small fraction of the incoming data may be effectively labeled. Semi-supervised learning methods may help in this scenario, as they use both labeled and unlabeled data in the training process. However, most of them are also based on the assumption that the data is static. Therefore, semi-supervised learning with concept drifts is still an open challenge in machine learning. Recently, a particle competition and cooperation approach was used to realize graph-based semi-supervised learning from static data. In this paper, we extend that approach to handle data streams and concept drift. The result is a passive algorithm using a single classifier, which naturally adapts to concept changes, without any explicit drift detection mechanism. Its built-in mechanisms provide a natural way of learning from new data, gradually forgetting older knowledge as older labeled data items became less influent on the classification of newer data items. Some computer simulation are presented, showing the effectiveness of the proposed method.

Semi-supervised learning with concept drift using particle dynamics applied to network intrusion detection data

Concept drift, which refers to non stationary learning problems over time, has increasing importance in machine learning and data mining. Many concept drift applications require fast response, which means an algorithm must always be (re)trained with the latest available data. But the process of data labeling is usually expensive and/or time consuming when compared to acquisition of unlabeled data, thus usually only a small fraction of the incoming data may be effectively labeled. Semi-supervised learning methods may help in this scenario, as they use both labeled and unlabeled data in the training process. However, most of them are based on assumptions that the data is static. Therefore, semi-supervised learning with concept drifts is still an open challenging task in machine learning. Recently, a particle competition and cooperation approach has been developed to realize graph-based semi-supervised learning from static data. We have extend that approach to handle data streams and concept drift. The result is a passive algorithm which uses a single classifier approach, naturally adapted to concept changes without any explicit drift detection mechanism. It has built-in mechanisms that provide a natural way of learning from new data, gradually "forgetting" older knowledge as older data items are no longer useful for the classification of newer data items. The proposed algorithm is applied to the KDD Cup 1999 Data of network intrusion, showing its effectiveness.

Investigation of auditory processing disorder and language impairment using the speech-evoked auditory brainstem response

This study investigated whether there are differences in the Speech-Evoked Auditory Brainstem Response among children with Typical Development (TD), (Central) Auditory Processing Disorder (C) APD, and Language Impairment (LI). The speech-evoked Auditory Brainstem Response was tested in 57 children (ages 6-12). The children were placed into three groups: TD (n = 18), (C)APD (n = 18) and LI (n = 21). Speech-evoked ABR were elicited using the five-formant syllable/da/. Three dimensions were defined for analysis, including timing, harmonics, and pitch. A comparative analysis of the responses between the typical development children and children with (C)APD and LI revealed abnormal encoding of the speech acoustic features that are characteristics of speech perception in children with (C)APD and LI, although the two groups differed in their abnormalities. While the children with (C)APD might had a greater difficulty distinguishing stimuli based on timing cues, the children with LI had the additional difficulty of distinguishing speech harmonics, which are important to the identification of speech sounds. These data suggested that an inefficient representation of crucial components of speech sounds may contribute to the difficulties with language processing found in children with LI. Furthermore, these findings may indicate that the neural processes mediated by the auditory brainstem differ among children with auditory processing and speech-language disorders. (C) 2012 Elsevier B.V. All rights reserved.

Frame-driven Extraction of Linked Data and Ontologies from Text

Ontology design and population -core aspects of semantic technologies- re- cently have become fields of great interest due to the increasing need of domain-specific knowledge bases that can boost the use of Semantic Web. For building such knowledge resources, the state of the art tools for ontology design require a lot of human work. Producing meaningful schemas and populating them with domain-specific data is in fact a very difficult and time-consuming task. Even more if the task consists in modelling knowledge at a web scale. The primary aim of this work is to investigate a novel and flexible method- ology for automatically learning ontology from textual data, lightening the human workload required for conceptualizing domain-specific knowledge and populating an extracted schema with real data, speeding up the whole ontology production process. Here computational linguistics plays a fundamental role, from automati- cally identifying facts from natural language and extracting frame of relations among recognized entities, to producing linked data with which extending existing knowledge bases or creating new ones. In the state of the art, automatic ontology learning systems are mainly based on plain-pipelined linguistics classifiers performing tasks such as Named Entity recognition, Entity resolution, Taxonomy and Relation extraction [11]. These approaches present some weaknesses, specially in capturing struc- tures through which the meaning of complex concepts is expressed [24]. Humans, in fact, tend to organize knowledge in well-defined patterns, which include participant entities and meaningful relations linking entities with each other. In literature, these structures have been called Semantic Frames by Fill- 6 Introduction more [20], or more recently as Knowledge Patterns [23]. Some NLP studies has recently shown the possibility of performing more accurate deep parsing with the ability of logically understanding the structure of discourse [7]. In this work, some of these technologies have been investigated and em- ployed to produce accurate ontology schemas. The long-term goal is to collect large amounts of semantically structured information from the web of crowds, through an automated process, in order to identify and investigate the cognitive patterns used by human to organize their knowledge.