Machine learning for matching astronomy catalogues

An emerging issue in the field of astronomy is the integration, management and utilization of databases from around the world to facilitate scientific discovery. In this paper, we investigate application of the machine learning techniques of support vector machines and neural networks to the problem of amalgamating catalogues of galaxies as objects from two disparate data sources: radio and optical. Formulating this as a classification problem presents several challenges, including dealing with a highly unbalanced data set. Unlike the conventional approach to the problem (which is based on a likelihood ratio) machine learning does not require density estimation and is shown here to provide a significant improvement in performance. We also report some experiments that explore the importance of the radio and optical data features for the matching problem.

Probabilistic methods for drug discovery

This thesis introduces a flexible visual data exploration framework which combines advanced projection algorithms from the machine learning domain with visual representation techniques developed in the information visualisation domain to help a user to explore and understand effectively large multi-dimensional datasets. The advantage of such a framework to other techniques currently available to the domain experts is that the user is directly involved in the data mining process and advanced machine learning algorithms are employed for better projection. A hierarchical visualisation model guided by a domain expert allows them to obtain an informed segmentation of the input space. Two other components of this thesis exploit properties of these principled probabilistic projection algorithms to develop a guided mixture of local experts algorithm which provides robust prediction and a model to estimate feature saliency simultaneously with the training of a projection algorithm.Local models are useful since a single global model cannot capture the full variability of a heterogeneous data space such as the chemical space. Probabilistic hierarchical visualisation techniques provide an effective soft segmentation of an input space by a visualisation hierarchy whose leaf nodes represent different regions of the input space. We use this soft segmentation to develop a guided mixture of local experts (GME) algorithm which is appropriate for the heterogeneous datasets found in chemoinformatics problems. Moreover, in this approach the domain experts are more involved in the model development process which is suitable for an intuition and domain knowledge driven task such as drug discovery. We also derive a generative topographic mapping (GTM) based data visualisation approach which estimates feature saliency simultaneously with the training of a visualisation model.

A constructive learning algorithm based on back-propagation

There are been a resurgence of interest in the neural networks field in recent years, provoked in part by the discovery of the properties of multi-layer networks. This interest has in turn raised questions about the possibility of making neural network behaviour more adaptive by automating some of the processes involved. Prior to these particular questions, the process of determining the parameters and network architecture required to solve a given problem had been a time consuming activity. A number of researchers have attempted to address these issues by automating these processes, concentrating in particular on the dynamic selection of an appropriate network architecture.The work presented here specifically explores the area of automatic architecture selection; it focuses upon the design and implementation of a dynamic algorithm based on the Back-Propagation learning algorithm. The algorithm constructs a single hidden layer as the learning process proceeds using individual pattern error as the basis of unit insertion. This algorithm is applied to several problems of differing type and complexity and is found to produce near minimal architectures that are shown to have a high level of generalisation ability.

Relation discovery from web data for competency management

In current organizations, valuable enterprise knowledge is often buried under rapidly expanding huge amount of unstructured information in the form of web pages, blogs, and other forms of human text communications. We present a novel unsupervised machine learning method called CORDER (COmmunity Relation Discovery by named Entity Recognition) to turn these unstructured data into structured information for knowledge management in these organizations. CORDER exploits named entity recognition and co-occurrence data to associate individuals in an organization with their expertise and associates. We discuss the problems associated with evaluating unsupervised learners and report our initial evaluation experiments in an expert evaluation, a quantitative benchmarking, and an application of CORDER in a social networking tool called BuddyFinder.

Community relation discovery by named entities

Discovering who works with whom, on which projects and with which customers is a key task in knowledge management. Although most organizations keep models of organizational structures, these models do not necessarily accurately reflect the reality on the ground. In this paper we present a text mining method called CORDER which first recognizes named entities (NEs) of various types from Web pages, and then discovers relations from a target NE to other NEs which co-occur with it. We evaluated the method on our departmental Website. We used the CORDER method to first find related NEs of four types (organizations, people, projects, and research areas) from Web pages on the Website and then rank them according to their co-occurrence with each of the people in our department. 20 representative people were selected and each of them was presented with ranked lists of each type of NE. Each person specified whether these NEs were related to him/her and changed or confirmed their rankings. Our results indicate that the method can find the NEs with which these people are closely related and provide accurate rankings.

A novel adaptive weight selection algorithm for multi-objective multi-agent reinforcement learning

To solve multi-objective problems, multiple reward signals are often scalarized into a single value and further processed using established single-objective problem solving techniques. While the field of multi-objective optimization has made many advances in applying scalarization techniques to obtain good solution trade-offs, the utility of applying these techniques in the multi-objective multi-agent learning domain has not yet been thoroughly investigated. Agents learn the value of their decisions by linearly scalarizing their reward signals at the local level, while acceptable system wide behaviour results. However, the non-linear relationship between weighting parameters of the scalarization function and the learned policy makes the discovery of system wide trade-offs time consuming. Our first contribution is a thorough analysis of well known scalarization schemes within the multi-objective multi-agent reinforcement learning setup. The analysed approaches intelligently explore the weight-space in order to find a wider range of system trade-offs. In our second contribution, we propose a novel adaptive weight algorithm which interacts with the underlying local multi-objective solvers and allows for a better coverage of the Pareto front. Our third contribution is the experimental validation of our approach by learning bi-objective policies in self-organising smart camera networks. We note that our algorithm (i) explores the objective space faster on many problem instances, (ii) obtained solutions that exhibit a larger hypervolume, while (iii) acquiring a greater spread in the objective space.

The evolution of learning systems:to Bayes or not to be

Bayesian algorithms pose a limit to the performance learning algorithms can achieve. Natural selection should guide the evolution of information processing systems towards those limits. What can we learn from this evolution and what properties do the intermediate stages have? While this question is too general to permit any answer, progress can be made by restricting the class of information processing systems under study. We present analytical and numerical results for the evolution of on-line algorithms for learning from examples for neural network classifiers, which might include or not a hidden layer. The analytical results are obtained by solving a variational problem to determine the learning algorithm that leads to maximum generalization ability. Simulations using evolutionary programming, for programs that implement learning algorithms, confirm and expand the results. The principal result is not just that the evolution is towards a Bayesian limit. Indeed it is essentially reached. In addition we find that evolution is driven by the discovery of useful structures or combinations of variables and operators. In different runs the temporal order of the discovery of such combinations is unique. The main result is that combinations that signal the surprise brought by an example arise always before combinations that serve to gauge the performance of the learning algorithm. This latter structures can be used to implement annealing schedules. The temporal ordering can be understood analytically as well by doing the functional optimization in restricted functional spaces. We also show that there is data suggesting that the appearance of these traits also follows the same temporal ordering in biological systems. © 2006 American Institute of Physics.

Applications of Nonclassical Logic Methods for Purposes of Knowledge Discovery and Data Mining

* The work is partially supported by Grant no. NIP917 of the Ministry of Science and Education – Republic of Bulgaria.

Learner support and discovery in a virtual non-judgmental environment

This thesis is a qualitative case study that draws upon a grounded genre analysis approach situated within the social constructivist paradigm. The study describes the various obligatory, desired, and optional moves used by post-graduate students as they interacted within an online, non-judgmental environment in order to seek solutions to issues they were experiencing with their research projects or teaching. The postgraduate students or case participants met individually online with me at pre-arranged times to take part in Instant Messenger Cooperative Development (IMCD) (Boon, 2005) 30-minute to one hour sessions via the text-chat function of Skype. Participants took on the role of ‘Explorer’ in order to articulate their thoughts and ideas about their research. I took on the role of ‘Understander’ to provide support to each Explorer by reflecting my understanding of the ongoing articulations as the Explorers investigated their specific issues, determined possible ways to overcome them, made new discoveries, and formulated plans of action regarding the best way for them to move forward. The description of generic moves covers 32 IMCD sessions collected over a threeyear period (2009-2012) from 10 different participants (A-J). Data collected is drawn from live IMCD sessions, field notes, and post-session email feedback from participants. In particular, the thesis focuses on describing the specific generic moves of Explorers within IMCD sessions as they seek satisfactory resolutions to particular research or pedagogic puzzles. It also provides a detailed description of a longitudinal case (Participant A – four sessions), a one-session case (Participant B – one session), and an outlier case in which the Explorer underwent a negative IMCD experience. The thesis concludes by arguing that IMCD is a highly effective tool that helps facilitate the research process for both distance-learning and on-campus students and has the potential to be utilized across all disciplines at the tertiary level.

A case study of adults in college who developed an experiential learning portfolio

This single-case study provides a description and explanation of selected adult students' perspectives on the impact that the development of an experiential learning portfolio had on their understanding of their professional and personal lives. The conceptual framework that undergirded the study included theoretical and empirical studies on adult learning, experiential learning, and the academic quality of nontraditional degree programs with a portfolio component. The study employed qualitative data collection techniques of individual interviews, document review, field notes, and researcher journal. A purposive sample of 8 adult students who completed portfolios as a component of their undergraduate degrees participated in the study. The 4 male and 4 female students who were interviewed represented 4 ethnic/racial groups and ranged in age from 32 to 55 years. Each student's portfolio was read prior to the interview to frame the semi-structured interview questions in light of written portfolio documents. ^ Students were interviewed twice over a 3-month period. The study lasted 8 months from data collection to final presentation of the findings. The data from interview transcriptions and student portfolios were analyzed, categorized, coded, and sorted into 4 major themes and 2 additional themes and submitted to interpretive analysis. ^ Participants' attitudes, perceptions, and opinions of their learning from the portfolio development experience were presented in the findings, which were illustrated through the use of excerpts from interview responses and individual portfolios. The participants displayed a positive reaction to the learning they acquired from the portfolio development process, regardless of their initial concerns about the challenges of creating a portfolio. Concerns were replaced by a greater recognition and understanding of their previous professional and personal accomplishments and their ability to reach future goals. Other key findings included (a) a better understanding of the role work played in their learning and development, (b) a deeper recognition of the impact of mentors and role models throughout their lives, (c) an increase in writing and organizational competencies, and (d) a sense of self-discovery and personal empowerment. ^

Knowledge discovery using multiple sources of biological data

The primary aim of this dissertation is to develop data mining tools for knowledge discovery in biomedical data when multiple (homogeneous or heterogeneous) sources of data are available. The central hypothesis is that, when information from multiple sources of data are used appropriately and effectively, knowledge discovery can be better achieved than what is possible from only a single source. ^ Recent advances in high-throughput technology have enabled biomedical researchers to generate large volumes of diverse types of data on a genome-wide scale. These data include DNA sequences, gene expression measurements, and much more; they provide the motivation for building analysis tools to elucidate the modular organization of the cell. The challenges include efficiently and accurately extracting information from the multiple data sources; representing the information effectively, developing analytical tools, and interpreting the results in the context of the domain. ^ The first part considers the application of feature-level integration to design classifiers that discriminate between soil types. The machine learning tools, SVM and KNN, were used to successfully distinguish between several soil samples. ^ The second part considers clustering using multiple heterogeneous data sources. The resulting Multi-Source Clustering (MSC) algorithm was shown to have a better performance than clustering methods that use only a single data source or a simple feature-level integration of heterogeneous data sources. ^ The third part proposes a new approach to effectively incorporate incomplete data into clustering analysis. Adapted from K-means algorithm, the Generalized Constrained Clustering (GCC) algorithm makes use of incomplete data in the form of constraints to perform exploratory analysis. Novel approaches for extracting constraints were proposed. For sufficiently large constraint sets, the GCC algorithm outperformed the MSC algorithm. ^ The last part considers the problem of providing a theme-specific environment for mining multi-source biomedical data. The database called PlasmoTFBM, focusing on gene regulation of Plasmodium falciparum, contains diverse information and has a simple interface to allow biologists to explore the data. It provided a framework for comparing different analytical tools for predicting regulatory elements and for designing useful data mining tools. ^ The conclusion is that the experiments reported in this dissertation strongly support the central hypothesis.^

Agent-Based Knowledge Discovery

Postprint

AR-based Technoself Enhanced Learning Approach to Improving Student Engagement

The emerging technologies have expanded a new dimension of self – ‘technoself’ driven by socio-technical innovations and taken an important step forward in pervasive learning. Technology Enhanced Learning (TEL) research has increasingly focused on emergent technologies such as Augmented Reality (AR) for augmented learning, mobile learning, and game-based learning in order to improve self-motivation and self-engagement of the learners in enriched multimodal learning environments. These researches take advantage of technological innovations in hardware and software across different platforms and devices including tablets, phoneblets and even game consoles and their increasing popularity for pervasive learning with the significant development of personalization processes which place the student at the center of the learning process. In particular, augmented reality (AR) research has matured to a level to facilitate augmented learning, which is defined as an on-demand learning technique where the learning environment adapts to the needs and inputs from learners. In this paper we firstly study the role of Technology Acceptance Model (TAM) which is one of the most influential theories applied in TEL on how learners come to accept and use a new technology. Then we present the design methodology of the technoself approach for pervasive learning and introduce technoself enhanced learning as a novel pedagogical model to improve student engagement by shaping personal learning focus and setting. Furthermore we describe the design and development of an AR-based interactive digital interpretation system for augmented learning and discuss key features. By incorporating mobiles, game simulation, voice recognition, and multimodal interaction through Augmented Reality, the learning contents can be geared toward learner's needs and learners can stimulate discovery and gain greater understanding. The system demonstrates that Augmented Reality can provide rich contextual learning environment and contents tailored for individuals. Augment learning via AR can bridge this gap between the theoretical learning and practical learning, and focus on how the real and virtual can be combined together to fulfill different learning objectives, requirements, and even environments. Finally, we validate and evaluate the AR-based technoself enhanced learning approach to enhancing the student motivation and engagement in the learning process through experimental learning practices. It shows that Augmented Reality is well aligned with constructive learning strategies, as learners can control their own learning and manipulate objects that are not real in augmented environment to derive and acquire understanding and knowledge in a broad diversity of learning practices including constructive activities and analytical activities.

Teaching down or learning up

Résumé : L'élément important que cette thèse sous-tend est que l'enseignement efficace n'est pas seulement constitué de techniques et de méthodologie, mais plutôt d'attitude et d'approche envers l'enseignement. Ceci ne veut pas nécessairement dire que plusieurs méthodes d'enseignement reçues dans un cours avec l'intention d'optimaliser les mécanismes de transmission et d'assimilation de la matière sont inappropriées. Cependant, l'absence de ce que nous pourrions définir comme un ton pédagogique est essentiel, c'est-à-dire, qu'une attitude positive à la productivité autant vis-à-vis de la matière à transmettre que vis-à-vis de l'individu impliqué dans "l'acte" de réception versus la découverte, aura davantage de succès. Toute autre méthode sera complètement inefficace, inaccessible, voire même inutile. D'emblée, dans l'hypothèse de départ, l'argument principal présente une attitude générale d'enseignement à divers échelons ; soit au niveau secondaire ou collégial qui est inappropriée, incomplète ou négative. En d'autres mots, cette approche thérapeutise l'éducation. Dans l'exercice de cette approche, l'enseignant ou l'enseignante adopte plutôt le rôle d'un thérapeute que celui d'un éducateur. De ce fait, le professeur en situation a une approche plutôt de thérapeute que celle de maître-précepteur et que la matière présentée est souvent diluée, et réduite à des niveaux d'apprentissage accompagnés de carences notoires et d'échecs académiques. Les attentes d'une performance dans le milieu académique sont souvent des plus modestes. Cette même tendance d'une éducation à la baisse est évidente aussi dans le processus d'évaluation. Il est certain que dans les disciplines non scientifiques, l'évaluation formative a grandement suivi l'évaluation normative conduisant le précepteur, tour à tour, dans une évaluation dormative dans laquelle l'effort et l'intention remplacent les aptitudes et les habilitées réelles. Si l'approche pédagogique est vraiment l'élément crucial de l'éducation, il Importe que l'approche générale influence le climat de l'éducation contemporaine, de fait, devienne un palliatif contre-productif souvent réhabilitant. De plus, cette pseudo-thérapie d'où d'écoule une attitude exigeante envers l'enseignant et l'apprenant dont le fondement est la reconnaissance des impératifs culturels qui en sont le reflet et le corps doit-être affirmé et transposé dans la réalité. Cette dernière comprend des attentes très poussées en ce qui concerne la performance en classe et aussi le respect de la matière qui contient la présentation routinière et fondamentale; renouveau intense du processus d'évaluation qui fournira des standards communs et des objectifs externes dans l'évaluation du travail de l'étudiant. Cette connaissance et domestication empirique que nous présente Vygotsky dans un climat contemporain qu'il a expliqué ces termes comme "des zones de développement proximales" basées sur la doctrine suivante que le bon apprentissage précède le développement et que conséquemment s'ensuit une pédagogie d'apprentissage plutôt qu'une pédagogie centrée sur l'apprenant. L'application significative de ces derniers principes ou de ces épistémologiques s'imbriquent dans une situation d'apprentissage ascentionnel dont la structure est détaillée et considérée par différentes perspectives de la recherche qui suit.||Abstract : The central tenet of this thesis is that effective teaching is not only and perhaps not primarily a matter of technique and methodology but of attitude and approach. This is not to say that diverse methods of classroom instruction intended to optimize the mechanics of transmission and the assimilation of data are inappropriate but that in the absence of what we might denominate as a certain pedagogical tone. that is, a productive attitude toward both the material to be conveyed and the individuel engaged in the 'act' of reception-and-discovery, even the most powerful methods will be differentially unavailing or, at best, inefficient. Given this initial assumption, the argument proceeds that the general attitude toward instruction currently in place at the secondary echelons, that is, on the high school and college levels, may be popularly represented as a 'teaching down' approach, in other words, as one which seeks to therapeuticize education. In practice this means that the teacher tends to manifest in situ more as a therapist than as a preceptor, that the material to be presented is frequently diluted or scaled down to perceived levels of cognitive (dis)ability (as is also the case with the rate of instruction), and that performance expectations in the current pedagogical milieu are commonly quite modest. The same downward trend is evident in assessment protocols as well. Certainly in the nonscientific disciplines, normative evaluation has been widely succeeded by formative evaluation, leading in turn to a peculiar kind of dormative evaluation in which intangibles such as effort and intention may deputize for realized ability. If pedagogical approach is indeed the crucial element in instruction, and if the general approach that pervades the contemporary climate of instruction is indeed counter-productively remedial or rehabilitory, that is, therapeutic, then it should follow that a more demanding attitude toward teaching and learning founded on the recognition of the culturel imperative which teaching both reflects and embodies needs to be re-affirmed and translated into practice. This latter would entail the maintenance of high expectations with regard to classroom performance, a respect for the material which precludes its routine mitigation or debasement, a renewed insistance on grading protocols that provide an external, 'objective' or communal standard against which the student's work can be measured, the empirical acknowledgment or domestication of what Vygotsky has termed "the zone of proximal development," based on the doctrine that good learning proceeds in advance of development, and conséquently, a learning-centered rather than learner-centered pedagogy. The meaningful application of this latter set of principles or epistemological gradients comprises the 'learning up' situation whose structure is excunined in some détail and considered from various perspectives in the ensuing.