Regularization for sparsity in statistical analysis and machine learning

Pragmatism is the leading motivation of regularization. We can understand regularization as a modification of the maximum-likelihood estimator so that a reasonable answer could be given in an unstable or ill-posed situation. To mention some typical examples, this happens when fitting parametric or non-parametric models with more parameters than data or when estimating large covariance matrices. Regularization is usually used, in addition, to improve the bias-variance tradeoff of an estimation. Then, the definition of regularization is quite general, and, although the introduction of a penalty is probably the most popular type, it is just one out of multiple forms of regularization. In this dissertation, we focus on the applications of regularization for obtaining sparse or parsimonious representations, where only a subset of the inputs is used. A particular form of regularization, L1-regularization, plays a key role for reaching sparsity. Most of the contributions presented here revolve around L1-regularization, although other forms of regularization are explored (also pursuing sparsity in some sense). In addition to present a compact review of L1-regularization and its applications in statistical and machine learning, we devise methodology for regression, supervised classification and structure induction of graphical models. Within the regression paradigm, we focus on kernel smoothing learning, proposing techniques for kernel design that are suitable for high dimensional settings and sparse regression functions. We also present an application of regularized regression techniques for modeling the response of biological neurons. Supervised classification advances deal, on the one hand, with the application of regularization for obtaining a na¨ıve Bayes classifier and, on the other hand, with a novel algorithm for brain-computer interface design that uses group regularization in an efficient manner. Finally, we present a heuristic for inducing structures of Gaussian Bayesian networks using L1-regularization as a filter. El pragmatismo es la principal motivación de la regularización. Podemos entender la regularización como una modificación del estimador de máxima verosimilitud, de tal manera que se pueda dar una respuesta cuando la configuración del problema es inestable. A modo de ejemplo, podemos mencionar el ajuste de modelos paramétricos o no paramétricos cuando hay más parámetros que casos en el conjunto de datos, o la estimación de grandes matrices de covarianzas. Se suele recurrir a la regularización, además, para mejorar el compromiso sesgo-varianza en una estimación. Por tanto, la definición de regularización es muy general y, aunque la introducción de una función de penalización es probablemente el método más popular, éste es sólo uno de entre varias posibilidades. En esta tesis se ha trabajado en aplicaciones de regularización para obtener representaciones dispersas, donde sólo se usa un subconjunto de las entradas. En particular, la regularización L1 juega un papel clave en la búsqueda de dicha dispersión. La mayor parte de las contribuciones presentadas en la tesis giran alrededor de la regularización L1, aunque también se exploran otras formas de regularización (que igualmente persiguen un modelo disperso). Además de presentar una revisión de la regularización L1 y sus aplicaciones en estadística y aprendizaje de máquina, se ha desarrollado metodología para regresión, clasificación supervisada y aprendizaje de estructura en modelos gráficos. Dentro de la regresión, se ha trabajado principalmente en métodos de regresión local, proponiendo técnicas de diseño del kernel que sean adecuadas a configuraciones de alta dimensionalidad y funciones de regresión dispersas. También se presenta una aplicación de las técnicas de regresión regularizada para modelar la respuesta de neuronas reales. Los avances en clasificación supervisada tratan, por una parte, con el uso de regularización para obtener un clasificador naive Bayes y, por otra parte, con el desarrollo de un algoritmo que usa regularización por grupos de una manera eficiente y que se ha aplicado al diseño de interfaces cerebromáquina. Finalmente, se presenta una heurística para inducir la estructura de redes Bayesianas Gaussianas usando regularización L1 a modo de filtro.

Proceedings of the First International Conference on Principles of knowledge Representation and Reasoning /

Includes bibliographical references and index.

Principles of knowledge representation and reasoning : proceedings of the fourth international conference (KR'94), Bonn, Germany, May 24-27, 1994 /

Includes bibliographical references and index.

Principles of knowledge representation and reasoning : proceedings of the Seventh International Conference (KR2000), Breckenridge, Colorado, April 12-15 2000 /

Includes bibliographical references and index.

Principles of knowledge representation and reasoning : proceedings of the Sixth International Conference (KR '98) /

Includes bibliographical references and index.

Optimization and Machine Learning Frameworks for Complex Network Analysis

Thesis (Ph.D.)--University of Washington, 2016-06

Blood type classification using computer vision and machine learning

In emergency situations, where time for blood transfusion is reduced, the O negative blood type (the universal donor) is administrated. However, sometimes even the universal donor can cause transfusion reactions that can be fatal to the patient. As commercial systems do not allow fast results and are not suitable for emergency situations, this paper presents the steps considered for the development and validation of a prototype, able to determine blood type compatibilities, even in emergency situations. Thus it is possible, using the developed system, to administer a compatible blood type, since the first blood unit transfused. In order to increase the system’s reliability, this prototype uses different approaches to classify blood types, the first of which is based on Decision Trees and the second one based on support vector machines. The features used to evaluate these classifiers are the standard deviation values, histogram, Histogram of Oriented Gradients and fast Fourier transform, computed on different regions of interest. The main characteristics of the presented prototype are small size, lightweight, easy transportation, ease of use, fast results, high reliability and low cost. These features are perfectly suited for emergency scenarios, where the prototype is expected to be used.

A tudásreprezentáció és a tudásmegosztás információtechnológiai megoldásai a személyazonosság-menedzsment területén (Knowledge Representation and Sharing for Identity Management Domain)

A személyazonosság-menedzsment napjaink fontos kutatási területe mind elméleti, mind gyakorlati szempontból. A szakterületen megvalósuló együttműködés, elektronikus tudásáramoltatás és csere hosszú távon csak úgy képzelhető el, hogy az azonos módon történő értelmezést automatikus eszközök támogatják. A szerző cikkében azokat a kutatási tevékenységeket foglalja össze, amelyeket - felhasználva a tudásmenedzsment, a mesterséges intelligencia és az információtechnológia eszközeit - a személyazonosság-menedzsment terület fogalmi leképezésére, leírására használt fel. Kutatási célja olyan közös fogalmi bázis kialakítása volt személyazonosság-menedzsment területre, amely lehetővé teszi az őt körülvevő multidimenzionális környezet kezelését. A kutatás kapcsolódik a GUIDE kutatási projekthez is, amelynek a szerző résztvevője. ______________ Identity management is an important research field from theoretical and practical aspects as well. The task itself is not new, identification and authentication was necessary always in public administration and business life. Information Society offers new services for citizens, which dramatically change the way of administration and results additional risks and opportunities. The goal of the demonstrated research was to formulate a common basis for the identity management domain in order to support the management of the surrounding multidimensional environment. There is a need for capturing, mapping, processing knowledge concerning identity management in order to support reusability, interoperability; to help common sharing and understanding the domain and to avoid inconsistency. The paper summarizes research activities for the identification, conceptualisation and representation of domain knowledge related to identity management, using the results of knowledge management, artificial intelligence and information technology. I utilized the experiences of Guide project, in which I participate. The paper demonstrates, that domain ontologies could offer a proper solution for identity management domain conceptualisation.

Knowledge representation and evaluation an ontology-based knowledge management approach

Competition between Higher Education Institutions is increasing at an alarming rate, while changes of the surrounding environment and demands of labour market are frequent and substantial. Universities must meet the requirements of both the national and European legislation environment. The Bologna Declaration aims at providing guidelines and solutions for these problems and challenges of European Higher Education. One of its main goals is the introduction of a common framework of transparent and comparable degrees that ensures the recognition of knowledge and qualifications of citizens all across the European Union. This paper will discuss a knowledge management approach that highlights the importance of such knowledge representation tools as ontologies. The discussed ontology-based model supports the creation of transparent curricula content (Educational Ontology) and the promotion of reliable knowledge testing (Adaptive Knowledge Testing System).

Document understanding using data mining and machine learning techniques

With the explosive growth of the volume and complexity of document data (e.g., news, blogs, web pages), it has become a necessity to semantically understand documents and deliver meaningful information to users. Areas dealing with these problems are crossing data mining, information retrieval, and machine learning. For example, document clustering and summarization are two fundamental techniques for understanding document data and have attracted much attention in recent years. Given a collection of documents, document clustering aims to partition them into different groups to provide efficient document browsing and navigation mechanisms. One unrevealed area in document clustering is that how to generate meaningful interpretation for the each document cluster resulted from the clustering process. Document summarization is another effective technique for document understanding, which generates a summary by selecting sentences that deliver the major or topic-relevant information in the original documents. How to improve the automatic summarization performance and apply it to newly emerging problems are two valuable research directions. To assist people to capture the semantics of documents effectively and efficiently, the dissertation focuses on developing effective data mining and machine learning algorithms and systems for (1) integrating document clustering and summarization to obtain meaningful document clusters with summarized interpretation, (2) improving document summarization performance and building document understanding systems to solve real-world applications, and (3) summarizing the differences and evolution of multiple document sources.

Evaluating statistical and machine learning methods to predict risk of in-hospital child mortality in Uganda

Thesis (Master's)--University of Washington, 2016-08

Dimensionality hyper-reduction and machine learning for dynamical systems with varying parameters

Thesis (Ph.D.)--University of Washington, 2016-08

Combination of Evolutionary Algorithms with Experimental Design, Traditional Optimization and Machine Learning

Evolutionary algorithms alone cannot solve optimization problems very efficiently since there are many random (not very rational) decisions in these algorithms. Combination of evolutionary algorithms and other techniques have been proven to be an efficient optimization methodology. In this talk, I will explain the basic ideas of our three algorithms along this line (1): Orthogonal genetic algorithm which treats crossover/mutation as an experimental design problem, (2) Multiobjective evolutionary algorithm based on decomposition (MOEA/D) which uses decomposition techniques from traditional mathematical programming in multiobjective optimization evolutionary algorithm, and (3) Regular model based multiobjective estimation of distribution algorithms (RM-MEDA) which uses the regular property and machine learning methods for improving multiobjective evolutionary algorithms.

Innovative ways of knowledge representation and management

Knowledge is the key for success. The adequate treatment you make on data for generating knowledge can make a difference in projects, processes, and networks. Such a treatment is the main goal of two important areas: knowledger representation and management. Our aim, in this book, is collecting sorne innovative ways of representing and managing knowledge proposed by several Latin American researchers under the premise of improving knowledge.