Computing the Stochastic Complexity of Simple Probabilistic Graphical Models

Minimum Description Length (MDL) is an information-theoretic principle that can be used for model selection and other statistical inference tasks. There are various ways to use the principle in practice. One theoretically valid way is to use the normalized maximum likelihood (NML) criterion. Due to computational difficulties, this approach has not been used very often. This thesis presents efficient floating-point algorithms that make it possible to compute the NML for multinomial, Naive Bayes and Bayesian forest models. None of the presented algorithms rely on asymptotic analysis and with the first two model classes we also discuss how to compute exact rational number solutions.

Computationally Efficient Methods for MDL-Optimal Density Estimation and Data Clustering

The Minimum Description Length (MDL) principle is a general, well-founded theoretical formalization of statistical modeling. The most important notion of MDL is the stochastic complexity, which can be interpreted as the shortest description length of a given sample of data relative to a model class. The exact definition of the stochastic complexity has gone through several evolutionary steps. The latest instantation is based on the so-called Normalized Maximum Likelihood (NML) distribution which has been shown to possess several important theoretical properties. However, the applications of this modern version of the MDL have been quite rare because of computational complexity problems, i.e., for discrete data, the definition of NML involves an exponential sum, and in the case of continuous data, a multi-dimensional integral usually infeasible to evaluate or even approximate accurately. In this doctoral dissertation, we present mathematical techniques for computing NML efficiently for some model families involving discrete data. We also show how these techniques can be used to apply MDL in two practical applications: histogram density estimation and clustering of multi-dimensional data.

Discriminative learning of Bayesian networks via factorized conditional log-likelihood

We propose an efficient and parameter-free scoring criterion, the factorized conditional log-likelihood (ˆfCLL), for learning Bayesian network classifiers. The proposed score is an approximation of the conditional log-likelihood criterion. The approximation is devised in order to guarantee decomposability over the network structure, as well as efficient estimation of the optimal parameters, achieving the same time and space complexity as the traditional log-likelihood scoring criterion. The resulting criterion has an information-theoretic interpretation based on interaction information, which exhibits its discriminative nature. To evaluate the performance of the proposed criterion, we present an empirical comparison with state-of-the-art classifiers. Results on a large suite of benchmark data sets from the UCI repository show that ˆfCLL-trained classifiers achieve at least as good accuracy as the best compared classifiers, using significantly less computational resources.

Migration-Development Nexus in the Malawian Health Sector: European Union Blue Card – circular migration for development?

The European Union has agreed on implementing the Policy Coherence for Development (PCD) principle in all policy sectors that are likely to have a direct impact on developing countries. This is in order to take account of and support the EU development cooperation objectives and the achievement of the internationally agreed Millennium Development Goals. The common EU migration policy and the newly introduced EU Blue Card directive present an example of the implementation of the principle in practice: the directive is not only designed to respond to the occurring EU labour demand by attracting highly skilled third-country professionals, but is also intended to contribute to the development objectives of the migrant-sending developing countries, primarily through the tool of circular migration and the consequent skills transfers. My objective in this study is to assess such twofold role of the EU Blue Card and to explore the idea that migration could be harnessed for the benefit of development in conformity with the notion that the two form a positive nexus. Seeing that the EU Blue Card fails to differentiate the most vulnerable countries and sectors from those that are in a better position to take advantage of the global migration flows, the developmental consequences of the directive must be accounted for even in the most severe settings. Accordingly, my intention is to question whether circular migration, as claimed, could address the problem of brain drain in the Malawian health sector, which has witnessed an excessive outflow of its professionals to the UK during the past decade. In order to assess the applicability, likelihood and relevance of circular migration and consequent skills transfers for development in the Malawian context, a field study of a total of 23 interviews with local health professionals was carried out in autumn 2010. The selected approach not only allows me to introduce a developing country perspective to the on-going discussion at the EU level, but also enables me to assess the development dimension of the EU Blue Card and the intended PCD principle through a local lens. Thus these interviews and local viewpoints are at the very heart of this study. Based on my findings from the field, the propensity of the EU Blue Card to result in circular migration and to address the persisting South-North migratory flows as well as the relevance of skills transfers can be called to question. This is as due to the bias in its twofold role the directive overlooks the importance of the sending country circumstances, which are known to determine any developmental outcomes of migration, and assumes that circular migration alone could bring about immediate benefits. Without initial emphasis on local conditions, however, positive outcomes for vulnerable countries such as Malawi are ever more distant. Indeed it seems as if the EU internal interests in migration policy forbid the fulfilment of the PCD principle and diminish the attempt to harness migration for development to bare rhetoric.