Characterisation of pore structure of carbon adsorbents using regularisation procedure

The constrained regularisation procedure was applied to compute the pore size distributions (PSDs, f(x)) for a variety of activated carbons using overall adsorption equation based on the combination of the Kelvin equation and the statistical adsorbed film thickness. The impact of the boundary values of relative nitrogen pressure p/p(0) was analysed on the basis of the corresponding alterations in the PSDs. Changes in microporosity and mesoporosity of activated carbons can be described adequately only when the range of p/p(0) is as wide as possible, as at a high initial p/p(0) value, the f(x) curves can be broadened with shifted maxima especially for micropores and narrow mesopores. Comparative analysis of the PSDs and the adsorption potential, adsorption energy and fractal dimension distributions gives useful information on the complete description of the adsorbent characteristics. (C) 2001 Elsevier Science B.V. All rights reserved.

L1-regularisation for ill-posed problems in variational data assimilation

We consider four-dimensional variational data assimilation (4DVar) and show that it can be interpreted as Tikhonov or L2-regularisation, a widely used method for solving ill-posed inverse problems. It is known from image restoration and geophysical problems that an alternative regularisation, namely L1-norm regularisation, recovers sharp edges better than L2-norm regularisation. We apply this idea to 4DVar for problems where shocks and model error are present and give two examples which show that L1-norm regularisation performs much better than the standard L2-norm regularisation in 4DVar.

Minimum description length, regularisation and multi-modal data

Relationships between clustering, description length, and regularisation are pointed out, motivating the introduction of a cost function with a description length interpretation and the unusual and useful property of having its minimum approximated by the densest mode of a distribution. A simple inverse kinematics example is used to demonstrate that this property can be used to select and learn one branch of a multi-valued mapping. This property is also used to develop a method for setting regularisation parameters according to the scale on which structure is exhibited in the training data. The regularisation technique is demonstrated on two real data sets, a classification problem and a regression problem.

Regularisation of mixture density networks

Mixture Density Networks are a principled method to model conditional probability density functions which are non-Gaussian. This is achieved by modelling the conditional distribution for each pattern with a Gaussian Mixture Model for which the parameters are generated by a neural network. This thesis presents a novel method to introduce regularisation in this context for the special case where the mean and variance of the spherical Gaussian Kernels in the mixtures are fixed to predetermined values. Guidelines for how these parameters can be initialised are given, and it is shown how to apply the evidence framework to mixture density networks to achieve regularisation. This also provides an objective stopping criteria that can replace the `early stopping' methods that have previously been used. If the neural network used is an RBF network with fixed centres this opens up new opportunities for improved initialisation of the network weights, which are exploited to start training relatively close to the optimum. The new method is demonstrated on two data sets. The first is a simple synthetic data set while the second is a real life data set, namely satellite scatterometer data used to infer the wind speed and wind direction near the ocean surface. For both data sets the regularisation method performs well in comparison with earlier published results. Ideas on how the constraint on the kernels may be relaxed to allow fully adaptable kernels are presented.

Regularisation of mixture density networks

Mixture Density Networks are a principled method to model conditional probability density functions which are non-Gaussian. This is achieved by modelling the conditional distribution for each pattern with a Gaussian Mixture Model for which the parameters are generated by a neural network. This thesis presents a novel method to introduce regularisation in this context for the special case where the mean and variance of the spherical Gaussian Kernels in the mixtures are fixed to predetermined values. Guidelines for how these parameters can be initialised are given, and it is shown how to apply the evidence framework to mixture density networks to achieve regularisation. This also provides an objective stopping criteria that can replace the `early stopping' methods that have previously been used. If the neural network used is an RBF network with fixed centres this opens up new opportunities for improved initialisation of the network weights, which are exploited to start training relatively close to the optimum. The new method is demonstrated on two data sets. The first is a simple synthetic data set while the second is a real life data set, namely satellite scatterometer data used to infer the wind speed and wind direction near the ocean surface. For both data sets the regularisation method performs well in comparison with earlier published results. Ideas on how the constraint on the kernels may be relaxed to allow fully adaptable kernels are presented.

Minimum description length, regularisation and multi-modal data

Conventional feed forward Neural Networks have used the sum-of-squares cost function for training. A new cost function is presented here with a description length interpretation based on Rissanen's Minimum Description Length principle. It is a heuristic that has a rough interpretation as the number of data points fit by the model. Not concerned with finding optimal descriptions, the cost function prefers to form minimum descriptions in a naive way for computational convenience. The cost function is called the Naive Description Length cost function. Finding minimum description models will be shown to be closely related to the identification of clusters in the data. As a consequence the minimum of this cost function approximates the most probable mode of the data rather than the sum-of-squares cost function that approximates the mean. The new cost function is shown to provide information about the structure of the data. This is done by inspecting the dependence of the error to the amount of regularisation. This structure provides a method of selecting regularisation parameters as an alternative or supplement to Bayesian methods. The new cost function is tested on a number of multi-valued problems such as a simple inverse kinematics problem. It is also tested on a number of classification and regression problems. The mode-seeking property of this cost function is shown to improve prediction in time series problems. Description length principles are used in a similar fashion to derive a regulariser to control network complexity.

Inverse analysis FOR two-dimensional structures using the boundary element method

Inverse analysis is currently an important subject of study in several fields of science and engineering. The identification of physical and geometric parameters using experimental measurements is required in many applications. In this work a boundary element formulation to identify boundary and interface values as well as material properties is proposed. In particular the proposed formulation is dedicated to identifying material parameters when a cohesive crack model is assumed for 2D problems. A computer code is developed and implemented using the BEM multi-region technique and regularisation methods to perform the inverse analysis. Several examples are shown to demonstrate the efficiency of the proposed model. (C) 2010 Elsevier Ltd. All rights reserved,

Prognostic modelling of breast cancer patients: a benchmark of predictive models with external validation

Dissertação apresentada para obtenção do Grau de Doutor em Engenharia Electrotécnica e de Computadores – Sistemas Digitais e Percepcionais pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

Active civic participation of immigrants in Portugal

Portugal’s historical past strongly influences the composition of the country’s immigrant population. The main third-country foreign nationals in Portugal originate traditionally from Portuguese-speaking African countries (namely Cape Verde, Angola, Guinea Bissau, and S. Tomé e Príncipe) and Brazil. In 2001, a newly created immigrant status entitled “permanence” authorization uncovered a quantitative and a qualitative change in the structure of immigrant population in Portugal. First, there was a quantitative jump from 223.602 foreigners in 2001 to 364.203 regularized foreigners in 2003. Secondly, there was a substantial qualitative shift in the composition of immigrants. The majority of the new immigrants began coming from Eastern European countries, such as Ukraine, Moldavia, Romania, and the Russian Federation. Thus, European countries outside the E.U. zone now rank second (after African countries) in their contribution of individuals to the stocks of immigrant population in Portugal. The differences between the new and traditional immigration flows are visible in the geographical distribution of immigrants and in their insertion into the labour market. While the traditional flows would congregate around the metropolitan area of Lisbon and in the Algarve, the new migratory flows tend to be more geographically dispersed and present in less urbanized areas of Portugal. In terms of insertion in the labour market, although the construction sector is still the most important industry for immigrant labour, Eastern European workers may also be found in the agriculture and manufacturing sectors. The institutional conditions that encourage immigrants’ civic participation are divided at three different levels: the state, the local, and the civil society levels. At the state level, the High Commissioner for Migrations and Ethnic Minorities is the main organizational structure along with a set of interrelated initiatives operating under specific regulatory frameworks, which act as mediators between state officials and the Portuguese civil society, and more specifically, immigrant communities. At the local level, some municipalities created consultative councils and municipal departments aiming at encouraging the participation and representation of interests from immigrant groups and association in local policies. In the civil society sphere, the main actors in Portugal spurring immigrants civic participation are immigrant associations, mainstream associations directed toward immigration topics, and unions. The legal conditions framing immigrants’ access to social housing, education, health, and social security in Portugal are also considered to be positive. Conditions restricting immigrants’ civic participation are mainly normative and include the Portuguese nationality law, the regulations shaping the political participation of immigrants, namely in what concerns their right to vote, and employment regulations restricting immigrants’ access to public administration positions. Part II of the report focuses on the active civic participation of third country immigrants. First, reasons for the lack of research on this issue in Portugal are explained. On the one hand, the recent immigration history and the more urgent needs regarding school and economic integration kept this issue out of the research spotlight. On the other hand, it was just in the beginning of the 1990s that immigrants took the very first steps toward collective mobilisation. Secondly, the literature review of Portuguese bibliography covers research on third country immigrants’ associative movement, research on local authorities’ policies and discussion about ethnic politics and political mobilisation of immigrants in Portugal. As political mobilisation of these groups has been made mainly through ethnic and/or migrant organisations, a brief history of immigrants' associative movement is given. Immigrant associations develop multiple roles, covering the social, the cultural, the economic and the political domains. Political claiming for the regularisation of illegal immigrants has been a permanent and important field of intervention since the mid-1990s. Research results reveal the com5 plex relations between ethnic mobilisation and the set of legal and institutional frameworks developed by local and national governmental authorities targeted to the incorporation of minority groups. Case studies on the Oeiras district and on the Amadora district are then presented. Conclusions underline that the most active immigrant groups are those from Cape Verde and Guinea Bissau, since these groups have constituted a higher number of ethnic associations, give priority to political claiming and present a more politicised discourse. Reflecting on the future of research on civic participation of third country immigrants in Portugal, the authors state that it would be interesting and relevant to compare the Portuguese situation with those of other European countries, with an older immigration history, and analyse how the Portuguese immigrants’ associative movement will be affected by a changing legal framework and the emergence of new opportunities within the set of structures regarding the political participation of minority groups.

Reconsidering the "policy gap":policy implementation and outcomes in Spain

By analysing entry policies and regularisation procedures in Spain from the 1990s to 2007, this article examines how the mismatch between very restrictive immigration policies and increasing foreign labour demands translated into a model of illegal migration, which in turn gave rise to the need to carry out periodical regularisation drives. This double 'policy gap' between legality and reality, and between entry policies and regularisation procedures, is explained as a policy in itself and as a way to solve in practice the apparently unsolvable dilemma between the demands for closure and the insatiable demands for foreign workers.

Structured sparsity for spatially coherent fibre orientation estimation in diffusion MRI.

We propose a novel formulation to solve the problem of intra-voxel reconstruction of the fibre orientation distribution function (FOD) in each voxel of the white matter of the brain from diffusion MRI data. The majority of the state-of-the-art methods in the field perform the reconstruction on a voxel-by-voxel level, promoting sparsity of the orientation distribution. Recent methods have proposed a global denoising of the diffusion data using spatial information prior to reconstruction, while others promote spatial regularisation through an additional empirical prior on the diffusion image at each q-space point. Our approach reconciles voxelwise sparsity and spatial regularisation and defines a spatially structured FOD sparsity prior, where the structure originates from the spatial coherence of the fibre orientation between neighbour voxels. The method is shown, through both simulated and real data, to enable accurate FOD reconstruction from a much lower number of q-space samples than the state of the art, typically 15 samples, even for quite adverse noise conditions.

Apprentissage des réseaux de neurones profonds et applications en traitement automatique de la langue naturelle

En apprentissage automatique, domaine qui consiste à utiliser des données pour apprendre une solution aux problèmes que nous voulons confier à la machine, le modèle des Réseaux de Neurones Artificiels (ANN) est un outil précieux. Il a été inventé voilà maintenant près de soixante ans, et pourtant, il est encore de nos jours le sujet d'une recherche active. Récemment, avec l'apprentissage profond, il a en effet permis d'améliorer l'état de l'art dans de nombreux champs d'applications comme la vision par ordinateur, le traitement de la parole et le traitement des langues naturelles. La quantité toujours grandissante de données disponibles et les améliorations du matériel informatique ont permis de faciliter l'apprentissage de modèles à haute capacité comme les ANNs profonds. Cependant, des difficultés inhérentes à l'entraînement de tels modèles, comme les minima locaux, ont encore un impact important. L'apprentissage profond vise donc à trouver des solutions, en régularisant ou en facilitant l'optimisation. Le pré-entraînnement non-supervisé, ou la technique du ``Dropout'', en sont des exemples. Les deux premiers travaux présentés dans cette thèse suivent cette ligne de recherche. Le premier étudie les problèmes de gradients diminuants/explosants dans les architectures profondes. Il montre que des choix simples, comme la fonction d'activation ou l'initialisation des poids du réseaux, ont une grande influence. Nous proposons l'initialisation normalisée pour faciliter l'apprentissage. Le second se focalise sur le choix de la fonction d'activation et présente le rectifieur, ou unité rectificatrice linéaire. Cette étude a été la première à mettre l'accent sur les fonctions d'activations linéaires par morceaux pour les réseaux de neurones profonds en apprentissage supervisé. Aujourd'hui, ce type de fonction d'activation est une composante essentielle des réseaux de neurones profonds. Les deux derniers travaux présentés se concentrent sur les applications des ANNs en traitement des langues naturelles. Le premier aborde le sujet de l'adaptation de domaine pour l'analyse de sentiment, en utilisant des Auto-Encodeurs Débruitants. Celui-ci est encore l'état de l'art de nos jours. Le second traite de l'apprentissage de données multi-relationnelles avec un modèle à base d'énergie, pouvant être utilisé pour la tâche de désambiguation de sens.

Probability density function estimation using orthogonal forward regression

Using the classical Parzen window estimate as the target function, the kernel density estimation is formulated as a regression problem and the orthogonal forward regression technique is adopted to construct sparse kernel density estimates. The proposed algorithm incrementally minimises a leave-one-out test error score to select a sparse kernel model, and a local regularisation method is incorporated into the density construction process to further enforce sparsity. The kernel weights are finally updated using the multiplicative nonnegative quadratic programming algorithm, which has the ability to reduce the model size further. Except for the kernel width, the proposed algorithm has no other parameters that need tuning, and the user is not required to specify any additional criterion to terminate the density construction procedure. Two examples are used to demonstrate the ability of this regression-based approach to effectively construct a sparse kernel density estimate with comparable accuracy to that of the full-sample optimised Parzen window density estimate.

A tunable radial basis function model for nonlinear system identification using particle swarm optimisation

A tunable radial basis function (RBF) network model is proposed for nonlinear system identification using particle swarm optimisation (PSO). At each stage of orthogonal forward regression (OFR) model construction, PSO optimises one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is computationally more efficient.

Nonlinear system identification using particle swarm optimisation tuned radial basis function models

A novel particle swarm optimisation (PSO) tuned radial basis function (RBF) network model is proposed for identification of non-linear systems. At each stage of orthogonal forward regression (OFR) model construction process, PSO is adopted to tune one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is often more efficient in model construction. The effectiveness of the proposed PSO aided OFR algorithm for constructing tunable node RBF models is demonstrated using three real data sets.