Regularization for sparsity in statistical analysis and machine learning

Pragmatism is the leading motivation of regularization. We can understand regularization as a modification of the maximum-likelihood estimator so that a reasonable answer could be given in an unstable or ill-posed situation. To mention some typical examples, this happens when fitting parametric or non-parametric models with more parameters than data or when estimating large covariance matrices. Regularization is usually used, in addition, to improve the bias-variance tradeoff of an estimation. Then, the definition of regularization is quite general, and, although the introduction of a penalty is probably the most popular type, it is just one out of multiple forms of regularization. In this dissertation, we focus on the applications of regularization for obtaining sparse or parsimonious representations, where only a subset of the inputs is used. A particular form of regularization, L1-regularization, plays a key role for reaching sparsity. Most of the contributions presented here revolve around L1-regularization, although other forms of regularization are explored (also pursuing sparsity in some sense). In addition to present a compact review of L1-regularization and its applications in statistical and machine learning, we devise methodology for regression, supervised classification and structure induction of graphical models. Within the regression paradigm, we focus on kernel smoothing learning, proposing techniques for kernel design that are suitable for high dimensional settings and sparse regression functions. We also present an application of regularized regression techniques for modeling the response of biological neurons. Supervised classification advances deal, on the one hand, with the application of regularization for obtaining a na¨ıve Bayes classifier and, on the other hand, with a novel algorithm for brain-computer interface design that uses group regularization in an efficient manner. Finally, we present a heuristic for inducing structures of Gaussian Bayesian networks using L1-regularization as a filter. El pragmatismo es la principal motivación de la regularización. Podemos entender la regularización como una modificación del estimador de máxima verosimilitud, de tal manera que se pueda dar una respuesta cuando la configuración del problema es inestable. A modo de ejemplo, podemos mencionar el ajuste de modelos paramétricos o no paramétricos cuando hay más parámetros que casos en el conjunto de datos, o la estimación de grandes matrices de covarianzas. Se suele recurrir a la regularización, además, para mejorar el compromiso sesgo-varianza en una estimación. Por tanto, la definición de regularización es muy general y, aunque la introducción de una función de penalización es probablemente el método más popular, éste es sólo uno de entre varias posibilidades. En esta tesis se ha trabajado en aplicaciones de regularización para obtener representaciones dispersas, donde sólo se usa un subconjunto de las entradas. En particular, la regularización L1 juega un papel clave en la búsqueda de dicha dispersión. La mayor parte de las contribuciones presentadas en la tesis giran alrededor de la regularización L1, aunque también se exploran otras formas de regularización (que igualmente persiguen un modelo disperso). Además de presentar una revisión de la regularización L1 y sus aplicaciones en estadística y aprendizaje de máquina, se ha desarrollado metodología para regresión, clasificación supervisada y aprendizaje de estructura en modelos gráficos. Dentro de la regresión, se ha trabajado principalmente en métodos de regresión local, proponiendo técnicas de diseño del kernel que sean adecuadas a configuraciones de alta dimensionalidad y funciones de regresión dispersas. También se presenta una aplicación de las técnicas de regresión regularizada para modelar la respuesta de neuronas reales. Los avances en clasificación supervisada tratan, por una parte, con el uso de regularización para obtener un clasificador naive Bayes y, por otra parte, con el desarrollo de un algoritmo que usa regularización por grupos de una manera eficiente y que se ha aplicado al diseño de interfaces cerebromáquina. Finalmente, se presenta una heurística para inducir la estructura de redes Bayesianas Gaussianas usando regularización L1 a modo de filtro.

Environmental variables modeling based on remote sensing data using time series analysis: forestry and agricultural applications

En la actualidad, el seguimiento de la dinámica de los procesos medio ambientales está considerado como un punto de gran interés en el campo medioambiental. La cobertura espacio temporal de los datos de teledetección proporciona información continua con una alta frecuencia temporal, permitiendo el análisis de la evolución de los ecosistemas desde diferentes escalas espacio-temporales. Aunque el valor de la teledetección ha sido ampliamente probado, en la actualidad solo existe un número reducido de metodologías que permiten su análisis de una forma cuantitativa. En la presente tesis se propone un esquema de trabajo para explotar las series temporales de datos de teledetección, basado en la combinación del análisis estadístico de series de tiempo y la fenometría. El objetivo principal es demostrar el uso de las series temporales de datos de teledetección para analizar la dinámica de variables medio ambientales de una forma cuantitativa. Los objetivos específicos son: (1) evaluar dichas variables medio ambientales y (2) desarrollar modelos empíricos para predecir su comportamiento futuro. Estos objetivos se materializan en cuatro aplicaciones cuyos objetivos específicos son: (1) evaluar y cartografiar estados fenológicos del cultivo del algodón mediante análisis espectral y fenometría, (2) evaluar y modelizar la estacionalidad de incendios forestales en dos regiones bioclimáticas mediante modelos dinámicos, (3) predecir el riesgo de incendios forestales a nivel pixel utilizando modelos dinámicos y (4) evaluar el funcionamiento de la vegetación en base a la autocorrelación temporal y la fenometría. Los resultados de esta tesis muestran la utilidad del ajuste de funciones para modelizar los índices espectrales AS1 y AS2. Los parámetros fenológicos derivados del ajuste de funciones permiten la identificación de distintos estados fenológicos del cultivo del algodón. El análisis espectral ha demostrado, de una forma cuantitativa, la presencia de un ciclo en el índice AS2 y de dos ciclos en el AS1 así como el comportamiento unimodal y bimodal de la estacionalidad de incendios en las regiones mediterránea y templada respectivamente. Modelos autorregresivos han sido utilizados para caracterizar la dinámica de la estacionalidad de incendios y para predecir de una forma muy precisa el riesgo de incendios forestales a nivel pixel. Ha sido demostrada la utilidad de la autocorrelación temporal para definir y caracterizar el funcionamiento de la vegetación a nivel pixel. Finalmente el concepto “Optical Functional Type” ha sido definido, donde se propone que los pixeles deberían ser considerados como unidades temporales y analizados en función de su dinámica temporal. ix SUMMARY A good understanding of land surface processes is considered as a key subject in environmental sciences. The spatial-temporal coverage of remote sensing data provides continuous observations with a high temporal frequency allowing the assessment of ecosystem evolution at different temporal and spatial scales. Although the value of remote sensing time series has been firmly proved, only few time series methods have been developed for analyzing this data in a quantitative and continuous manner. In the present dissertation a working framework to exploit Remote Sensing time series is proposed based on the combination of Time Series Analysis and phenometric approach. The main goal is to demonstrate the use of remote sensing time series to analyze quantitatively environmental variable dynamics. The specific objectives are (1) to assess environmental variables based on remote sensing time series and (2) to develop empirical models to forecast environmental variables. These objectives have been achieved in four applications which specific objectives are (1) assessing and mapping cotton crop phenological stages using spectral and phenometric analyses, (2) assessing and modeling fire seasonality in two different ecoregions by dynamic models, (3) forecasting forest fire risk on a pixel basis by dynamic models, and (4) assessing vegetation functioning based on temporal autocorrelation and phenometric analysis. The results of this dissertation show the usefulness of function fitting procedures to model AS1 and AS2. Phenometrics derived from function fitting procedure makes it possible to identify cotton crop phenological stages. Spectral analysis has demonstrated quantitatively the presence of one cycle in AS2 and two in AS1 and the unimodal and bimodal behaviour of fire seasonality in the Mediterranean and temperate ecoregions respectively. Autoregressive models has been used to characterize the dynamics of fire seasonality in two ecoregions and to forecasts accurately fire risk on a pixel basis. The usefulness of temporal autocorrelation to define and characterized land surface functioning has been demonstrated. And finally the “Optical Functional Types” concept has been proposed, in this approach pixels could be as temporal unities based on its temporal dynamics or functioning.

Modelling of spectral properties and population kinetics studies of inertial fusión and laboratory astrophysical plasmas

Fundamental research and modelling in plasma atomic physics continue to be essential for providing basic understanding of many different topics relevant to high-energy-density plasmas. The Atomic Physics Group at the Institute of Nuclear Fusion has accumulated experience over the years in developing a collection of computational models and tools for determining the atomic energy structure, ionization balance and radiative properties of, mainly, inertial fusion and laser-produced plasmas in a variety of conditions. In this work, we discuss some of the latest advances and results of our research, with emphasis on inertial fusion and laboratory-astrophysical applications.

Analysis of spontaneous MEG activity in mild cognitive impairment and Alzheimer's disease using spectral entropies and statistical complexity measures

Alzheimer's disease (AD) is the most common cause of dementia. Over the last few years, a considerable effort has been devoted to exploring new biomarkers. Nevertheless, a better understanding of brain dynamics is still required to optimize therapeutic strategies. In this regard, the characterization of mild cognitive impairment (MCI) is crucial, due to the high conversion rate from MCI to AD. However, only a few studies have focused on the analysis of magnetoencephalographic (MEG) rhythms to characterize AD and MCI. In this study, we assess the ability of several parameters derived from information theory to describe spontaneous MEG activity from 36 AD patients, 18 MCI subjects and 26 controls. Three entropies (Shannon, Tsallis and Rényi entropies), one disequilibrium measure (based on Euclidean distance ED) and three statistical complexities (based on Lopez Ruiz–Mancini–Calbet complexity LMC) were used to estimate the irregularity and statistical complexity of MEG activity. Statistically significant differences between AD patients and controls were obtained with all parameters (p < 0.01). In addition, statistically significant differences between MCI subjects and controls were achieved by ED and LMC (p < 0.05). In order to assess the diagnostic ability of the parameters, a linear discriminant analysis with a leave-one-out cross-validation procedure was applied. The accuracies reached 83.9% and 65.9% to discriminate AD and MCI subjects from controls, respectively. Our findings suggest that MCI subjects exhibit an intermediate pattern of abnormalities between normal aging and AD. Furthermore, the proposed parameters provide a new description of brain dynamics in AD and MCI.

Ab initio calculations of Ca III Stark broadening parameters,transition probabilities and radiative lifetimes

We have determined matrix elements for all experimental configurations of Ca III, including the 3s3p63d configuration. These values have been obtained using intermediate coupling (IC). For these IC calculations, we have used the standard method of least-squares fitting from the experimental energy levels, using the computer code developed by Robert Cowan. In this paper, using these matrix elements, we report the calculated values of the Ca III Stark widths and shifts for 148 spectral lines, of 56 Ca III spectral line transition probabilities and of eight radiative lifetimes of Ca III levels. The Stark widths and shifts, calculated using the Griem semi-empirical approach, correspond to the spectral lines of Ca III and are presented for an electron density of 1017 cm?3 and temperatures T = 1.0?10.0 (×104 K). The theoretical trends of the Stark broadening parameter versus the temperature are presented for transitions that are of astrophysical interest. There is good agreement between our calculations, for transition probabilities and radiative lifetimes, and the experimental values presented in the literature. We have not been able to find any values for the Stark parameters in the references.

Uncertainty in ellipse fitting using a flatbed scanner: development and experimental verification

In the field of dimensional metrology, the use of optical measuring machines requires the handling of a large number of measurement points, or scanning points, taken from the image of the measurand. The presence of correlation between these measurement points has a significant influence on the uncertainty of the result. The aim of this work is the development of an estimation procedure for the uncertainty of measurement in a geometrically elliptical shape, taking into account the correlation between the scanning points. These points are obtained from an image produced using a commercial flat bed scanner. The characteristic parameters of the ellipse (coordinates of the center, semi-axes and the angle of the semi-major axis with regard to the horizontal) are determined using a least squares fit and orthogonal distance regression. The uncertainty is estimated using the information from the auto-correlation function of the residuals and is propagated through the fitting algorithm according to the rules described in Evaluation of Measurement Data—Supplement 2 to the ‘Guide to the Expression of Uncertainty in Measurement’—Extension to any number of output quantities. By introducing the concept of cut-off length, it can be observed how it is possible to take into account the presence of the correlation in the estimation of uncertainty in a very simple way while avoiding underestimation.

Genetic algorithms and Gaussian Bayesian networks to uncover the predictive core set of bibliometric indices

The diversity of bibliometric indices today poses the challenge of exploiting the relationships among them. Our research uncovers the best core set of relevant indices for predicting other bibliometric indices. An added difficulty is to select the role of each variable, that is, which bibliometric indices are predictive variables and which are response variables. This results in a novel multioutput regression problem where the role of each variable (predictor or response) is unknown beforehand. We use Gaussian Bayesian networks to solve the this problem and discover multivariate relationships among bibliometric indices. These networks are learnt by a genetic algorithm that looks for the optimal models that best predict bibliometric data. Results show that the optimal induced Gaussian Bayesian networks corroborate previous relationships between several indices, but also suggest new, previously unreported interactions. An extended analysis of the best model illustrates that a set of 12 bibliometric indices can be accurately predicted using only a smaller predictive core subset composed of citations, g-index, q2-index, and hr-index. This research is performed using bibliometric data on Spanish full professors associated with the computer science area.

Debonding detection of FRP strengthened concrete beams by using impedance measurements and an ensemble PSO adaptive spectral model

An impedance-based midspan debonding identification method for RC beams strengthened with FRP strips is presented in this paper using piezoelectric ceramic (PZT) sensor?actuators. To reach this purpose, firstly, a two-dimensional electromechanical impedance model is proposed to predict the electrical admittance of the PZT transducer bonded to the FRP strips of an RC beam. Considering the impedance is measured in high frequencies, a spectral element model of the bonded-PZT?FRP strengthened beam is developed. This model, in conjunction with experimental measurements of PZT transducers, is used to present an updating methodology to quantitatively detect interfacial debonding of these kinds of structures. To improve the performance and accuracy of the detection algorithm in a challenging problem such as ours, the structural health monitoring approach is solved with an ensemble process based on particle of swarm. An adaptive mesh scheme has also been developed to increase the reliability in locating the area in which debonding initiates. Predictions carried out with experimental results have showed the effectiveness and potential of the proposed method to detect prematurely at its earliest stages a critical failure mode such as that due to midspan debonding of the FRP strip.

Assessment of diastolic chamber properties of the right ventricle by global fitting of pressure-volume data and conformational analysis of 3D + T echocardiographic sequences

Assessment of diastolic chamber properties of the right ventricle by global fitting of pressure-volume data and conformational analysis of 3D + T echocardiographic sequences

Determination of spectral variations by means of component cells useful for CPV rating and design

A methodology is presented to determine both the short-term and the long-term influence of the spectral variations on the performance of Multi-Junction (MJ) solar cells and Concentrating "This is the peer reviewed version of the following article: R. Núñez, C. Domínguez, S. Askins, M. Victoria, R. Herrero, I. Antón, and G. Sala, “Determination of spectral variations by means of component cells useful for CPV rating and design,” Prog. Photovolt: Res. Appl., 2015., which has been published in final form at http://onlinelibrary.wiley.com/doi/10.1002/pip.2715/full. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving [http://olabout.wiley.com/WileyCDA/Section/id-820227.html#terms]." Photovoltaic (CPV) modules. Component cells with the same optical behavior as MJ solar cells are used to characterize the spectrum. A set of parameters, namely Spectral Matching Ratios (SMRs), is used to characterize spectrally a particular Direct Normal Irradiance (DNI) by comparison to the reference spectrum (AM1.5D-ASTM-G173-03). Furthermore, the spectrally corrected DNI for a given MJ solar cell technology is defined providing a way to estimate the losses associated to the spectral variations. The last section analyzes how the spectrum evolves throughout a year in a given place and the set of SMRs representative for that location are calculated. This information can be used to maximize the energy harvested by the MJ solar cell throughout the year. As an example, three years of data recorded in Madrid shows that losses lower than 5% are expected due to current mismatch for state-of-the-art MJ solar cells.

Atmospheric parameters, spectral indexes and their relation to CPV spectral performance

Air Mass and atmosphere components (basically aerosol (AOD) and precipitable water (PW)) define the absorption of the sunlight that arrive to Earth. Radiative models such as SMARTS or MODTRAN use these parameters to generate an equivalent spectrum. However, complex and expensive instruments (as AERONET network devices) are needed to obtain AOD and PW. On the other hand, the use of isotype cells is a convenient way to characterize spectrally a place for CPV considering that they provide the photocurrent of the different internal subcells individually. Crossing data from AERONET station and a Tri-band Spectroheliometer, a model that correlates Spectral Mismatch Ratios and atmospheric parameters is proposed. Considering the amount of stations of AERONET network, this model may be used to estimate the spectral influence on energy performance of CPV systems close to all the stations worldwide.

Spectral study and classification of worldwide locations considering several multijunction solar cell technologies

Multi-junction solar cells are widely used in high-concentration photovoltaic systems (HCPV) attaining the highest efficiencies in photovoltaic energy generation. This technology is more dependent on the spectral variations of the impinging Direct Normal Irradiance (DNI) than conventional photovoltaics based on silicon solar cells and consequently demands a deeper knowledge of the solar resource characteristics. This article explores the capabilities of spectral indexes, namely, spectral matching ratios (SMR), to spectrally characterize the annual irradiation reaching a particular location on the Earth and to provide the necessary information for the spectral optimization of a MJ solar cell in that location as a starting point for CPV module spectral tuning. Additionally, the relationship between such indexes and the atmosphere parameters, such as the aerosol optical depth (AOD), precipitable water (PW), and air mass (AM), is discussed using radiative transfer models such as SMARTS to generate the spectrally-resolved DNI. The network of ground-based sun and sky-scanning radiometers AERONET (AErosol RObotic NETwork) is exploited to obtain the atmosphere parameters for a selected bunch of 34 sites worldwide. Finally, the SMR indexes are obtained for every location, and a comparative analysis is carried out for four architectures of triple junction solar cells, covering both lattice match and metamorphic technologies. The differences found among cell technologies are much less significant than among locations.

Determination of a refractive index and an extinction coefficient of standard production of CVD-graphene

The refractive index and extinction coefficient of chemical vapour deposition grown graphene are determined by ellipsometry analysis. Graphene films were grown on copper substrates and transferred as both monolayers and bilayers onto SiO2/Si substrates by using standard manufacturing procedures. The chemical nature and thickness of residual debris formed after the transfer process were elucidated using photoelectron spectroscopy. The real layered structure so deduced has been used instead of the nominal one as the input in the ellipsometry analysis of monolayer and bilayer graphene, transferred onto both native and thermal silicon oxide. The effect of these contamination layers on the optical properties of the stacked structure is noticeable both in the visible and the ultraviolet spectral regions, thus masking the graphene optical response. Finally, the use of heat treatment under a nitrogen atmosphere of the graphene-based stacked structures, as a method to reduce the water content of the sample, and its effect on the optical response of both graphene and the residual debris layer are presented. The Lorentz-Drude model proposed for the optical response of graphene fits fairly well the experimental ellipsometric data for all the analysed graphene-based stacked structures.