Convex Optimization Algorithms and Statistical Bounds for Learning Structured Models

Thesis (Ph.D.)--University of Washington, 2016-08

Likelihood-Based Inference for Partially Observed Multi-Type Markov Branching Processes

Thesis (Ph.D.)--University of Washington, 2016-08

Tensor Completion for Multidimensional Inverse Problems with Applications to Magnetic Resonance Relaxometry

This thesis deals with tensor completion for the solution of multidimensional inverse problems. We study the problem of reconstructing an approximately low rank tensor from a small number of noisy linear measurements. New recovery guarantees, numerical algorithms, non-uniform sampling strategies, and parameter selection algorithms are developed. We derive a fixed point continuation algorithm for tensor completion and prove its convergence. A restricted isometry property (RIP) based tensor recovery guarantee is proved. Probabilistic recovery guarantees are obtained for sub-Gaussian measurement operators and for measurements obtained by non-uniform sampling from a Parseval tight frame. We show how tensor completion can be used to solve multidimensional inverse problems arising in NMR relaxometry. Algorithms are developed for regularization parameter selection, including accelerated k-fold cross-validation and generalized cross-validation. These methods are validated on experimental and simulated data. We also derive condition number estimates for nonnegative least squares problems. Tensor recovery promises to significantly accelerate N-dimensional NMR relaxometry and related experiments, enabling previously impractical experiments. Our methods could also be applied to other inverse problems arising in machine learning, image processing, signal processing, computer vision, and other fields.

Hierarchical Reconstruction Method for Solving Ill-posed Linear Inverse Problems

We present a detailed analysis of the application of a multi-scale Hierarchical Reconstruction method for solving a family of ill-posed linear inverse problems. When the observations on the unknown quantity of interest and the observation operators are known, these inverse problems are concerned with the recovery of the unknown from its observations. Although the observation operators we consider are linear, they are inevitably ill-posed in various ways. We recall in this context the classical Tikhonov regularization method with a stabilizing function which targets the specific ill-posedness from the observation operators and preserves desired features of the unknown. Having studied the mechanism of the Tikhonov regularization, we propose a multi-scale generalization to the Tikhonov regularization method, so-called the Hierarchical Reconstruction (HR) method. First introduction of the HR method can be traced back to the Hierarchical Decomposition method in Image Processing. The HR method successively extracts information from the previous hierarchical residual to the current hierarchical term at a finer hierarchical scale. As the sum of all the hierarchical terms, the hierarchical sum from the HR method provides an reasonable approximate solution to the unknown, when the observation matrix satisfies certain conditions with specific stabilizing functions. When compared to the Tikhonov regularization method on solving the same inverse problems, the HR method is shown to be able to decrease the total number of iterations, reduce the approximation error, and offer self control of the approximation distance between the hierarchical sum and the unknown, thanks to using a ladder of finitely many hierarchical scales. We report numerical experiments supporting our claims on these advantages the HR method has over the Tikhonov regularization method.

Sparse Signal Representation in Digital and Biological Systems

Theories of sparse signal representation, wherein a signal is decomposed as the sum of a small number of constituent elements, play increasing roles in both mathematical signal processing and neuroscience. This happens despite the differences between signal models in the two domains. After reviewing preliminary material on sparse signal models, I use work on compressed sensing for the electron tomography of biological structures as a target for exploring the efficacy of sparse signal reconstruction in a challenging application domain. My research in this area addresses a topic of keen interest to the biological microscopy community, and has resulted in the development of tomographic reconstruction software which is competitive with the state of the art in its field. Moving from the linear signal domain into the nonlinear dynamics of neural encoding, I explain the sparse coding hypothesis in neuroscience and its relationship with olfaction in locusts. I implement a numerical ODE model of the activity of neural populations responsible for sparse odor coding in locusts as part of a project involving offset spiking in the Kenyon cells. I also explain the validation procedures we have devised to help assess the model's similarity to the biology. The thesis concludes with the development of a new, simplified model of locust olfactory network activity, which seeks with some success to explain statistical properties of the sparse coding processes carried out in the network.

Intelligent Sensor Systems for Structural Health Monitoring Applications

The convergence between the recent developments in sensing technologies, data science, signal processing and advanced modelling has fostered a new paradigm to the Structural Health Monitoring (SHM) of engineered structures, which is the one based on intelligent sensors, i.e., embedded devices capable of stream processing data and/or performing structural inference in a self-contained and near-sensor manner. To efficiently exploit these intelligent sensor units for full-scale structural assessment, a joint effort is required to deal with instrumental aspects related to signal acquisition, conditioning and digitalization, and those pertaining to data management, data analytics and information sharing. In this framework, the main goal of this Thesis is to tackle the multi-faceted nature of the monitoring process, via a full-scale optimization of the hardware and software resources involved by the {SHM} system. The pursuit of this objective has required the investigation of both: i) transversal aspects common to multiple application domains at different abstraction levels (such as knowledge distillation, networking solutions, microsystem {HW} architectures), and ii) the specificities of the monitoring methodologies (vibrations, guided waves, acoustic emission monitoring). The key tools adopted in the proposed monitoring frameworks belong to the embedded signal processing field: namely, graph signal processing, compressed sensing, ARMA System Identification, digital data communication and TinyML.

Antipodal random sequences with prescribed second-order statistics: application to Compressive Sensing and UWB system based on DS-CDMA

It is usual to hear a strange short sentence: «Random is better than...». Why is randomness a good solution to a certain engineering problem? There are many possible answers, and all of them are related to the considered topic. In this thesis I will discuss about two crucial topics that take advantage by randomizing some waveforms involved in signals manipulations. In particular, advantages are guaranteed by shaping the second order statistic of antipodal sequences involved in an intermediate signal processing stages. The first topic is in the area of analog-to-digital conversion, and it is named Compressive Sensing (CS). CS is a novel paradigm in signal processing that tries to merge signal acquisition and compression at the same time. Consequently it allows to direct acquire a signal in a compressed form. In this thesis, after an ample description of the CS methodology and its related architectures, I will present a new approach that tries to achieve high compression by design the second order statistics of a set of additional waveforms involved in the signal acquisition/compression stage. The second topic addressed in this thesis is in the area of communication system, in particular I focused the attention on ultra-wideband (UWB) systems. An option to produce and decode UWB signals is direct-sequence spreading with multiple access based on code division (DS-CDMA). Focusing on this methodology, I will address the coexistence of a DS-CDMA system with a narrowband interferer. To do so, I minimize the joint effect of both multiple access (MAI) and narrowband (NBI) interference on a simple matched filter receiver. I will show that, when spreading sequence statistical properties are suitably designed, performance improvements are possible with respect to a system exploiting chaos-based sequences minimizing MAI only.

Long wave theory for experimental devices with compressed/expanded surfactant monolayers

Surfactant monolayers are of interest in a variety of phenomena, including thin film dynamics and the formation and dynamics of foams. Measurement of surface properties has received a continuous attention and requires good theoretical models to extract the relevant physico- chemical information from experimental data. A common experimental set up consists in a shallow liquid layer whose free surface is slowly com- pressed/expanded in periodic fashion by moving two slightly immersed solid barriers, which varies the free surface area and thus the surfactant concentration. The simplest theory ignores the fluid dynamics in the bulk fluid, assuming spatially uniform surfactant concentration, which requires quite small forcing frequencies and provides reversible dynamics in the compression/expansion cycles. Sometimes, it is not clear whether depar- ture from reversibility is due to non-equilibrium effects or to the ignored fluid dynamics. Here we present a long wave theory that takes the fluid dynamics and the symmetries of the problem into account. In particular, the validity of the spatially-uniform-surfactant-concentration assumption is established and a nonlinear diffusion equation is derived. This allows for calculating spatially nonuniform monolayer dynamics and uncovering the physical mechanisms involved in the surfactant behavior. Also, this analysis can be considered a good means for extracting more relevant information from each experimental run.

Implementation of the Heated Pulsed Theory using actively heated fiber optics: measurements of soil volumetric water content and soil volumetric heat capacity

Existe una creciente necesidad de hacer el mejor uso del agua para regadío. Una alternativa eficiente consiste en la monitorización del contenido volumétrico de agua (θ), utilizando sensores de humedad. A pesar de existir una gran diversidad de sensores y tecnologías disponibles, actualmente ninguna de ellas permite obtener medidas distribuidas en perfiles verticales de un metro y en escalas laterales de 0.1-1,000 m. En este sentido, es necesario buscar tecnologías alternativas que sirvan de puente entre las medidas puntuales y las escalas intermedias. Esta tesis doctoral se basa en el uso de Fibra Óptica (FO) con sistema de medida de temperatura distribuida (DTS), una tecnología alternativa de reciente creación que ha levantado gran expectación en las últimas dos décadas. Específicamente utilizamos el método de fibra calentada, en inglés Actively Heated Fiber Optic (AHFO), en la cual los cables de Fibra Óptica se utilizan como sondas de calor mediante la aplicación de corriente eléctrica a través de la camisa de acero inoxidable, o de un conductor eléctrico simétricamente posicionado, envuelto, alrededor del haz de fibra óptica. El uso de fibra calentada se basa en la utilización de la teoría de los pulsos de calor, en inglés Heated Pulsed Theory (HPP), por la cual el conductor se aproxima a una fuente de calor lineal e infinitesimal que introduce calor en el suelo. Mediante el análisis del tiempo de ocurrencia y magnitud de la respuesta térmica ante un pulso de calor, es posible estimar algunas propiedades específicas del suelo, tales como el contenido de humedad, calor específico (C) y conductividad térmica. Estos parámetros pueden ser estimados utilizando un sensor de temperatura adyacente a la sonda de calor [método simple, en inglés single heated pulsed probes (SHPP)], ó a una distancia radial r [método doble, en inglés dual heated pulsed probes (DHPP)]. Esta tesis doctoral pretende probar la idoneidad de los sistemas de fibra óptica calentada para la aplicación de la teoría clásica de sondas calentadas. Para ello, se desarrollarán dos sistemas FO-DTS. El primero se sitúa en un campo agrícola de La Nava de Arévalo (Ávila, España), en el cual se aplica la teoría SHPP para estimar θ. El segundo sistema se desarrolla en laboratorio y emplea la teoría DHPP para medir tanto θ como C. La teoría SHPP puede ser implementada con fibra óptica calentada para obtener medidas distribuidas de θ, mediante la utilización de sistemas FO-DTS y el uso de curvas de calibración específicas para cada suelo. Sin embargo, la mayoría de aplicaciones AHFO se han desarrollado exclusivamente en laboratorio utilizando medios porosos homogéneos. En esta tesis se utiliza el programa Hydrus 2D/3D para definir tales curvas de calibración. El modelo propuesto es validado en un segmento de cable enterrado en una instalación de fibra óptica y es capaz de predecir la respuesta térmica del suelo en puntos concretos de la instalación una vez que las propiedades físicas y térmicas de éste son definidas. La exactitud de la metodología para predecir θ frente a medidas puntuales tomadas con sensores de humedad comerciales fue de 0.001 a 0.022 m3 m-3 La implementación de la teoría DHPP con AHFO para medir C y θ suponen una oportunidad sin precedentes para aplicaciones medioambientales. En esta tesis se emplean diferentes combinaciones de cables y fuentes emisoras de calor, que se colocan en paralelo y utilizan un rango variado de espaciamientos, todo ello en el laboratorio. La amplitud de la señal y el tiempo de llegada se han observado como funciones del calor específico del suelo. Medidas de C, utilizando esta metodología y ante un rango variado de contenidos de humedad, sugirieron la idoneidad del método, aunque también se observaron importantes errores en contenidos bajos de humedad de hasta un 22%. La mejora del método requerirá otros modelos más precisos que tengan en cuenta el diámetro del cable, así como la posible influencia térmica del mismo. ABSTRACT There is an increasing need to make the most efficient use of water for irrigation. A good approach to make irrigation as efficient as possible is to monitor soil water content (θ) using soil moisture sensors. Although, there is a broad range of different sensors and technologies, currently, none of them can practically and accurately provide vertical and lateral moisture profiles spanning 0-1 m depth and 0.1-1,000 m lateral scales. In this regard, further research to fulfill the intermediate scale and to bridge single-point measurement with the broaden scales is still needed. This dissertation is based on the use of Fiber Optics with Distributed Temperature Sensing (FO-DTS), a novel approach which has been receiving growing interest in the last two decades. Specifically, we employ the so called Actively Heated Fiber Optic (AHFO) method, in which FO cables are employed as heat probe conductors by applying electricity to the stainless steel armoring jacket or an added conductor symmetrically positioned (wrapped) about the FO cable. AHFO is based on the classic Heated Pulsed Theory (HPP) which usually employs a heat probe conductor that approximates to an infinite line heat source which injects heat into the soil. Observation of the timing and magnitude of the thermal response to the energy input provide enough information to derive certain specific soil thermal characteristics such as the soil heat capacity, soil thermal conductivity or soil water content. These parameters can be estimated by capturing the soil thermal response (using a thermal sensor) adjacent to the heat source (the heating and the thermal sources are mounted together in the so called single heated pulsed probe (SHPP)), or separated at a certain distance, r (dual heated pulsed method (DHPP) This dissertation aims to test the feasibility of heated fiber optics to implement the HPP theory. Specifically, we focus on measuring soil water content (θ) and soil heat capacity (C) by employing two types of FO-DTS systems. The first one is located in an agricultural field in La Nava de Arévalo (Ávila, Spain) and employ the SHPP theory to estimate θ. The second one is developed in the laboratory using the procedures described in the DHPP theory, and focuses on estimating both C and θ. The SHPP theory can be implemented with actively heated fiber optics (AHFO) to obtain distributed measurements of soil water content (θ) by using reported soil thermal responses in Distributed Temperature Sensing (DTS) and with a soil-specific calibration relationship. However, most reported AHFO applications have been calibrated under laboratory homogeneous soil conditions, while inexpensive efficient calibration procedures useful in heterogeneous soils are lacking. In this PhD thesis, we employ the Hydrus 2D/3D code to define these soil-specific calibration curves. The model is then validated at a selected FO transect of the DTS installation. The model was able to predict the soil thermal response at specific locations of the fiber optic cable once the surrounding soil hydraulic and thermal properties were known. Results using electromagnetic moisture sensors at the same specific locations demonstrate the feasibility of the model to detect θ within an accuracy of 0.001 to 0.022 m3 m-3. Implementation of the Dual Heated Pulsed Probe (DPHP) theory for measurement of volumetric heat capacity (C) and water content (θ) with Distributed Temperature Sensing (DTS) heated fiber optic (FO) systems presents an unprecedented opportunity for environmental monitoring. We test the method using different combinations of FO cables and heat sources at a range of spacings in a laboratory setting. The amplitude and phase-shift in the heat signal with distance was found to be a function of the soil volumetric heat capacity (referred, here, to as Cs). Estimations of Cs at a range of θ suggest feasibility via responsiveness to the changes in θ (we observed a linear relationship in all FO combinations), though observed bias with decreasing soil water contents (up to 22%) was also reported. Optimization will require further models to account for the finite radius and thermal influence of the FO cables, employed here as “needle probes”. Also, consideration of the range of soil conditions and cable spacing and jacket configurations, suggested here to be valuable subjects of further study and development.

Dynamic Behavior of the Jahn-Teller distorted Cu(H2O)(6)(2+) ion in Cu2+ doped Cs-2[Zn(H2O)(6)](ZrF6)(2) and the crystal structure of the host lattice

The temperature dependence of the X- and Q-band EPR spectra of Cs-2[Zn(H2O)(6)](ZrF6)(2) containing similar to1% Cu2+ is reported. All three molecular g-values vary with temperature, and their behavior is interpreted using a model in which the potential surface of the Jahn-Teller distorted Cu(H2O)(6)(2+) ion is perturbed by an orthorhombic strain induced by interactions with the surrounding lattice. The strain parameters are significantly smaller than those reported previously for the Cu(H2O)(6)(2+) ion in similar lattices. The temperature dependence of the two higher g-values suggests that in the present compound the lattice interactions change slightly with temperature. The crystal structure of the Cs-2[Zn(H2O)(6)](ZrF6)(2) host is reported, and the geometry of the Zn(H2O)(6)(2+) ion is correlated with lattice strain parameters derived from the EPR spectrum of the guest Cu2+ complex.

Evaluating two morningness scales with item response theory

Using a student sample (n = 692) and an organization sample (n = 180), we scrutinized two morning-evening orientation scales using item response theory (IRT) methods. We used IRT to compare the measurement precision of the Composite Scale (CS) and the Early/Late Preferences Scale (PS). The CS had slightly higher measurement precision at all ranges of orientations, except for extreme morning and evening orientations for which the PS had slightly higher precision. IRT item-level statistics were also computed to try to understand how morning-orientation items functioned. Items that asked questions about morning activities tended to be more discriminating indicators of morning-orientation than items that asked about evening or peak performance activities. Items that involved unpleasant activities were less frequently endorsed than items that involved neutral or enjoyable activities. Implications for measurement of morning-evening orientation are discussed. (C) 2002 Elsevier Science Ltd. All rights reserved.

GPU implementation of a hyperspectral coded aperture algorithm for compressive sensing

This paper presents a new parallel implementation of a previously hyperspectral coded aperture (HYCA) algorithm for compressive sensing on graphics processing units (GPUs). HYCA method combines the ideas of spectral unmixing and compressive sensing exploiting the high spatial correlation that can be observed in the data and the generally low number of endmembers needed in order to explain the data. The proposed implementation exploits the GPU architecture at low level, thus taking full advantage of the computational power of GPUs using shared memory and coalesced accesses to memory. The proposed algorithm is evaluated not only in terms of reconstruction error but also in terms of computational performance using two different GPU architectures by NVIDIA: GeForce GTX 590 and GeForce GTX TITAN. Experimental results using real data reveals signficant speedups up with regards to serial implementation.

Parallel hyperspectral coded aperture for compressive sensing on GPUs

The application of compressive sensing (CS) to hyperspectral images is an active area of research over the past few years, both in terms of the hardware and the signal processing algorithms. However, CS algorithms can be computationally very expensive due to the extremely large volumes of data collected by imaging spectrometers, a fact that compromises their use in applications under real-time constraints. This paper proposes four efficient implementations of hyperspectral coded aperture (HYCA) for CS, two of them termed P-HYCA and P-HYCA-FAST and two additional implementations for its constrained version (CHYCA), termed P-CHYCA and P-CHYCA-FAST on commodity graphics processing units (GPUs). HYCA algorithm exploits the high correlation existing among the spectral bands of the hyperspectral data sets and the generally low number of endmembers needed to explain the data, which largely reduces the number of measurements necessary to correctly reconstruct the original data. The proposed P-HYCA and P-CHYCA implementations have been developed using the compute unified device architecture (CUDA) and the cuFFT library. Moreover, this library has been replaced by a fast iterative method in the P-HYCA-FAST and P-CHYCA-FAST implementations that leads to very significant speedup factors in order to achieve real-time requirements. The proposed algorithms are evaluated not only in terms of reconstruction error for different compressions ratios but also in terms of computational performance using two different GPU architectures by NVIDIA: 1) GeForce GTX 590; and 2) GeForce GTX TITAN. Experiments are conducted using both simulated and real data revealing considerable acceleration factors and obtaining good results in the task of compressing remotely sensed hyperspectral data sets.

Spectrum sensing through software defined radio

Dissertação apresentada para obtenção do Grau de Mestre em Engenharia Electrotécnica e de Computadores, pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

Helium in polygonal nanopores at zero temperature: Density functional theory calculations

We investigate adsorption of helium in nanoscopic polygonal pores at zero temperature using a finite-range density functional theory. The adsorption potential is computed by means of a technique denoted as the elementary source method. We analyze a rhombic pore with Cs walls, where we show the existence of multiple interfacial configurations at some linear densities, which correspond to metastable states. Shape transitions and hysterectic loops appear in patterns which are richer and more complex than in a cylindrical tube with the same transverse area.