Penalized Least Squares Fitting

* Supported by the Army Research Office under grant DAAD-19-02-10059.

Strong-weak Stackelberg Problems in Finite Dimensional Spaces

We are concerned with two-level optimization problems called strongweak Stackelberg problems, generalizing the class of Stackelberg problems in the strong and weak sense. In order to handle the fact that the considered two-level optimization problems may fail to have a solution under mild assumptions, we consider a regularization involving ε-approximate optimal solutions in the lower level problems. We prove the existence of optimal solutions for such regularized problems and present some approximation results when the parameter ǫ goes to zero. Finally, as an example, we consider an optimization problem associated to a best bound given in [2] for a system of nondifferentiable convex inequalities.

Training a Linear Neural Network with a Stable LSP Solution for Jamming Cancellation

Two jamming cancellation algorithms are developed based on a stable solution of least squares problem (LSP) provided by regularization. They are based on filtered singular value decomposition (SVD) and modifications of the Greville formula. Both algorithms allow an efficient hardware implementation. Testing results on artificial data modeling difficult real-world situations are also provided.

Semi-Smooth Newton Methods for the Time Optimal Control of Nonautonomous Ordinary Differential Equations

AMS Subj. Classification: 49J15, 49M15

A PVT-Type Algorithm for Minimizing a Nonsmooth Convex Function

2000 Mathematics Subject Classification: 90C25, 68W10, 49M37.

An efficient, approximate path-following algorithm for elastic net based nonlinear spike enhancement

Unwanted spike noise in a digital signal is a common problem in digital filtering. However, sometimes the spikes are wanted and other, superimposed, signals are unwanted, and linear, time invariant (LTI) filtering is ineffective because the spikes are wideband - overlapping with independent noise in the frequency domain. So, no LTI filter can separate them, necessitating nonlinear filtering. However, there are applications in which the noise includes drift or smooth signals for which LTI filters are ideal. We describe a nonlinear filter formulated as the solution to an elastic net regularization problem, which attenuates band-limited signals and independent noise, while enhancing superimposed spikes. Making use of known analytic solutions a novel, approximate path-following algorithm is given that provides a good, filtered output with reduced computational effort by comparison to standard convex optimization methods. Accurate performance is shown on real, noisy electrophysiological recordings of neural spikes.

Application of Regularized Discriminant Analysis

2000 Mathematics Subject Classification: 62H30, 62P99

About Strictly Hyperbolic Operators with Non-Regular Coefficients

2002 Mathematics Subject Classification: 35L15, 35L80, 35S05, 35S30

A Taxonomy of Big Data for Optimal Predictive Machine Learning and Data Mining

Big data comes in various ways, types, shapes, forms and sizes. Indeed, almost all areas of science, technology, medicine, public health, economics, business, linguistics and social science are bombarded by ever increasing flows of data begging to be analyzed efficiently and effectively. In this paper, we propose a rough idea of a possible taxonomy of big data, along with some of the most commonly used tools for handling each particular category of bigness. The dimensionality p of the input space and the sample size n are usually the main ingredients in the characterization of data bigness. The specific statistical machine learning technique used to handle a particular big data set will depend on which category it falls in within the bigness taxonomy. Large p small n data sets for instance require a different set of tools from the large n small p variety. Among other tools, we discuss Preprocessing, Standardization, Imputation, Projection, Regularization, Penalization, Compression, Reduction, Selection, Kernelization, Hybridization, Parallelization, Aggregation, Randomization, Replication, Sequentialization. Indeed, it is important to emphasize right away that the so-called no free lunch theorem applies here, in the sense that there is no universally superior method that outperforms all other methods on all categories of bigness. It is also important to stress the fact that simplicity in the sense of Ockham’s razor non-plurality principle of parsimony tends to reign supreme when it comes to massive data. We conclude with a comparison of the predictive performance of some of the most commonly used methods on a few data sets.

A Comparative Analysis of Predictive Learning Algorithms on High-Dimensional Microarray Cancer Data

This research evaluates pattern recognition techniques on a subclass of big data where the dimensionality of the input space (p) is much larger than the number of observations (n). Specifically, we evaluate massive gene expression microarray cancer data where the ratio κ is less than one. We explore the statistical and computational challenges inherent in these high dimensional low sample size (HDLSS) problems and present statistical machine learning methods used to tackle and circumvent these difficulties. Regularization and kernel algorithms were explored in this research using seven datasets where κ < 1. These techniques require special attention to tuning necessitating several extensions of cross-validation to be investigated to support better predictive performance. While no single algorithm was universally the best predictor, the regularization technique produced lower test errors in five of the seven datasets studied.

Quantization of the Inhomogeneous Bianchi I model:quasi-Heisenberg picture

The quantization scheme is suggested for a spatially inhomogeneous 1+1 Bianchi I model. The scheme consists in quantization of the equations of motion and gives the operator (so called quasi-Heisenberg) equations describing explicit evolution of a system. Some particular gauge suitable for quantization is proposed. The Wheeler-DeWitt equation is considered in the vicinity of zero scale factor and it is used to construct a space where the quasi-Heisenberg operators act. Spatial discretization as a UV regularization procedure is suggested for the equations of motion.

Identification of a time-dependent bio-heat blood perfusion coefficient

In the paper the identification of the time-dependent blood perfusion coefficient is formulated as an inverse problem. The bio-heat conduction problem is transformed into the classical heat conduction problem. Then the transformed inverse problem is solved using the method of fundamental solutions together with the Tikhonov regularization. Some numerical results are presented in order to demonstrate the accuracy and the stability of the proposed meshless numerical algorithm.

A method of fundamental solutions for radially symmetric and axisymmetric backward heat conduction problems

We propose and investigate an application of the method of fundamental solutions (MFS) to the radially symmetric and axisymmetric backward heat conduction problem (BHCP) in a solid or hollow cylinder. In the BHCP, the initial temperature is to be determined from the temperature measurements at a later time. This is an inverse and ill-posed problem, and we employ and generalize the MFS regularization approach [B.T. Johansson and D. Lesnic, A method of fundamental solutions for transient heat conduction, Eng. Anal. Boundary Elements 32 (2008), pp. 697–703] for the time-dependent heat equation to obtain a stable and accurate numerical approximation with small computational cost.

A method of fundamental solutions for theaxisymmetric backward heat equation

We propose and investigate an application of the method of fundamental solutions (MFS) to the radially symmetric and axisymmetric backward heat conduction problem (BHCP) in a solid or hollow cylinder. In the BHCP, the initial temperature is to be determined from the temperature measurements at a later time. This is an inverse and ill-posed problem, and we employ and generalize the MFS regularization approach [B.T. Johansson and D. Lesnic, A method of fundamental solutions for transient heat conduction, Eng. Anal. Boundary Elements 32 (2008), pp. 697–703] for the time-dependent heat equation to obtain a stable and accurate numerical approximation with small computational cost.

Numerical solution of a Cauchy problem for Laplace equation in 3-dimensional domains by integral equations

A numerical method based on integral equations is proposed and investigated for the Cauchy problem for the Laplace equation in 3-dimensional smooth bounded doubly connected domains. To numerically reconstruct a harmonic function from knowledge of the function and its normal derivative on the outer of two closed boundary surfaces, the harmonic function is represented as a single-layer potential. Matching this representation against the given data, a system of boundary integral equations is obtained to be solved for two unknown densities. This system is rewritten over the unit sphere under the assumption that each of the two boundary surfaces can be mapped smoothly and one-to-one to the unit sphere. For the discretization of this system, Weinert’s method (PhD, Göttingen, 1990) is employed, which generates a Galerkin type procedure for the numerical solution, and the densities in the system of integral equations are expressed in terms of spherical harmonics. Tikhonov regularization is incorporated, and numerical results are included showing the efficiency of the proposed procedure.