Adaptive-multi-reference least means squares filter

Adaptive filters are now becoming increasingly studied for their suitability in application to complex and non-stationary signals. Many adaptive filters utilise a reference input, that is used to form an estimate of the noise in the target signal. In this paper we discuss the application of adaptive filters for high electromyography contaminated electroencephalography data. We propose the use of multiple referential inputs instead of the traditional single input. These references are formed using multiple EMG sensors during an EEG experiment, each reference input is processed and ordered through firstly determining the Pearson’s r-squared correlation coefficient, from this a weighting metric is determined and used to scale and order the reference channels according to the paradigm shown in this paper. This paper presents the use and application of the Adaptive-Multi-Reference (AMR) Least Means Square adaptive filter in the domain of electroencephalograph signal acquisition.

OWA operators in linear regression and detection of outliers

We consider the use of Ordered Weighted Averaging (OWA) in linear regression. Our goal is to replace the traditional least squares, least absolute deviation, and maximum likelihood criteria with an OWA function of the residuals. We obtain several high breakdown robust regression methods as special cases (least median, least trimmed squares, trimmed likelihood methods). We also present new formulations of regression problem. OWA-based regression is particularly useful in the presence of outliers.

OWA operators in regression problems

We consider an application of fuzzy logic connectives to statistical regression. We replace the standard least squares, least absolute deviation, and maximum likelihood criteria with an ordered weighted averaging (OWA) function of the residuals. Depending on the choice of the weights, we obtain the standard regression problems, high-breakdown robust methods (least median, least trimmed squares, and trimmed likelihood methods), as well as new formulations. We present various approaches to numerical solution of such regression problems. OWA-based regression is particularly useful in the presence of outliers, and we illustrate the performance of the new methods on several instances of linear regression problems with multiple outliers.

On performance of transform domain adaptive filters with Markov-2 inputs

In this paper, the analysis for the performance of the discrete Fourier transform LMS adaptive filter (DFT-LMS) and the discrete cosine transform LMS adaptive filter (DCT-LMS) for the Markov-2 inputs is presented. To improve the convergence property of the least mean squares (LMS) adaptive filter, the DFT-LMS and DCT-LMS preprocess the inputs with the fixed orthogonal transforms and power normalization. We derive the asymptotic results for the eigenvalues and eigenvalue distributions of the preprocessed input autocorrelation matrices with DFT-LMS and DCT-LMS for Markov-2 inputs. These results explicitly show the superior decorrelation property of DCT-LMS over that of DFT-LMS, and also provide the upper bounds for the eigenvalue spreads of the finite-length DFT-LMS and DCT-LMS adaptive filters. Simulation results are demonstrated to support the analytic results.

Global non-smooth optimization in robust multivariate regression

Robust regression in statistics leads to challenging optimization problems. Here, we study one such problem, in which the objective is non-smooth, non-convex and expensive to calculate. We study the numerical performance of several derivative-free optimization algorithms with the aim of computing robust multivariate estimators. Our experiences demonstrate that the existing algorithms often fail to deliver optimal solutions. We introduce three new methods that use Powell's derivative-free algorithm. The proposed methods are reliable and can be used when processing very large data sets containing outliers.

Derivative-free optimization and neural networks for robust regression

Large outliers break down linear and nonlinear regression models. Robust regression methods allow one to filter out the outliers when building a model. By replacing the traditional least squares criterion with the least trimmed squares (LTS) criterion, in which half of data is treated as potential outliers, one can fit accurate regression models to strongly contaminated data. High-breakdown methods have become very well established in linear regression, but have started being applied for non-linear regression only recently. In this work, we examine the problem of fitting artificial neural networks (ANNs) to contaminated data using LTS criterion. We introduce a penalized LTS criterion which prevents unnecessary removal of valid data. Training of ANNs leads to a challenging non-smooth global optimization problem. We compare the efficiency of several derivative-free optimization methods in solving it, and show that our approach identifies the outliers correctly when ANNs are used for nonlinear regression.

Biomedical signal filtering for noisy environments

 Luke's work addresses issue of robustly attenuating multi-source noise from surface EEG signals using a novel Adaptive-Multiple-Reference Least-Means-Squares filter (AMR-LMS). In practice, the filter successfully removes electrical interference and muscle noise generated during movement which contaminates EEG, allowing subjects to maintain maximum mobility throughout signal acquisition and during the use of a Brain Computer Interface.

Neither fixed nor random: weighted least squares meta-analysis

This study challenges two core conventional meta-analysis methods: fixed effect and random effects. We show how and explain why an unrestricted weighted least squares estimator is superior to conventional random-effects meta-analysis when there is publication (or small-sample) bias and better than a fixed-effect weighted average if there is heterogeneity. Statistical theory and simulations of effect sizes, log odds ratios and regression coefficients demonstrate that this unrestricted weighted least squares estimator provides satisfactory estimates and confidence intervals that are comparable to random effects when there is no publication (or small-sample) bias and identical to fixed-effect meta-analysis when there is no heterogeneity. When there is publication selection bias, the unrestricted weighted least squares approach dominates random effects; when there is excess heterogeneity, it is clearly superior to fixed-effect meta-analysis. In practical applications, an unrestricted weighted least squares weighted average will often provide superior estimates to both conventional fixed and random effects.

Neither fixed nor random: weighted least squares meta-regression

Our study revisits and challenges two core conventional meta-regression estimators: the prevalent use of‘mixed-effects’ or random-effects meta-regression analysis and the correction of standard errors that defines fixed-effects meta-regression analysis (FE-MRA). We show how and explain why an unrestricted weighted least squares MRA (WLS-MRA) estimator is superior to conventional random-effects (or mixed-effects) meta-regression when there is publication (or small-sample) bias that is as good as FE-MRA in all cases and better than fixed effects in most practical applications. Simulations and statistical theory show that WLS-MRA provides satisfactory estimates of meta-regression coefficients that are practically equivalent to mixed effects or random effects when there is no publication bias. When there is publication selection bias, WLS-MRA always has smaller bias than mixed effects or random effects. In practical applications, an unrestricted WLS meta-regression is likely to give practically equivalent or superior estimates to fixed-effects, random-effects, and mixed-effects meta-regression approaches. However, random-effects meta-regression remains viable and perhaps somewhat preferable if selection for statistical significance (publication bias) can be ruled out and when random, additive normal heterogeneity is known to directly affect the ‘true’ regression coefficient.

Monotone approximation of aggregation operators using least squares splines

The need for monotone approximation of scattered data often arises in many problems of regression, when the monotonicity is semantically important. One such domain is fuzzy set theory, where membership functions and aggregation operators are order preserving. Least squares polynomial splines provide great flexbility when modeling non-linear functions, but may fail to be monotone. Linear restrictions on spline coefficients provide necessary and sufficient conditions for spline monotonicity. The basis for splines is selected in such a way that these restrictions take an especially simple form. The resulting non-negative least squares problem can be solved by a variety of standard proven techniques. Additional interpolation requirements can also be imposed in the same framework. The method is applied to fuzzy systems, where membership functions and aggregation operators are constructed from empirical data.

Least squares splines with free knots: global optimization approach

Splines with free knots have been extensively studied in regard to calculating the optimal knot positions. The dependence of the accuracy of approximation on the knot distribution is highly nonlinear, and optimisation techniques face a difficult problem of multiple local minima. The domain of the problem is a simplex, which adds to the complexity. We have applied a recently developed cutting angle method of deterministic global optimisation, which allows one to solve a wide class of optimisation problems on a simplex. The results of the cutting angle method are subsequently improved by local discrete gradient method. The resulting algorithm is sufficiently fast and guarantees that the global minimum has been reached. The results of numerical experiments are presented.

Shape preserving approximation using least squares splines

Least squares polynomial splines are an effective tool for data fitting, but they may fail to preserve essential properties of the underlying function, such as monotonicity or convexity. The shape restrictions are translated into linear inequality conditions on spline coefficients. The basis functions are selected in such a way that these conditions take a simple form, and the problem becomes non-negative least squares problem, for which effecitive and robust methods of solution exist. Multidimensional monotone approximation is achieved by using tensor-product splines with the appropriate restrictions. Additional inter polation conditions can also be introduced. The conversion formulas to traditional B-spline representation are provided.

Should partial least squares be used for analysing psychological constructs?

This paper critically examines an analytical tool - partial least squares regression, or P LS - that is increasingly being used in the academic business literature to validate measures of psychological constructs, and to test hypotheses based on these. The paper provides a contextual and historical review of the resurgence of P LS, and explores several of the claims made by its developers and supporters when it was first promoted in the 1980s, and, more recently, when it reappeared in the information systems literature. Many claims appear plausible but rest on non-mainstream uses of terms and concepts taken from the psychometric field. The paper also canvasses why P LS is a poor analytical tool for research that involves psychological constructs.

Partial least squares and principal components analysis of wine vintage by high performance liquid chromatography with chemiluminescence detection

HPLC with acidic potassium permanganate chemiluminescence detection was employed to analyse 17 Cabernet Sauvignon wines across a range of vintages (1971–2003). Partial least squares regression analysis and principal components analysis was used in order to investigate the relationship between wine composition and vintage. Tartaric acid, vanillic acid, catechin, sinapic acid, ethyl gallate, myricetin, procyanadin B and resveratrol were found to be important components in terms of differences between the vintages.