Robust nonlinear model identification methods using forward regression

In this correspondence new robust nonlinear model construction algorithms for a large class of linear-in-the-parameters models are introduced to enhance model robustness via combined parameter regularization and new robust structural selective criteria. In parallel to parameter regularization, we use two classes of robust model selection criteria based on either experimental design criteria that optimizes model adequacy, or the predicted residual sums of squares (PRESS) statistic that optimizes model generalization capability, respectively. Three robust identification algorithms are introduced, i.e., combined A- and D-optimality with regularized orthogonal least squares algorithm, respectively; and combined PRESS statistic with regularized orthogonal least squares algorithm. A common characteristic of these algorithms is that the inherent computation efficiency associated with the orthogonalization scheme in orthogonal least squares or regularized orthogonal least squares has been extended such that the new algorithms are computationally efficient. Numerical examples are included to demonstrate effectiveness of the algorithms.

Particle swarm optimization aided orthogonal forward regression for unified data modeling

We propose a unified data modeling approach that is equally applicable to supervised regression and classification applications, as well as to unsupervised probability density function estimation. A particle swarm optimization (PSO) aided orthogonal forward regression (OFR) algorithm based on leave-one-out (LOO) criteria is developed to construct parsimonious radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines the center vector and diagonal covariance matrix of one RBF node by minimizing the LOO statistics. For regression applications, the LOO criterion is chosen to be the LOO mean square error, while the LOO misclassification rate is adopted in two-class classification applications. By adopting the Parzen window estimate as the desired response, the unsupervised density estimation problem is transformed into a constrained regression problem. This PSO aided OFR algorithm for tunable-node RBF networks is capable of constructing very parsimonious RBF models that generalize well, and our analysis and experimental results demonstrate that the algorithm is computationally even simpler than the efficient regularization assisted orthogonal least square algorithm based on LOO criteria for selecting fixed-node RBF models. Another significant advantage of the proposed learning procedure is that it does not have learning hyperparameters that have to be tuned using costly cross validation. The effectiveness of the proposed PSO aided OFR construction procedure is illustrated using several examples taken from regression and classification, as well as density estimation applications.

Optimal observers for ARMA models

A polynomial-based ARMA model, when posed in a state-space framework can be regarded in many different ways. In this paper two particular state-space forms of the ARMA model are considered, and although both are canonical in structure they differ in respect of the mode in which disturbances are fed into the state and output equations. For both forms a solution is found to the optimal discrete-time observer problem and algebraic connections between the two optimal observers are shown. The purpose of the paper is to highlight the fact that the optimal observer obtained from the first state-space form, commonly known as the innovations form, is not that employed in an optimal controller, in the minimum-output variance sense, whereas the optimal observer obtained from the second form is. Hence the second form is a much more appropriate state-space description to use for controller design, particularly when employed in self-tuning control schemes.

Parameter estimation based on stacked regression and evolutionary algorithms

A new parameter-estimation algorithm, which minimises the cross-validated prediction error for linear-in-the-parameter models, is proposed, based on stacked regression and an evolutionary algorithm. It is initially shown that cross-validation is very important for prediction in linear-in-the-parameter models using a criterion called the mean dispersion error (MDE). Stacked regression, which can be regarded as a sophisticated type of cross-validation, is then introduced based on an evolutionary algorithm, to produce a new parameter-estimation algorithm, which preserves the parsimony of a concise model structure that is determined using the forward orthogonal least-squares (OLS) algorithm. The PRESS prediction errors are used for cross-validation, and the sunspot and Canadian lynx time series are used to demonstrate the new algorithms.

Linear-wavelet models for non-linear identification applied to a pressure plant

A nonlinear regression structure comprising a wavelet network and a linear term is proposed for system identification. The theoretical foundation of the approach is laid by proving that radial wavelets are orthogonal to linear functions. A constructive procedure for building such models is described and the approach is tested with experimental data.

Linear-wavelet models applied to the identification of a two-link manipulator

This paper shows that a wavelet network and a linear term can be advantageously combined for the purpose of non linear system identification. The theoretical foundation of this approach is laid by proving that radial wavelets are orthogonal to linear functions. A constructive procedure for building such nonlinear regression structures, termed linear-wavelet models, is described. For illustration, sim ulation data are used to identify a model for a two-link robotic manipulator. The results show that the introduction of wavelets does improve the prediction ability of a linear model.

Comparing a discrete and continuum model of the intestinal crypt

The integration of processes at different scales is a key problem in the modelling of cell populations. Owing to increased computational resources and the accumulation of data at the cellular and subcellular scales, the use of discrete, cell-level models, which are typically solved using numerical simulations, has become prominent. One of the merits of this approach is that important biological factors, such as cell heterogeneity and noise, can be easily incorporated. However, it can be difficult to efficiently draw generalizations from the simulation results, as, often, many simulation runs are required to investigate model behaviour in typically large parameter spaces. In some cases, discrete cell-level models can be coarse-grained, yielding continuum models whose analysis can lead to the development of insight into the underlying simulations. In this paper we apply such an approach to the case of a discrete model of cell dynamics in the intestinal crypt. An analysis of the resulting continuum model demonstrates that there is a limited region of parameter space within which steady-state (and hence biologically realistic) solutions exist. Continuum model predictions show good agreement with corresponding results from the underlying simulations and experimental data taken from murine intestinal crypts.

Discrete time representation of continuous time ARMA processes

This paper derives exact discrete time representations for data generated by a continuous time autoregressive moving average (ARMA) system with mixed stock and flow data. The representations for systems comprised entirely of stocks or of flows are also given. In each case the discrete time representations are shown to be of ARMA form, the orders depending on those of the continuous time system. Three examples and applications are also provided, two of which concern the stationary ARMA(2, 1) model with stock variables (with applications to sunspot data and a short-term interest rate) and one concerning the nonstationary ARMA(2, 1) model with a flow variable (with an application to U.S. nondurable consumers’ expenditure). In all three examples the presence of an MA(1) component in the continuous time system has a dramatic impact on eradicating unaccounted-for serial correlation that is present in the discrete time version of the ARMA(2, 0) specification, even though the form of the discrete time model is ARMA(2, 1) for both models.

Superensembles of linear viscoelastic models of polymer melts

The idea of incorporating multiple models of linear rheology into a superensemble, to forge a consensus forecast from the individual model predictions, is investigated. The relative importance of the individual models in the so-called multimodel superensemble (MMSE) was inferred by evaluating their performance on a set of experimental training data, via nonlinear regression. The predictive ability of the MMSE model was tested by comparing its predictions on test data that were similar (in-sample) and dissimilar (out-of-sample) to the training data used in the calibration. For the in-sample forecasts, we found that the MMSE model easily outperformed the best constituent model. The presence of good individual models greatly enhanced the MMSE forecast, while the presence of some bad models in the superensemble also improved the MMSE forecast modestly. While the performance of the MMSE model on the out-of-sample training data was not as spectacular, it demonstrated the robustness of this approach.

An elastic net orthogonal forward regression algorithm

In this paper we propose an efficient two-level model identification method for a large class of linear-in-the-parameters models from the observational data. A new elastic net orthogonal forward regression (ENOFR) algorithm is employed at the lower level to carry out simultaneous model selection and elastic net parameter estimation. The two regularization parameters in the elastic net are optimized using a particle swarm optimization (PSO) algorithm at the upper level by minimizing the leave one out (LOO) mean square error (LOOMSE). Illustrative examples are included to demonstrate the effectiveness of the new approaches.

Validation of three new measure-correlate-predict models for the long-term prospection of the wind resource

The estimation of the long-term wind resource at a prospective site based on a relatively short on-site measurement campaign is an indispensable task in the development of a commercial wind farm. The typical industry approach is based on the measure-correlate-predict �MCP� method where a relational model between the site wind velocity data and the data obtained from a suitable reference site is built from concurrent records. In a subsequent step, a long-term prediction for the prospective site is obtained from a combination of the relational model and the historic reference data. In the present paper, a systematic study is presented where three new MCP models, together with two published reference models �a simple linear regression and the variance ratio method�, have been evaluated based on concurrent synthetic wind speed time series for two sites, simulating the prospective and the reference site. The synthetic method has the advantage of generating time series with the desired statistical properties, including Weibull scale and shape factors, required to evaluate the five methods under all plausible conditions. In this work, first a systematic discussion of the statistical fundamentals behind MCP methods is provided and three new models, one based on a nonlinear regression and two �termed kernel methods� derived from the use of conditional probability density functions, are proposed. All models are evaluated by using five metrics under a wide range of values of the correlation coefficient, the Weibull scale, and the Weibull shape factor. Only one of all models, a kernel method based on bivariate Weibull probability functions, is capable of accurately predicting all performance metrics studied.

On a stochastic Trotter formula with application to spontaneous localization models

We consider the relation between so called continuous localization models—i.e. non-linear stochastic Schrödinger evolutions—and the discrete GRW-model of wave function collapse. The former can be understood as scaling limit of the GRW process. The proof relies on a stochastic Trotter formula, which is of interest in its own right. Our Trotter formula also allows to complement results on existence theory of stochastic Schrödinger evolutions by Holevo and Mora/Rebolledo.

Regularized logistic models for probabilistic forecasting and diagnostics

Logistic models are studied as a tool to convert dynamical forecast information (deterministic and ensemble) into probability forecasts. A logistic model is obtained by setting the logarithmic odds ratio equal to a linear combination of the inputs. As with any statistical model, logistic models will suffer from overfitting if the number of inputs is comparable to the number of forecast instances. Computational approaches to avoid overfitting by regularization are discussed, and efficient techniques for model assessment and selection are presented. A logit version of the lasso (originally a linear regression technique), is discussed. In lasso models, less important inputs are identified and the corresponding coefficient is set to zero, providing an efficient and automatic model reduction procedure. For the same reason, lasso models are particularly appealing for diagnostic purposes.

Equator-to-pole temperature differences and the extra-tropical storm track responses of the CMIP5 climate models

This paper aims to understand the physical processes causing the large spread in the storm track projections of the CMIP5 climate models. In particular, the relationship between the climate change responses of the storm tracks, as measured by the 2–6 day mean sea level pressure variance, and the equator-to-pole temperature differences at upper- and lower-tropospheric levels is investigated. In the southern hemisphere the responses of the upper- and lower-tropospheric temperature differences are correlated across the models and as a result they share similar associations with the storm track responses. There are large regions in which the storm track responses are correlated with the temperature difference responses, and a simple linear regression model based on the temperature differences at either level captures the spatial pattern of the mean storm track response as well explaining between 30 and 60 % of the inter-model variance of the storm track responses. In the northern hemisphere the responses of the two temperature differences are not significantly correlated and their associations with the storm track responses are more complicated. In summer, the responses of the lower-tropospheric temperature differences dominate the inter-model spread of the storm track responses. In winter, the responses of the upper- and lower-temperature differences both play a role. The results suggest that there is potential to reduce the spread in storm track responses by constraining the relative magnitudes of the warming in the tropical and polar regions.

Rheology of discrete failure regimes of anisotropic sea ice

A rheological model of sea ice is presented that incorporates the orientational distribution of ice thickness in leads embedded in isotropic floe ice. Sea ice internal stress is determined by coulombic, ridging and tensile failure at orientations where corresponding failure criteria are satisfied at minimum stresses. Because sea ice traction increases in thinner leads and cohesion is finite, such failure line angles are determined by the orientational distribution of sea ice thickness relative to the imposed stresses. In contrast to the isotropic case, sea ice thickness anisotropy results in these failure lines becoming dependent on the stress magnitude. Although generally a given failure criteria type can be satisfied at many directions, only two at most are considered. The strain rate is determined by shearing along slip lines accompanied by dilatancy and closing or opening across orientations affected by ridging or tensile failure. The rheology is illustrated by a yield curve determined by combining coulombic and ridging failure for the case of two pairs of isotropically formed leads of different thicknesses rotated with regard to each other, which models two events of coulombic failure followed by dilatancy and refreezing. The yield curve consists of linear segments describing coulombic and ridging yield as failure switches from one lead to another as the stress grows. Because sliding along slip lines is accompanied by dilatancy, at typical Arctic sea ice deformation rates a one-day-long deformation event produces enough open water that these freshly formed slip lines are preferential places of ridging failure.