A tunable radial basis function model for nonlinear system identification using particle swarm optimisation

A tunable radial basis function (RBF) network model is proposed for nonlinear system identification using particle swarm optimisation (PSO). At each stage of orthogonal forward regression (OFR) model construction, PSO optimises one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is computationally more efficient.

Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

Bayesian model averaging and identification of structural breaks in time series

Bayesian Model Averaging (BMA) is used for testing for multiple break points in univariate series using conjugate normal-gamma priors. This approach can test for the number of structural breaks and produce posterior probabilities for a break at each point in time. Results are averaged over specifications including: stationary; stationary around trend and unit root models, each containing different types and number of breaks and different lag lengths. The procedures are used to test for structural breaks on 14 annual macroeconomic series and 11 natural resource price series. The results indicate that there are structural breaks in all of the natural resource series and most of the macroeconomic series. Many of the series had multiple breaks. Our findings regarding the existence of unit roots, having allowed for structural breaks in the data, are largely consistent with previous work.

Fast identification algorithms for Gaussian process model

A class identification algorithms is introduced for Gaussian process(GP)models.The fundamental approach is to propose a new kernel function which leads to a covariance matrix with low rank,a property that is consequently exploited for computational efficiency for both model parameter estimation and model predictions.The objective of either maximizing the marginal likelihood or the Kullback–Leibler (K–L) divergence between the estimated output probability density function(pdf)and the true pdf has been used as respective cost functions.For each cost function,an efficient coordinate descent algorithm is proposed to estimate the kernel parameters using a one dimensional derivative free search, and noise variance using a fast gradient descent algorithm. Numerical examples are included to demonstrate the effectiveness of the new identification approaches.

The Phosphorus Indicators Tool: a simple model of diffuse P loss from agricultural land to water

The Phosphorus Indicators Tool provides a catchment-scale estimation of diffuse phosphorus (P) loss from agricultural land to surface waters using the most appropriate indicators of P loss. The Tool provides a framework that may be applied across the UK to estimate P loss, which is sensitive not only to land use and management but also to environmental factors such as climate, soil type and topography. The model complexity incorporated in the P Indicators Tool has been adapted to the level of detail in the available data and the need to reflect the impact of changes in agriculture. Currently, the Tool runs on an annual timestep and at a 1 km(2) grid scale. We demonstrate that the P Indicators Tool works in principle and that its modular structure provides a means of accounting for P loss from one layer to the next, and ultimately to receiving waters. Trial runs of the Tool suggest that modelled P delivery to water approximates measured water quality records. The transparency of the structure of the P Indicators Tool means that identification of poorly performing coefficients is possible, and further refinements of the Tool can be made to ensure it is better calibrated and subsequently validated against empirical data, as it becomes available.

Cone model-based SEP event calculations for applications to multipoint observations

The problem of modeling solar energetic particle (SEP) events is important to both space weather research and forecasting, and yet it has seen relatively little progress. Most important SEP events are associated with coronal mass ejections (CMEs) that drive coronal and interplanetary shocks. These shocks can continuously produce accelerated particles from the ambient medium to well beyond 1 AU. This paper describes an effort to model real SEP events using a Center for Integrated Space weather Modeling (CISM) MHD solar wind simulation including a cone model of CMEs to initiate the related shocks. In addition to providing observation-inspired shock geometry and characteristics, this MHD simulation describes the time-dependent observer field line connections to the shock source. As a first approximation, we assume a shock jump-parameterized source strength and spectrum, and that scatter-free transport occurs outside of the shock source, thus emphasizing the role the shock evolution plays in determining the modeled SEP event profile. Three halo CME events on May 12, 1997, November 4, 1997 and December 13, 2006 are used to test the modeling approach. While challenges arise in the identification and characterization of the shocks in the MHD model results, this approach illustrates the importance to SEP event modeling of globally simulating the underlying heliospheric event. The results also suggest the potential utility of such a model for forcasting and for interpretation of separated multipoint measurements such as those expected from the STEREO mission.

Recategorization and subgroup identification: predicting and preventing threats from common ingroups

Much work has supported the idea that recategorization of ingroups and outgroups into a superordinate category can have beneficial effects for intergroup relations. Recently, however, increases in bias following recategorization have been observed in some contexts. It is argued that such unwanted consequences of recategorization will only be apparent for perceivers who are highly committed to their ingroup subgroups. In Experiments 1 to 3, the authors observed, on both explicit and implicit measures, that an increase in bias following recategorization occurred only for high subgroup identifiers. In Experiment 4, it was found that maintaining the salience of subgroups within a recategorized superordinate group averted this increase in bias for high identifiers and led overall to the lowest levels of bias. These findings are discussed in the context of recent work on the Common Ingroup Identity Model.

Identification of trade-offs underlying the primary strategies of plants

Question: What are the key physiological and life-history trade-offs responsible for the evolution of different suites of plant traits (strategies) in different environments? Experimental methods: Common-garden experiments were performed on physiologically realistic model plants, evolved in contrasting environments, in computer simulations. This allowed the identification of the trade-offs that resulted in different suites of traits (strategies). The environments considered were: resource rich, low disturbance (competitive); resource poor, low disturbance (stressed); resource rich, high disturbance (disturbed); and stressed environments containing herbivores (grazed). Results: In disturbed environments, plants increased reproduction at the expense of ability to compete for light and nitrogen. In competitive environments, plants traded off reproductive output and leaf production for vertical growth. In stressed environments, plants traded off vertical growth and reproductive output for nitrogen acquisition, contradicting Grime's (2001) theory that slow-growing, competitively inferior strategies are selected in stressed environments. The contradiction is partly resolved by incorporating herbivores into the stressed environment, which selects for increased investment in defence, at the expense of competitive ability and reproduction. Conclusion: Our explicit modelling of trade-offs produces rigorous testable explanations of observed associations between suites of traits and environments.

Identification of stimulatory and inhibitory inputs to the hypothalamic-pituitary-adrenal axis during hypoglycaemia or transport in ewes

This study used the novel approach of statistical modelling to investigate the control of hypothalamic-pituitary-adrenal (HPA) axis and quantify temporal relationships between hormones. Two experimental paradigms were chosen, insulin-induced hypoglycaemia and 2 h transport, to assess differences in control between noncognitive and cognitive stimuli. Vasopressin and corticotropin-releasing hormone (CRH) were measured in hypophysial portal plasma, and adrenocorticotropin hormone (ACTH) and cortisol in jugular plasma of conscious sheep, and deconvolution analysis was used to calculate secretory rates, before modelling. During hypoglycaemia, the relationship between plasma glucose and vasopressin or CRH was best described by log(10) transforming variables (i.e. a positive power-curve relationship). A negative-feedback relationship with log(10) cortisol concentration 2 h previously was detected. Analysis of the 'transport' stimulus suggested that the strength of the perceived stimulus decreased over time after accounting for cortisol facilitation and negative-feedback. The time course of vasopressin and CRH responses to each stimulus were different However, at the pituitary level, the data suggested that log(10) ACTH secretion rate was related to log(10) vasopressin and CRH concentrations with very similar regression coefficients and an identical ratio of actions (2.3 : 1) for both stimuli. Similar magnitude negative-feedback effects of log(10) cortisol at -110 min (hypoglycaemia) or -40 min (transport) were detected, and both models contained a stimulatory relationship with cortisol at 0 min (facilitation). At adrenal gland level, cortisol secretory rates were related to simultaneously measured untransformed ACTH concentration but the regression coefficient for the hypoglycaemia model was 2.5-fold greater than for transport. No individual sustained maximum cortisol secretion for longer than 20 min during hypoglycaemia and 40 min during transport. These unique models demonstrate that corticosteroid negative-feedback is a significant control mechanism at both the pituitary and hypothalamus. The amplitude of HPA response may be related to stimulus intensity and corticosteroid negative-feedback, while duration depended on feedback alone.

Robust identification for linear-in-the-parameters models

In this paper new robust nonlinear model construction algorithms for a large class of linear-in-the-parameters models are introduced to enhance model robustness, including three algorithms using combined A- or D-optimality or PRESS statistic (Predicted REsidual Sum of Squares) with regularised orthogonal least squares algorithm respectively. A common characteristic of these algorithms is that the inherent computation efficiency associated with the orthogonalisation scheme in orthogonal least squares or regularised orthogonal least squares has been extended such that the new algorithms are computationally efficient. A numerical example is included to demonstrate effectiveness of the algorithms. Copyright (C) 2003 IFAC.

An Efficient Parameterization of Dynamic Neural Networks for Nonlinear System Identification

Dynamic neural networks (DNNs), which are also known as recurrent neural networks, are often used for nonlinear system identification. The main contribution of this letter is the introduction of an efficient parameterization of a class of DNNs. Having to adjust less parameters simplifies the training problem and leads to more parsimonious models. The parameterization is based on approximation theory dealing with the ability of a class of DNNs to approximate finite trajectories of nonautonomous systems. The use of the proposed parameterization is illustrated through a numerical example, using data from a nonlinear model of a magnetic levitation system.

Nonlinear system identification using particle swarm optimisation tuned radial basis function models

A novel particle swarm optimisation (PSO) tuned radial basis function (RBF) network model is proposed for identification of non-linear systems. At each stage of orthogonal forward regression (OFR) model construction process, PSO is adopted to tune one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is often more efficient in model construction. The effectiveness of the proposed PSO aided OFR algorithm for constructing tunable node RBF models is demonstrated using three real data sets.

Identification of nonlinear systems using generalized kernel models

Nonlinear system identification is considered using a generalized kernel regression model. Unlike the standard kernel model, which employs a fixed common variance for all the kernel regressors, each kernel regressor in the generalized kernel model has an individually tuned diagonal covariance matrix that is determined by maximizing the correlation between the training data and the regressor using a repeated guided random search based on boosting optimization. An efficient construction algorithm based on orthogonal forward regression with leave-one-out (LOO) test statistic and local regularization (LR) is then used to select a parsimonious generalized kernel regression model from the resulting full regression matrix. The proposed modeling algorithm is fully automatic and the user is not required to specify any criterion to terminate the construction procedure. Experimental results involving two real data sets demonstrate the effectiveness of the proposed nonlinear system identification approach.

Subspace system identification framework for the analysis of multimoded propagation of THz-transient signals

We provide a system identification framework for the analysis of THz-transient data. The subspace identification algorithm for both deterministic and stochastic systems is used to model the time-domain responses of structures under broadband excitation. Structures with additional time delays can be modelled within the state-space framework using additional state variables. We compare the numerical stability of the commonly used least-squares ARX models to that of the subspace N4SID algorithm by using examples of fourth-order and eighth-order systems under pulse and chirp excitation conditions. These models correspond to structures having two and four modes simultaneously propagating respectively. We show that chirp excitation combined with the subspace identification algorithm can provide a better identification of the underlying mode dynamics than the ARX model does as the complexity of the system increases. The use of an identified state-space model for mode demixing, upon transformation to a decoupled realization form is illustrated. Applications of state-space models and the N4SID algorithm to THz transient spectroscopy as well as to optical systems are highlighted.