Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

A linear model for the structure of turbulence beneath surface water waves

The structure of turbulence in the ocean surface layer is investigated using a simplified semi-analytical model based on rapid-distortion theory. In this model, which is linear with respect to the turbulence, the flow comprises a mean Eulerian shear current, the Stokes drift of an irrotational surface wave, which accounts for the irreversible effect of the waves on the turbulence, and the turbulence itself, whose time evolution is calculated. By analysing the equations of motion used in the model, which are linearised versions of the Craik–Leibovich equations containing a ‘vortex force’, it is found that a flow including mean shear and a Stokes drift is formally equivalent to a flow including mean shear and rotation. In particular, Craik and Leibovich’s condition for the linear instability of the first kind of flow is equivalent to Bradshaw’s condition for the linear instability of the second. However, the present study goes beyond linear stability analyses by considering flow disturbances of finite amplitude, which allows calculating turbulence statistics and addressing cases where the linear stability is neutral. Results from the model show that the turbulence displays a structure with a continuous variation of the anisotropy and elongation, ranging from streaky structures, for distortion by shear only, to streamwise vortices resembling Langmuir circulations, for distortion by Stokes drift only. The TKE grows faster for distortion by a shear and a Stokes drift gradient with the same sign (a situation relevant to wind waves), but the turbulence is more isotropic in that case (which is linearly unstable to Langmuir circulations).

A linear model of gravity wave drag for hydrostatic sheared flow over elliptical mountains

An analytical model of orographic gravity wave drag due to sheared flow past elliptical mountains is developed. The model extends the domain of applicability of the well-known Phillips model to wind profiles that vary relatively slowly in the vertical, so that they may be treated using a WKB approximation. The model illustrates how linear processes associated with wind profile shear and curvature affect the drag force exerted by the airflow on mountains, and how it is crucial to extend the WKB approximation to second order in the small perturbation parameter for these effects to be taken into account. For the simplest wind profiles, the normalized drag depends only on the Richardson number, Ri, of the flow at the surface and on the aspect ratio, γ, of the mountain. For a linear wind profile, the drag decreases as Ri decreases, and this variation is faster when the wind is across the mountain than when it is along the mountain. For a wind that rotates with height maintaining its magnitude, the drag generally increases as Ri decreases, by an amount depending on γ and on the incidence angle. The results from WKB theory are compared with exact linear results and also with results from a non-hydrostatic nonlinear numerical model, showing in general encouraging agreement, down to values of Ri of order one.

The structure of the optimal income tax in the quasi-linear model

Existing numerical characterizations of the optimal income tax have been based on a limited number of model specifications. As a result, they do not reveal which properties are general. We determine the optimal tax in the quasi-linear model under weaker assumptions than have previously been used; in particular, we remove the assumption of a lower bound on the utility of zero consumption and the need to permit negative labor incomes. A Monte Carlo analysis is then conducted in which economies are selected at random and the optimal tax function constructed. The results show that in a significant proportion of economies the marginal tax rate rises at low skills and falls at high. The average tax rate is equally likely to rise or fall with skill at low skill levels, rises in the majority of cases in the centre of the skill range, and falls at high skills. These results are consistent across all the specifications we test. We then extend the analysis to show that these results also hold for Cobb-Douglas utility.

Improving power and accuracy of genome-wide association studies via a multi-locus mixed linear model methodology

Genome-wide association studies (GWAS) have been widely used in genetic dissection of complex traits. However, common methods are all based on a fixed-SNP-effect mixed linear model (MLM) and single marker analysis, such as efficient mixed model analysis (EMMA). These methods require Bonferroni correction for multiple tests, which often is too conservative when the number of markers is extremely large. To address this concern, we proposed a random-SNP-effect MLM (RMLM) and a multi-locus RMLM (MRMLM) for GWAS. The RMLM simply treats the SNP-effect as random, but it allows a modified Bonferroni correction to be used to calculate the threshold p value for significance tests. The MRMLM is a multi-locus model including markers selected from the RMLM method with a less stringent selection criterion. Due to the multi-locus nature, no multiple test correction is needed. Simulation studies show that the MRMLM is more powerful in QTN detection and more accurate in QTN effect estimation than the RMLM, which in turn is more powerful and accurate than the EMMA. To demonstrate the new methods, we analyzed six flowering time related traits in Arabidopsis thaliana and detected more genes than previous reported using the EMMA. Therefore, the MRMLM provides an alternative for multi-locus GWAS.

A linear optimization approach to the combined production planning model

Two fundamental processes usually arise in the production planning of many industries. The first one consists of deciding how many final products of each type have to be produced in each period of a planning horizon, the well-known lot sizing problem. The other process consists of cutting raw materials in stock in order to produce smaller parts used in the assembly of final products, the well-studied cutting stock problem. In this paper the decision variables of these two problems are dependent of each other in order to obtain a global optimum solution. Setups that are typically present in lot sizing problems are relaxed together with integer frequencies of cutting patterns in the cutting problem. Therefore, a large scale linear optimizations problem arises, which is exactly solved by a column generated technique. It is worth noting that this new combined problem still takes the trade-off between storage costs (for final products and the parts) and trim losses (in the cutting process). We present some sets of computational tests, analyzed over three different scenarios. These results show that, by combining the problems and using an exact method, it is possible to obtain significant gains when compared to the usual industrial practice, which solve them in sequence. (C) 2010 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.

Linear delta expansion applied to the O'Raifeartaigh model

We reassess the method of the linear delta expansion for the calculation of effective potentials in superspace, by adopting the improved version of the super-Feynman rules in the framework of the O'Raifeartaigh model for spontaneous supersymmetry breaking. The effective potential is calculated using both the fastest apparent convergence and the principle of minimal sensitivity criteria and the consistency and efficacy of the method are checked in deriving the Coleman-Weinberg potential.

Superspace approach to the renormalization of the O'Raifeartaigh model up to the second order in the linear delta expansion parameter

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

On non-linear dynamics and control designs applied to the ideal and non-ideal variants of the Fitzhugh-Nagumo (FN) mathematical model

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

On non-linear dynamics and an optimal control synthesis of the action potential of membranes (ideal and non-ideal cases) of the Hodgkin-Huxley (HH) mathematical model

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Multiobjective linear model for pre-feasibility design of cogeneration systems

This article deals with some methodologies for economic and technical evaluations of cogeneration projects proposed by several authors. A discussion on design philosophy applied to thermal power plants leads to the decision problem of a conflicting, multiobjective formulation that includes the most important parameters. This model is formulated to help decision makers and designers in choosing compromise values for included parameters. (C) 1997 Elsevier B.V. Ltd.

A linear optimization approach to the combined production planning model

Two fundamental processes usually arise in the production planning of many industries. The first one consists of deciding how many final products of each type have to be produced in each period of a planning horizon, the well-known lot sizing problem. The other process consists of cutting raw materials in stock in order to produce smaller parts used in the assembly of final products, the well-studied cutting stock problem. In this paper the decision variables of these two problems are dependent of each other in order to obtain a global optimum solution. Setups that are typically present in lot sizing problems are relaxed together with integer frequencies of cutting patterns in the cutting problem. Therefore, a large scale linear optimizations problem arises, which is exactly solved by a column generated technique. It is worth noting that this new combined problem still takes the trade-off between storage costs (for final products and the parts) and trim losses (in the cutting process). We present some sets of computational tests, analyzed over three different scenarios. These results show that, by combining the problems and using an exact method, it is possible to obtain significant gains when compared to the usual industrial practice, which solve them in sequence. (C) 2010 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.