Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

Dynamic Niche Clustering: a fuzzy variable radius niching technique for multimodal optimisation in GAs

This paper describes the recent developments and improvements made to the variable radius niching technique called Dynamic Niche Clustering (DNC). DNC is fitness sharing based technique that employs a separate population of overlapping fuzzy niches with independent radii which operate in the decoded parameter space, and are maintained alongside the normal GA population. We describe a speedup process that can be applied to the initial generation which greatly reduces the complexity of the initial stages. A split operator is also introduced that is designed to counteract the excessive growth of niches, and it is shown that this improves the overall robustness of the technique. Finally, the effect of local elitism is documented and compared to the performance of the basic DNC technique on a selection of 2D test functions. The paper is concluded with a view to future work to be undertaken on the technique.

A comparative review of dimension reduction methods in approximate Bayesian computation

Approximate Bayesian computation (ABC) methods make use of comparisons between simulated and observed summary statistics to overcome the problem of computationally intractable likelihood functions. As the practical implementation of ABC requires computations based on vectors of summary statistics, rather than full data sets, a central question is how to derive low-dimensional summary statistics from the observed data with minimal loss of information. In this article we provide a comprehensive review and comparison of the performance of the principal methods of dimension reduction proposed in the ABC literature. The methods are split into three nonmutually exclusive classes consisting of best subset selection methods, projection techniques and regularization. In addition, we introduce two new methods of dimension reduction. The first is a best subset selection method based on Akaike and Bayesian information criteria, and the second uses ridge regression as a regularization procedure. We illustrate the performance of these dimension reduction techniques through the analysis of three challenging models and data sets.

Host selection and performance of the giant willow aphid, Tuberolachnus salignus (Gmelin) - Implications for pest management

1 The recent increase in planting of selected willow clones as energy crops for biomass production has resulted in a need to understand the relationship between commonly grown, clonally propagated genotypes and their pests. 2 For the first time, we present a study of the interactions of six willow clones and a previously unconsidered pest, the giant willow aphid Tuberolachnus salignus. 3 Tuberolachnus salignus alatae displayed no preference between the clones, but there was genetic variation in resistance between the clones; Q83 was the most resistant and led to the lowest reproductive performance in the aphid 4 Maternal effects buffered changes in aphid performance. On four tested willow clones fecundity of first generation aphids on the new host clone was intermediate to that of the second generation and that of the clone used to maintain the aphids in culture. 5 In the field, patterns of aphid infestation were highly variable between years, with the duration of attack being up to four times longer in 1999. In both years there was a significant effect of willow clone on the intensity of infestation. However, whereas Orm had the lowest intensity of infestation in the first year, Dasyclados supported a lower population level than other monitored clones in the second year.

Selection of potential probiotic lactic acid bacteria from fermented olives by in vitro tests

The present study aims to evaluate the probiotic potential of lactic acid bacteria (LAB) isolated from naturally fermented olives and select candidates to be used as probiotic starters for the improvement of the traditional fermentation process and the production of newly added value functional foods. Seventy one (71) lactic acid bacterial strains (17 Leuconostoc mesenteroides, 1 Ln. pseudomesenteroides, 13 Lactobacillus plantarum, 37 Lb. pentosus, 1 Lb. paraplantarum, and 2 Lb. paracasei subsp. paracasei) isolated from table olives were screened for their probiotic potential. Lb. rhamnosus GG and Lb. casei Shirota were used as reference strains. The in vitro tests included survival in simulated gastrointestinal tract conditions, antimicrobial activity (against Listeria monocytogenes, Salmonella Enteritidis, Escherichia coli O157:H7), Caco-2 surface adhesion, resistance to 9 antibiotics and haemolytic activity. Three (3) Lb. pentosus, 4 Lb. plantarum and 2 Lb. paracasei subsp. paracasei strains demonstrated the highest final population (>8 log cfu/ml) after 3 h of exposure at low pH. The majority of the tested strains were resistant to bile salts even after 4 h of exposure, while 5 Lb. plantarum and 7 Lb. pentosus strains exhibited partial bile salt hydrolase activity. None of the strains inhibited the growth of the pathogens tested. Variable efficiency to adhere to Caco-2 cells was observed. This was the same regarding strains' susceptibility towards different antibiotics. None of the strains exhibited β-haemolytic activity. As a whole, 4 strains of Lb. pentosus, 3 strains of Lb. plantarum and 2 strains of Lb. paracasei subsp. paracasei were found to possess desirable in vitro probiotic properties similar to or even better than the reference probiotic strains Lb. casei Shirota and Lb. rhamnosus GG. These strains are good candidates for further investigation both with in vivo studies to elucidate their potential health benefits and in olive fermentation processes to assess their technological performance as novel probiotic starters.

A radial basis function network classifier to maximise leave-one-out mutual information

tWe develop an orthogonal forward selection (OFS) approach to construct radial basis function (RBF)network classifiers for two-class problems. Our approach integrates several concepts in probabilisticmodelling, including cross validation, mutual information and Bayesian hyperparameter fitting. At eachstage of the OFS procedure, one model term is selected by maximising the leave-one-out mutual infor-mation (LOOMI) between the classifier’s predicted class labels and the true class labels. We derive theformula of LOOMI within the OFS framework so that the LOOMI can be evaluated efficiently for modelterm selection. Furthermore, a Bayesian procedure of hyperparameter fitting is also integrated into theeach stage of the OFS to infer the l2-norm based local regularisation parameter from the data. Since eachforward stage is effectively fitting of a one-variable model, this task is very fast. The classifier construc-tion procedure is automatically terminated without the need of using additional stopping criterion toyield very sparse RBF classifiers with excellent classification generalisation performance, which is par-ticular useful for the noisy data sets with highly overlapping class distribution. A number of benchmarkexamples are employed to demonstrate the effectiveness of our proposed approach.

Forecasting with Bayesian multivariate vintage-based VARs

We consider the forecasting of macroeconomic variables that are subject to revisions, using Bayesian vintage-based vector autoregressions. The prior incorporates the belief that, after the first few data releases, subsequent ones are likely to consist of revisions that are largely unpredictable. The Bayesian approach allows the joint modelling of the data revisions of more than one variable, while keeping the concomitant increase in parameter estimation uncertainty manageable. Our model provides markedly more accurate forecasts of post-revision values of inflation than do other models in the literature.

Gaussian anamorphosis in the analysis step of the EnKF: a joint state-variable/observation approach

The analysis step of the (ensemble) Kalman filter is optimal when (1) the distribution of the background is Gaussian, (2) state variables and observations are related via a linear operator, and (3) the observational error is of additive nature and has Gaussian distribution. When these conditions are largely violated, a pre-processing step known as Gaussian anamorphosis (GA) can be applied. The objective of this procedure is to obtain state variables and observations that better fulfil the Gaussianity conditions in some sense. In this work we analyse GA from a joint perspective, paying attention to the effects of transformations in the joint state variable/observation space. First, we study transformations for state variables and observations that are independent from each other. Then, we introduce a targeted joint transformation with the objective to obtain joint Gaussianity in the transformed space. We focus primarily in the univariate case, and briefly comment on the multivariate one. A key point of this paper is that, when (1)-(3) are violated, using the analysis step of the EnKF will not recover the exact posterior density in spite of any transformations one may perform. These transformations, however, provide approximations of different quality to the Bayesian solution of the problem. Using an example in which the Bayesian posterior can be analytically computed, we assess the quality of the analysis distributions generated after applying the EnKF analysis step in conjunction with different GA options. The value of the targeted joint transformation is particularly clear for the case when the prior is Gaussian, the marginal density for the observations is close to Gaussian, and the likelihood is a Gaussian mixture.

Calibration and evaluation of individual-based models using Approximate Bayesian Computation

This paper investigates the feasibility of using approximate Bayesian computation (ABC) to calibrate and evaluate complex individual-based models (IBMs). As ABC evolves, various versions are emerging, but here we only explore the most accessible version, rejection-ABC. Rejection-ABC involves running models a large number of times, with parameters drawn randomly from their prior distributions, and then retaining the simulations closest to the observations. Although well-established in some fields, whether ABC will work with ecological IBMs is still uncertain. Rejection-ABC was applied to an existing 14-parameter earthworm energy budget IBM for which the available data consist of body mass growth and cocoon production in four experiments. ABC was able to narrow the posterior distributions of seven parameters, estimating credible intervals for each. ABC’s accepted values produced slightly better fits than literature values do. The accuracy of the analysis was assessed using cross-validation and coverage, currently the best available tests. Of the seven unnarrowed parameters, ABC revealed that three were correlated with other parameters, while the remaining four were found to be not estimable given the data available. It is often desirable to compare models to see whether all component modules are necessary. Here we used ABC model selection to compare the full model with a simplified version which removed the earthworm’s movement and much of the energy budget. We are able to show that inclusion of the energy budget is necessary for a good fit to the data. We show how our methodology can inform future modelling cycles, and briefly discuss how more advanced versions of ABC may be applicable to IBMs. We conclude that ABC has the potential to represent uncertainty in model structure, parameters and predictions, and to embed the often complex process of optimizing an IBM’s structure and parameters within an established statistical framework, thereby making the process more transparent and objective.