Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

Gene selection by cooperative competition clustering

Clustering analysis of data from DNA microarray hybridization studies is an essential task for identifying biologically relevant groups of genes. Attribute cluster algorithm (ACA) has provided an attractive way to group and select meaningful genes. However, ACA needs much prior knowledge about the genes to set the number of clusters. In practical applications, if the number of clusters is misspecified, the performance of the ACA will deteriorate rapidly. In fact, it is a very demanding to do that because of our little knowledge. We propose the Cooperative Competition Cluster Algorithm (CCCA) in this paper. In the algorithm, we assume that both cooperation and competition exist simultaneously between clusters in the process of clustering. By using this principle of Cooperative Competition, the number of clusters can be found in the process of clustering. Experimental results on a synthetic and gene expression data are demonstrated. The results show that CCCA can choose the number of clusters automatically and get excellent performance with respect to other competing methods.

Parent-off spring associations in wintering brent geese: parental investment or mutual assistance?

The continued parent-offspring associations in the Eastern Canadian High Arctic light-bellied brent goose Branta bernicla hrota was examined to determine whether this is an example of continued parental investment or mutual assistance. Adults with juveniles spend more than twice as much time being vigilant and aggressive than do those without offspring. The loss of a partner, however, does not result in the remaining parent increasing parental care but does result in increased 'self-care' by the juveniles. Neither parents nor single-parent juveniles appear to pay an energetic cost relative to non-parental adults and two-parent juveniles, respectively. Differences in the feeding distribution of parents and non-parents and equivalent or better physical condition suggests that families are able to maintain access to a superior food supply over the winter. Passive 'assistance' by juveniles may assist in maintaining this position in favoured areas, and this is achieved with little overt aggression. The present study thus provides no data that show a net cost to parents by remaining with their juveniles over the winter period. Thus, mutual assistance might be a better explanation of the prolonged association rather than a period of parental investment with an overall cost.

Shared or unshared parental care in overwintering brent geese (Branta bernicla hrota)

We examine brood size effects on the behaviour of wintering parent and juvenile brent geese (Branta bernicla hrota) to test predictions of shared and unshared parental care models. The behaviour of both parents and offspring appear to be influenced by declining food availability over the winter. Parental vigilance increased with brood size and may be explained by vigilance having functions in addition to antipredator behaviour where the benefits are shared among the brood. There was no increase in parental aggression with brood size and this does not fit the prediction of shared care. Nevertheless, large families are able to monopolize better feeding areas compared with smaller families and large families static feed more but walk feed less than do small families, the former apparently being the preferred mode. The presence of additional young, rather than increasing the amount of parental aggression, seems to enhance the family's competitive ability. Because parents with large broods benefit from enhanced access to resources there is likely to be no additional significant cost in the parental care of larger broods (sensu Trivers 1972).

Integrated structure selection and parameter optimisation for eng-genes neural models

The eng-genes concept involves the use of fundamental known system functions as activation functions in a neural model to create a 'grey-box' neural network. One of the main issues in eng-genes modelling is to produce a parsimonious model given a model construction criterion. The challenges are that (1) the eng-genes model in most cases is a heterogenous network consisting of more than one type of nonlinear basis functions, and each basis function may have different set of parameters to be optimised; (2) the number of hidden nodes has to be chosen based on a model selection criterion. This is a mixed integer hard problem and this paper investigates the use of a forward selection algorithm to optimise both the network structure and the parameters of the system-derived activation functions. Results are included from case studies performed on a simulated continuously stirred tank reactor process, and using actual data from a pH neutralisation plant. The resulting eng-genes networks demonstrate superior simulation performance and transparency over a range of network sizes when compared to conventional neural models. (c) 2007 Elsevier B.V. All rights reserved.

Desulfurisation of oils using ionic liquids: selection of cationic and anionic components to enhance extraction efficiency

Extraction of dibenzothiophene from dodecane using ionic liquids as the extracting phase has been investigated for a range of ionic liquids with varying cation classes (imidazolium, pyridinium, and pyrrolidinium) and a range of anion types using liquid-liquid partition studies and QSPR (quantitative structure-activity relationship) analysis. The partition ratio of dibenzothiophene to the ionic liquids showed a clear variation with cation class (dimethylpyridinium > methylpyridinium > pyridinium approximate to imidazolium approximate to pyrrolidinium), with much less significant variation with anion type. Polyaromatic quinolinium-based ionic liquids showed even greater extraction potential, but were compromised by higher melting points. For example, 1-butyl-6-methylquinolinium bis{(trifluoromethyl)sulfonyl} amide (mp 47 degrees C) extracted 90% of the available dibenzothiophene from dodecane at 60 degrees C.

Selection of an analytical method for evaluating bovine serum albumin concentrations in pharmaceutical polymeric formulations

Bovine serum albumin (BSA) is a commonly used model protein in the development of pharmaceutical formulations. In order to assay its release from various dosage forms, either the bicinchoninic acid (BCA) assay or a more specific size-exclusion high performance liquid chromatography (SE-HPLC) method are commonly employed. However, these can give erroneous results in the presence of some commonly-used pharmaceutical excipients. We therefore investigated the ability of these methods to accurately determine BSA concentrations in pharmaceutical formulations that also contained various polymers and compared them with a new and compared with a new reverse-phase (RP)–HPLC technique. We found that the RP-HPLC technique was the most suitable method. It gave a linear response in the range of 0.5 -100 µg/ml with a correlation coefficient of 0.9999, a limit of detection of 0.11 µg/ml and quantification of 0.33 µg/ml. The performed ‘t’ test for the estimated and theoretical concentration indicated no significant difference between them providing the accuracy. Low % relative standard deviation values (0.8-1.39%) indicate the precision of the method. Furthermore, the method was used to quantify in vitro BSA release from polymeric freeze-dried formulations.

Deciding between accounts of the selection task: A reply to Oaksford (2002)

In this paper we report on our attempts to fit the optimal data selection (ODS) model (Oaksford Chater, 1994; Oaksford, Chater, & Larkin, 2000) to the selection task data reported in Feeney and Handley (2000) and Handley, Feeney, and Harper (2002). Although Oaksford (2002b) reports good fits to the data described in Feeney and Handley (2000), the model does not adequately capture the data described in Handley et al. (2002). Furthermore, across all six of the experiments modelled here, the ODS model does not predict participants' behaviour at the level of selection rates for individual cards. Finally, when people's probability estimates are used in the modelling exercise, the model adequately captures only I out of 18 conditions described in Handley et al. We discuss the implications of these results for models of the selection task and claim that they support deductive, rather than probabilistic, accounts of the task.