M-estimator, and D-optimality model construction using orthogonal forward regression

This correspondence introduces a new orthogonal forward regression (OFR) model identification algorithm using D-optimality for model structure selection and is based on an M-estimators of parameter estimates. M-estimator is a classical robust parameter estimation technique to tackle bad data conditions such as outliers. Computationally, The M-estimator can be derived using an iterative reweighted least squares (IRLS) algorithm. D-optimality is a model structure robustness criterion in experimental design to tackle ill-conditioning in model Structure. The orthogonal forward regression (OFR), often based on the modified Gram-Schmidt procedure, is an efficient method incorporating structure selection and parameter estimation simultaneously. The basic idea of the proposed approach is to incorporate an IRLS inner loop into the modified Gram-Schmidt procedure. In this manner, the OFR algorithm for parsimonious model structure determination is extended to bad data conditions with improved performance via the derivation of parameter M-estimators with inherent robustness to outliers. Numerical examples are included to demonstrate the effectiveness of the proposed algorithm.

A tunable radial basis function model for nonlinear system identification using particle swarm optimisation

A tunable radial basis function (RBF) network model is proposed for nonlinear system identification using particle swarm optimisation (PSO). At each stage of orthogonal forward regression (OFR) model construction, PSO optimises one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is computationally more efficient.

An evaluation of a Bayesian method of dose escalation based on bivariate binary responses

Recently, various approaches have been suggested for dose escalation studies based on observations of both undesirable events and evidence of therapeutic benefit. This article concerns a Bayesian approach to dose escalation that requires the user to make numerous design decisions relating to the number of doses to make available, the choice of the prior distribution, the imposition of safety constraints and stopping rules, and the criteria by which the design is to be optimized. Results are presented of a substantial simulation study conducted to investigate the influence of some of these factors on the safety and the accuracy of the procedure with a view toward providing general guidance for investigators conducting such studies. The Bayesian procedures evaluated use logistic regression to model the two responses, which are both assumed to be binary. The simulation study is based on features of a recently completed study of a compound with potential benefit to patients suffering from inflammatory diseases of the lung.

Nonlinear system identification using particle swarm optimisation tuned radial basis function models

A novel particle swarm optimisation (PSO) tuned radial basis function (RBF) network model is proposed for identification of non-linear systems. At each stage of orthogonal forward regression (OFR) model construction process, PSO is adopted to tune one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is often more efficient in model construction. The effectiveness of the proposed PSO aided OFR algorithm for constructing tunable node RBF models is demonstrated using three real data sets.

Dynamic analysis of house price diffusion across Asian financial centres

The aim of this paper is to explore effects of macroeconomic variables on house prices and also, the lead-lag relationships of real estate markets to examine house price diffusion across Asian financial centres. The analysis is based on the Global Vector Auto-Regression (GVAR) model estimated using quarterly data for six Asian financial centres (Hong Kong, Tokyo, Seoul, Singapore, Taipei and Bangkok) from 1991Q1 to 2011Q2. The empirical results indicate that the global economic conditions play significant roles in shaping house price movements across Asian financial centres. In particular, a small open economy that heavily relies on international trade such as – Singapore and Tokyo - shows positive correlations between economy’s openness and house prices, consistent with the Balassa-Samuelson hypothesis in international trade. However, region-specific conditions do play important roles as determinants of house prices, partly due to restrictive housing policies and demand-supply imbalances, as found in Singapore and Bangkok.

Pollen-based continental climate reconstructions at 6 and 21 ka: a global synthesis

Subfossil pollen and plant macrofossil data derived from 14C-dated sediment profiles can provide quantitative information on glacial and interglacial climates. The data allow climate variables related to growing season warmth, winter cold, and plant-available moisture to be reconstructed. Continental-scale reconstructions have been made for the mid-Holocene (MH, around 6 ka) and Last Glacial Maximum (LGM, around 21 ka), allowing comparison with palaeoclimate simulations currently being carried out as part of the fifth Assessment Report (AR5) of the Intergovernmental Panel on Climate Change. The synthesis of the available MH and LGM climate reconstructions and their uncertainties, obtained using modern analogue, regression and model-inversion techniques, is presented for four temperature variables and two moisture variables. Reconstructions of the same variables based on surface-pollen assemblages are shown to be accurate and unbiased. Reconstructed LGM and MH climate anomaly patterns are coherent, consistent between variables, and robust with respect to the choice of technique. They support a conceptual model of the controls of Late Quaternary climate change whereby the first-order effects of orbital variations and greenhouse forcing on the seasonal cycle of temperature are predictably modified by responses of the atmospheric circulation and surface energy balance.

Sparse model identification using orthogonal forward regression with basis pursuit and D-optimality

An efficient model identification algorithm for a large class of linear-in-the-parameters models is introduced that simultaneously optimises the model approximation ability, sparsity and robustness. The derived model parameters in each forward regression step are initially estimated via the orthogonal least squares (OLS), followed by being tuned with a new gradient-descent learning algorithm based on the basis pursuit that minimises the l(1) norm of the parameter estimate vector. The model subset selection cost function includes a D-optimality design criterion that maximises the determinant of the design matrix of the subset to ensure model robustness and to enable the model selection procedure to automatically terminate at a sparse model. The proposed approach is based on the forward OLS algorithm using the modified Gram-Schmidt procedure. Both the parameter tuning procedure, based on basis pursuit, and the model selection criterion, based on the D-optimality that is effective in ensuring model robustness, are integrated with the forward regression. As a consequence the inherent computational efficiency associated with the conventional forward OLS approach is maintained in the proposed algorithm. Examples demonstrate the effectiveness of the new approach.

Robust nonlinear model identification methods using forward regression

In this correspondence new robust nonlinear model construction algorithms for a large class of linear-in-the-parameters models are introduced to enhance model robustness via combined parameter regularization and new robust structural selective criteria. In parallel to parameter regularization, we use two classes of robust model selection criteria based on either experimental design criteria that optimizes model adequacy, or the predicted residual sums of squares (PRESS) statistic that optimizes model generalization capability, respectively. Three robust identification algorithms are introduced, i.e., combined A- and D-optimality with regularized orthogonal least squares algorithm, respectively; and combined PRESS statistic with regularized orthogonal least squares algorithm. A common characteristic of these algorithms is that the inherent computation efficiency associated with the orthogonalization scheme in orthogonal least squares or regularized orthogonal least squares has been extended such that the new algorithms are computationally efficient. Numerical examples are included to demonstrate effectiveness of the algorithms.

Backward elimination model construction for regression and classification using leave-one-out criteria

A fundamental principle in practical nonlinear data modeling is the parsimonious principle of constructing the minimal model that explains the training data well. Leave-one-out (LOO) cross validation is often used to estimate generalization errors by choosing amongst different network architectures (M. Stone, "Cross validatory choice and assessment of statistical predictions", J. R. Stast. Soc., Ser. B, 36, pp. 117-147, 1974). Based upon the minimization of LOO criteria of either the mean squares of LOO errors or the LOO misclassification rate respectively, we present two backward elimination algorithms as model post-processing procedures for regression and classification problems. The proposed backward elimination procedures exploit an orthogonalization procedure to enable the orthogonality between the subspace as spanned by the pruned model and the deleted regressor. Subsequently, it is shown that the LOO criteria used in both algorithms can be calculated via some analytic recursive formula, as derived in this contribution, without actually splitting the estimation data set so as to reduce computational expense. Compared to most other model construction methods, the proposed algorithms are advantageous in several aspects; (i) There are no tuning parameters to be optimized through an extra validation data set; (ii) The procedure is fully automatic without an additional stopping criteria; and (iii) The model structure selection is directly based on model generalization performance. The illustrative examples on regression and classification are used to demonstrate that the proposed algorithms are viable post-processing methods to prune a model to gain extra sparsity and improved generalization.

Automatic nonlinear predictive model-construction algorithm using forward regression and the PRESS statistic

An automatic nonlinear predictive model-construction algorithm is introduced based on forward regression and the predicted-residual-sums-of-squares (PRESS) statistic. The proposed algorithm is based on the fundamental concept of evaluating a model's generalisation capability through crossvalidation. This is achieved by using the PRESS statistic as a cost function to optimise model structure. In particular, the proposed algorithm is developed with the aim of achieving computational efficiency, such that the computational effort, which would usually be extensive in the computation of the PRESS statistic, is reduced or minimised. The computation of PRESS is simplified by avoiding a matrix inversion through the use of the orthogonalisation procedure inherent in forward regression, and is further reduced significantly by the introduction of a forward-recursive formula. Based on the properties of the PRESS statistic, the proposed algorithm can achieve a fully automated procedure without resort to any other validation data set for iterative model evaluation. Numerical examples are used to demonstrate the efficacy of the algorithm.

Survival of Lactobacillus plantarum in model solutions and fruit juices

The aim of the work was to study the survival of Lactobacillus plantarum NCIMB 8826 in model solutions and develop a mathematical model describing its dependence on pH, citric acid and ascorbic acid. A Central Composite Design (CCD) was developed studying each of the three factors at five levels within the following ranges, i.e., pH (3.0-4.2), citric acid (6-40 g/L), and ascorbic acid (100-1000 mg/L). In total, 17 experimental runs were carried out. The initial cell concentration in the model solutions was approximately 1 × 10(8)CFU/mL; the solutions were stored at 4°C for 6 weeks. Analysis of variance (ANOVA) of the stepwise regression demonstrated that a second order polynomial model fits well the data. The results demonstrated that high pH and citric acid concentration enhanced cell survival; one the other hand, ascorbic acid did not have an effect. Cell survival during storage was also investigated in various types of juices, including orange, grapefruit, blackcurrant, pineapple, pomegranate, cranberry and lemon juice. The model predicted well the cell survival in orange, blackcurrant and pineapple, however it failed to predict cell survival in grapefruit and pomegranate, indicating the influence of additional factors, besides pH and citric acid, on cell survival. Very good cell survival (less than 0.4 log decrease) was observed after 6 weeks of storage in orange, blackcurrant and pineapple juice, all of which had a pH of about 3.8. Cell survival in cranberry and pomegranate decreased very quickly, whereas in the case of lemon juice, the cell concentration decreased approximately 1.1 logs after 6 weeks of storage, albeit the fact that lemon juice had the lowest pH (pH~2.5) among all the juices tested. Taking into account the results from the compositional analysis of the juices and the model, it was deduced that in certain juices, other compounds seemed to protect the cells during storage; these were likely to be proteins and dietary fibre In contrast, in certain juices, such as pomegranate, cell survival was much lower than expected; this could be due to the presence of antimicrobial compounds, such as phenolic compounds.

Estimation of Gaussian process regression model using probability distance measures

A new class of parameter estimation algorithms is introduced for Gaussian process regression (GPR) models. It is shown that the integration of the GPR model with probability distance measures of (i) the integrated square error and (ii) Kullback–Leibler (K–L) divergence are analytically tractable. An efficient coordinate descent algorithm is proposed to iteratively estimate the kernel width using golden section search which includes a fast gradient descent algorithm as an inner loop to estimate the noise variance. Numerical examples are included to demonstrate the effectiveness of the new identification approaches.

A fast radiative transfer model for the assimilation of water vapor radiances from the Kalpana very high resolution radiometer

A fast radiative transfer model (RTM) to compute emitted infrared radiances for a very high resolution radiometer (VHRR), onboard the operational Indian geostationary satellite Kalpana has been developed and verified. This work is a step towards the assimilation of Kalpana water vapor (WV) radiances into numerical weather prediction models. The fast RTM uses a regression‐based approach to parameterize channel‐specific convolved level to space transmittances. A comparison between the fast RTM and the line‐by‐line RTM demonstrated that the fast RTM can simulate line‐by‐line radiances for the Kalpana WV channel to an accuracy better than the instrument noise, while offering more rapid radiance calculations. A comparison of clear sky radiances of the Kalpana WV channel with the ECMWF model first guess radiances is also presented, aiming to demonstrate the fast RTM performance with the real observations. In order to assimilate the radiances from Kalpana, a simple scheme for bias correction has been suggested.

The interannual variability of energy transports within and over the Atlantic Ocean in a coupled climate model

To gain a new perspective on the interaction of the Atlantic Ocean and the atmosphere, the relationship between the atmospheric and oceanic meridional energy transports is studied in a version of HadCM3, the U.K. Hadley Centre's coupled climate model. The correlation structure of the energy transports in the atmosphere and Atlantic Ocean as a function of latitude, and the cross correlation between the two systems are analyzed. The processes that give rise to the correlations are then elucidated using regression analyses. In northern midlatitudes, the interannual variability of the Atlantic Ocean energy transport is dominated by Ekman processes. Anticorrelated zonal winds in the subtropics and midlatitudes, particularly associated with the North Atlantic Oscillation (NAO), drive anticorrelated meridional Ekman transports. Variability in the atmospheric energy transport is associated with changes in the stationary waves, but is only weakly related to the NAO. Nevertheless, atmospheric driving of the oceanic Ekman transports is responsible for a bipolar pattern in the correlation between the atmosphere and Atlantic Ocean energy transports. In the Tropics, the interannual variability of the Atlantic Ocean energy transport is dominated by an adjustment of the tropical ocean to coastal upwelling induced along the Venezuelan coast by a strengthening of the easterly trade winds. Variability in the atmospheric energy transport is associated with a cross-equatorial meridional overturning circulation that is only weakly associated with variability in the trade winds along the Venezuelan coast. In consequence, there is only very limited correlation between the atmosphere and Atlantic Ocean energy transports in the Tropics of HadCM3