Sparse kernel regression modeling using combined locally regularized orthogonal least squares and D-optimality experimental design

The note proposes an efficient nonlinear identification algorithm by combining a locally regularized orthogonal least squares (LROLS) model selection with a D-optimality experimental design. The proposed algorithm aims to achieve maximized model robustness and sparsity via two effective and complementary approaches. The LROLS method alone is capable of producing a very parsimonious model with excellent generalization performance. The D-optimality design criterion further enhances the model efficiency and robustness. An added advantage is that the user only needs to specify a weighting for the D-optimality cost in the combined model selecting criterion and the entire model construction procedure becomes automatic. The value of this weighting does not influence the model selection procedure critically and it can be chosen with ease from a wide range of values.

Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization

This paper presents an efficient construction algorithm for obtaining sparse kernel density estimates based on a regression approach that directly optimizes model generalization capability. Computational efficiency of the density construction is ensured using an orthogonal forward regression, and the algorithm incrementally minimizes the leave-one-out test score. A local regularization method is incorporated naturally into the density construction process to further enforce sparsity. An additional advantage of the proposed algorithm is that it is fully automatic and the user is not required to specify any criterion to terminate the density construction procedure. This is in contrast to an existing state-of-art kernel density estimation method using the support vector machine (SVM), where the user is required to specify some critical algorithm parameter. Several examples are included to demonstrate the ability of the proposed algorithm to effectively construct a very sparse kernel density estimate with comparable accuracy to that of the full sample optimized Parzen window density estimate. Our experimental results also demonstrate that the proposed algorithm compares favorably with the SVM method, in terms of both test accuracy and sparsity, for constructing kernel density estimates.

An orthogonal forward regression technique for sparse kernel density estimation

Using the classical Parzen window (PW) estimate as the desired response, the kernel density estimation is formulated as a regression problem and the orthogonal forward regression technique is adopted to construct sparse kernel density (SKD) estimates. The proposed algorithm incrementally minimises a leave-one-out test score to select a sparse kernel model, and a local regularisation method is incorporated into the density construction process to further enforce sparsity. The kernel weights of the selected sparse model are finally updated using the multiplicative nonnegative quadratic programming algorithm, which ensures the nonnegative and unity constraints for the kernel weights and has the desired ability to reduce the model size further. Except for the kernel width, the proposed method has no other parameters that need tuning, and the user is not required to specify any additional criterion to terminate the density construction procedure. Several examples demonstrate the ability of this simple regression-based approach to effectively construct a SKID estimate with comparable accuracy to that of the full-sample optimised PW density estimate. (c) 2007 Elsevier B.V. All rights reserved.

A kernel-based two-class classifier for imbalanced data sets

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm.

A forward-constrained regression algorithm for sparse kernel density estimation

Using the classical Parzen window (PW) estimate as the target function, the sparse kernel density estimator is constructed in a forward-constrained regression (FCR) manner. The proposed algorithm selects significant kernels one at a time, while the leave-one-out (LOO) test score is minimized subject to a simple positivity constraint in each forward stage. The model parameter estimation in each forward stage is simply the solution of jackknife parameter estimator for a single parameter, subject to the same positivity constraint check. For each selected kernels, the associated kernel width is updated via the Gauss-Newton method with the model parameter estimate fixed. The proposed approach is simple to implement and the associated computational cost is very low. Numerical examples are employed to demonstrate the efficacy of the proposed approach.

Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

Movement time for motion-impaired users assisted by force-feedback: effects of movement amplitude, target width and gravity well width

This paper presents a study investigating how the performance of motion-impaired computer users in point and click tasks varies with target distance (A), target width (W), and force-feedback gravity well width (GWW). Six motion-impaired users performed point and click tasks across a range of values for A, W, and GWW. Times were observed to increase with A, and to decrease with W. Times also improved with GWW, and, with the addition of a gravity well, a greater improvement was observed for smaller targets than for bigger ones. It was found that Fitts Law gave a good description of behaviour for each value of GWW, and that gravity wells reduced the effect of task difficulty on performance. A model based on Fitts Law is proposed, which incorporates the effect of GWW on movement time. The model accounts for 88.8% of the variance in the observed data.

Wigner Transport models of the electron-phonon kinetics in quantum wires

Two quantum-kinetic models of ultrafast electron transport in quantum wires are derived from the generalized electron-phonon Wigner equation. The various assumptions and approximations allowing one to find closed equations for the reduced electron Wigner function are discussed with an emphasis on their physical relevance. The models correspond to the Levinson and Barker-Ferry equations, now generalized to account for a space-dependent evolution. They are applied to study the quantum effects in the dynamics of an initial packet of highly nonequilibrium carriers, locally generated in the wire. The properties of the two model equations are compared and analyzed.

Sparse kernel density estimation technique based on zero-norm constraint

A sparse kernel density estimator is derived based on the zero-norm constraint, in which the zero-norm of the kernel weights is incorporated to enhance model sparsity. The classical Parzen window estimate is adopted as the desired response for density estimation, and an approximate function of the zero-norm is used for achieving mathemtical tractability and algorithmic efficiency. Under the mild condition of the positive definite design matrix, the kernel weights of the proposed density estimator based on the zero-norm approximation can be obtained using the multiplicative nonnegative quadratic programming algorithm. Using the -optimality based selection algorithm as the preprocessing to select a small significant subset design matrix, the proposed zero-norm based approach offers an effective means for constructing very sparse kernel density estimates with excellent generalisation performance.

Regression based D-optimality experimental design for sparse kernel density estimation

This paper derives an efficient algorithm for constructing sparse kernel density (SKD) estimates. The algorithm first selects a very small subset of significant kernels using an orthogonal forward regression (OFR) procedure based on the D-optimality experimental design criterion. The weights of the resulting sparse kernel model are then calculated using a modified multiplicative nonnegative quadratic programming algorithm. Unlike most of the SKD estimators, the proposed D-optimality regression approach is an unsupervised construction algorithm and it does not require an empirical desired response for the kernel selection task. The strength of the D-optimality OFR is owing to the fact that the algorithm automatically selects a small subset of the most significant kernels related to the largest eigenvalues of the kernel design matrix, which counts for the most energy of the kernel training data, and this also guarantees the most accurate kernel weight estimate. The proposed method is also computationally attractive, in comparison with many existing SKD construction algorithms. Extensive numerical investigation demonstrates the ability of this regression-based approach to efficiently construct a very sparse kernel density estimate with excellent test accuracy, and our results show that the proposed method compares favourably with other existing sparse methods, in terms of test accuracy, model sparsity and complexity, for constructing kernel density estimates.

The climatology of the middle atmosphere in a vertically extended version of the Met Office's climate model. Part I: mean state

The climatology of a stratosphere-resolving version of the Met Office’s climate model is studied and validated against ECMWF reanalysis data. Ensemble integrations are carried out at two different horizontal resolutions. Along with a realistic climatology and annual cycle in zonal mean zonal wind and temperature, several physical effects are noted in the model. The time of final warming of the winter polar vortex is found to descend monotonically in the Southern Hemisphere, as would be expected for purely radiative forcing. In the Northern Hemisphere, however, the time of final warming is driven largely by dynamical effects in the lower stratosphere and radiative effects in the upper stratosphere, leading to the earliest transition to westward winds being seen in the midstratosphere. A realistic annual cycle in stratospheric water vapor concentrations—the tropical “tape recorder”—is captured. Tropical variability in the zonal mean zonal wind is found to be in better agreement with the reanalysis for the model run at higher horizontal resolution because the simulated quasi-biennial oscillation has a more realistic amplitude. Unexpectedly, variability in the extratropics becomes less realistic under increased resolution because of reduced resolved wave drag and increased orographic gravity wave drag. Overall, the differences in climatology between the simulations at high and moderate horizontal resolution are found to be small.

A model study of factors influencing projected changes in regional sea level over the 21st century

In addition to projected increases in global mean sea level over the 21st century, model simulations suggest there will also be changes in the regional distribution of sea level relative to the global mean. There is a considerable spread in the projected patterns of these changes by current models, as shown by the recent Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment (AR4). This spread has not reduced from that given by the Third Assessment models. Comparison with projections by ensembles of models based on a single structure supports an earlier suggestion that models of similar formulation give more similar patterns of sea level change. Analysing an AR4 ensemble of model projections under a business-as-usual scenario shows that steric changes (associated with subsurface ocean density changes) largely dominate the sea level pattern changes. The relative importance of subsurface temperature or salinity changes in contributing to this differs from region to region and, to an extent, from model-to-model. In general, thermosteric changes give the spatial variations in the Southern Ocean, halosteric changes dominate in the Arctic and strong compensation between thermosteric and halosteric changes characterises the Atlantic. The magnitude of sea level and component changes in the Atlantic appear to be linked to the amount of Atlantic meridional overturning circulation (MOC) weakening. When the MOC weakening is substantial, the Atlantic thermosteric patterns of change arise from a dominant role of ocean advective heat flux changes.

Classification of instability modes in a model of aluminium reduction cells with a uniform magnetic field

A unified view on the interfacial instability in a model of aluminium reduction cells in the presence of a uniform, vertical, background magnetic field is presented. The classification of instability modes is based on the asymptotic theory for high values of parameter β, which characterises the ratio of the Lorentz force based on the disturbance current, and gravity. It is shown that the spectrum of the travelling waves consists of two parts independent of the horizontal cross-section of the cell: highly unstable wall modes and stable or weakly unstable centre, or Sele’s modes. The wall modes with the disturbance of the interface being localised at the sidewalls of the cell dominate the dynamics of instability. Sele’s modes are characterised by a distributed disturbance over the whole horizontal extent of the cell. As β increases these modes are stabilized by the field.

Time-resolved gas-phase kinetic, quantum chemical and RRKM studies of reactions of silylene with cyclic ethers

Time-resolved kinetic studies of silylene, SiH2, generated by laser flash photolysis of phenylsilane, have been carried out to obtain rate constants for its bimolecular reactions with oxirane, oxetane, and tetrahydrofuran (THF). The reactions were studied in the gas phase over the pressure range 1-100 Torr in SF6 bath gas, at four or five temperatures in the range 294-605 K. All three reactions showed pressure dependences characteristic of third-body-assisted association reactions with, surprisingly, SiH2 + oxirane showing the least and SiH2 + THF showing the most pressure dependence. The second-order rate constants obtained by extrapolation to the high-pressure limits at each temperature fitted the Arrhenius equations where the error limits are single standard deviations: log(k(oxirane)(infinity)/cm(3) molecule(-1) s(-1)) = (-11.03 +/- 0.07) + (5.70 +/- 0.51) kJ mol(-1)/RT In 10 log(k(oxetane)(infinity)/cm(3) molecule(-1) s(-1)) = (-11.17 +/- 0.11) + (9.04 +/- 0.78) kJ mol(-1)/RT In 10 log(k(THF)(infinity)/cm(3) molecule(-1) s(-1)) = (-10.59 +/- 0.10) + (5.76 +/- 0.65) kJ mol(-1)/RT In 10 Binding-energy values of 77, 97, and 92 kJ mol(-1) have been obtained for the donor-acceptor complexes of SiH2 with oxirane, oxetane, and THF, respectively, by means of quantum chemical (ab initio) calculations carried Out at the G3 level. The use of these values to model the pressure dependences of these reactions, via RRKM theory, provided a good fit only in the case of SiH2 + THF. The lack of fit in the other two cases is attributed to further reaction pathways for the association complexes of SiH2 with oxirane and oxetane. The finding of ethene as a product of the SiH2 + oxirane reaction supports a pathway leading to H2Si=O + C2H4 predicted by the theoretical calculations of Apeloig and Sklenak.

Sensitivity of GCM tropical middle atmosphere variability and climate to ozone and parametrized gravity wave changes

This paper describes the impact of changing the current imposed ozone climatology upon the tropical Quasi-Biennial Oscillation (QBO) in a high top climate configuration of the Met Office U.K. general circulation model. The aim is to help distinguish between QBO changes in chemistry climate models that result from temperature-ozone feedbacks and those that might be forced by differences in climatology between previously fixed and newly interactive ozone distributions. Different representations of zonal mean ozone climatology under present-day conditions are taken to represent the level of change expected between acceptable model realizations of the global ozone distribution and thus indicate whether more detailed investigation of such climatology issues might be required when assessing ozone feedbacks. Tropical stratospheric ozone concentrations are enhanced relative to the control climatology between 20–30 km, reduced from 30–40 km and enhanced above, impacting the model profile of clear-sky radiative heating, in particular warming the tropical stratosphere between 15–35 km. The outcome is consistent with a localized equilibrium response in the tropical stratosphere that generates increased upwelling between 100 and 4 hPa, sufficient to account for a 12 month increase of modeled mean QBO period. This response has implications for analysis of the tropical circulation in models with interactive ozone chemistry because it highlights the possibility that plausible changes in the ozone climatology could have a sizable impact upon the tropical upwelling and QBO period that ought to be distinguished from other dynamical responses such as ozone-temperature feedbacks.