Fast kernel classifier construction using orthogonal forward selection to minimise leave-one-out misclassification rate

We propose a simple yet computationally efficient construction algorithm for two-class kernel classifiers. In order to optimise classifier's generalisation capability, an orthogonal forward selection procedure is used to select kernels one by one by minimising the leave-one-out (LOO) misclassification rate directly. It is shown that the computation of the LOO misclassification rate is very efficient owing to orthogonalisation. Examples are used to demonstrate that the proposed algorithm is a viable alternative to construct sparse two-class kernel classifiers in terms of performance and computational efficiency.

A fast linear-in-the-parameters classifier construction algorithm using orthogonal forward selection to minimize leave-one-out misclassification rate

We propose a simple and computationally efficient construction algorithm for two class linear-in-the-parameters classifiers. In order to optimize model generalization, a forward orthogonal selection (OFS) procedure is used for minimizing the leave-one-out (LOO) misclassification rate directly. An analytic formula and a set of forward recursive updating formula of the LOO misclassification rate are developed and applied in the proposed algorithm. Numerical examples are used to demonstrate that the proposed algorithm is an excellent alternative approach to construct sparse two class classifiers in terms of performance and computational efficiency.

Orthogonal forward selection for constructing the radial basis function network with tunable nodes

An orthogonal forward selection (OFS) algorithm based on the leave-one-out (LOO) criterion is proposed for the construction of radial basis function (RBF) networks with tunable nodes. This OFS-LOO algorithm is computationally efficient and is capable of identifying parsimonious RBF networks that generalise well. Moreover, the proposed algorithm is fully automatic and the user does not need to specify a termination criterion for the construction process.

Construction of tunable radial basis function networks using orthogonal forward selection

An orthogonal forward selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO misclassification rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstrated using examples taken from both regression and classification applications.

Kernel classifier construction using orthogonal forward selection and boosting with Fisher ratio class separability measure

A greedy technique is proposed to construct parsimonious kernel classifiers using the orthogonal forward selection method and boosting based on Fisher ratio for class separability measure. Unlike most kernel classification methods, which restrict kernel means to the training input data and use a fixed common variance for all the kernel terms, the proposed technique can tune both the mean vector and diagonal covariance matrix of individual kernel by incrementally maximizing Fisher ratio for class separability measure. An efficient weighted optimization method is developed based on boosting to append kernels one by one in an orthogonal forward selection procedure. Experimental results obtained using this construction technique demonstrate that it offers a viable alternative to the existing state-of-the-art kernel modeling methods for constructing sparse Gaussian radial basis function network classifiers. that generalize well.

Fast-forward/fast-backward substitutions on vector computers

This paper deals with approaches for sparse matrix substitutions using vector processing. Many publications have used the W-matrix method to solve the forward/backward substitutions on vector computer. Recently a different approach has been presented using dependency-based substitution algorithm (DBSA). In this paper the focus is on new algorithms able to explore the sparsity of the vectors. The efficiency is tested using linear systems from power systems with 118, 320, 725 and 1729 buses. The tests were performed on a CRAY Y MP2E/232. The speedups for a fast-forward/fast-backward using a 1729-bus system are near 19 and 14 for real and complex arithmetic operations, respectively. When forward/backward is employed the speedups are about 8 and 6 to perform the same simulations.

Improved damage detectability in a wind turbine blade by optimal selection of vibration signal correlation coefficients

Piotr Omenzetter and Simon Hoell’s work within the Lloyd’s Register Foundation Centre for Safety and Reliability Engineering at the University of Aberdeen is supported by Lloyd’s Register Foundation. The Foundation helps to protect life and property by supporting engineering-related education, public engagement and the application of research.

Improved damage detectability in a wind turbine blade by optimal selection of vibration signal correlation coefficients

Piotr Omenzetter and Simon Hoell’s work within the Lloyd’s Register Foundation Centre for Safety and Reliability Engineering at the University of Aberdeen is supported by Lloyd’s Register Foundation. The Foundation helps to protect life and property by supporting engineering-related education, public engagement and the application of research.

A radial basis function network classifier to maximise leave-one-out mutual information

tWe develop an orthogonal forward selection (OFS) approach to construct radial basis function (RBF)network classifiers for two-class problems. Our approach integrates several concepts in probabilisticmodelling, including cross validation, mutual information and Bayesian hyperparameter fitting. At eachstage of the OFS procedure, one model term is selected by maximising the leave-one-out mutual infor-mation (LOOMI) between the classifier’s predicted class labels and the true class labels. We derive theformula of LOOMI within the OFS framework so that the LOOMI can be evaluated efficiently for modelterm selection. Furthermore, a Bayesian procedure of hyperparameter fitting is also integrated into theeach stage of the OFS to infer the l2-norm based local regularisation parameter from the data. Since eachforward stage is effectively fitting of a one-variable model, this task is very fast. The classifier construc-tion procedure is automatically terminated without the need of using additional stopping criterion toyield very sparse RBF classifiers with excellent classification generalisation performance, which is par-ticular useful for the noisy data sets with highly overlapping class distribution. A number of benchmarkexamples are employed to demonstrate the effectiveness of our proposed approach.

Elastic net orthogonal forward regression

An efficient two-level model identification method aiming at maximising a model׳s generalisation capability is proposed for a large class of linear-in-the-parameters models from the observational data. A new elastic net orthogonal forward regression (ENOFR) algorithm is employed at the lower level to carry out simultaneous model selection and elastic net parameter estimation. The two regularisation parameters in the elastic net are optimised using a particle swarm optimisation (PSO) algorithm at the upper level by minimising the leave one out (LOO) mean square error (LOOMSE). There are two elements of original contributions. Firstly an elastic net cost function is defined and applied based on orthogonal decomposition, which facilitates the automatic model structure selection process with no need of using a predetermined error tolerance to terminate the forward selection process. Secondly it is shown that the LOOMSE based on the resultant ENOFR models can be analytically computed without actually splitting the data set, and the associate computation cost is small due to the ENOFR procedure. Consequently a fully automated procedure is achieved without resort to any other validation data set for iterative model evaluation. Illustrative examples are included to demonstrate the effectiveness of the new approaches.

U-curve: A branch-and-bound optimization algorithm for U-shaped cost functions on Boolean lattices applied to the feature selection problem

This paper presents the formulation of a combinatorial optimization problem with the following characteristics: (i) the search space is the power set of a finite set structured as a Boolean lattice; (ii) the cost function forms a U-shaped curve when applied to any lattice chain. This formulation applies for feature selection in the context of pattern recognition. The known approaches for this problem are branch-and-bound algorithms and heuristics that explore partially the search space. Branch-and-bound algorithms are equivalent to the full search, while heuristics are not. This paper presents a branch-and-bound algorithm that differs from the others known by exploring the lattice structure and the U-shaped chain curves of the search space. The main contribution of this paper is the architecture of this algorithm that is based on the representation and exploration of the search space by new lattice properties proven here. Several experiments, with well known public data, indicate the superiority of the proposed method to the sequential floating forward selection (SFFS), which is a popular heuristic that gives good results in very short computational time. In all experiments, the proposed method got better or equal results in similar or even smaller computational time. (C) 2009 Elsevier Ltd. All rights reserved.

Fast food influences among Hispanic women

The purpose of this study was to evaluate the fast food influences such as restaurant frequency and restaurant choice among Hispanic women residing in Houston Texas. We also evaluated associations between BMI and frequency of fast food consumption. Methods: Data was obtained from the BOUNCE program and baseline data was evaluated from mothers enrolled in the study. Descriptive analysis and Fisher's exact test were conducted to evaluate patterns among fast food selection. Results: Nearly 88 percent of women were classified as overweight or obese, the population was predominately immigrants from Mexico with language preference of Spanish. Factors most influencing restaurant choice included quality of food, restaurant atmosphere, and healthy food availability. No associations were found between BMI and frequency of fast foods, however data show a slight association between duration in the U.S and increase in fast food frequency. Conclusion: Though statics are not statistically significant results demonstrate a possible trend in regards to length of stay and frequency eating out. This should be further explored. ^

Selection bias and the cross-validation of regression models for prediction

Strategies are compared for the development of a linear regression model with stochastic (multivariate normal) regressor variables and the subsequent assessment of its predictive ability. Bias and mean squared error of four estimators of predictive performance are evaluated in simulated samples of 32 population correlation matrices. Models including all of the available predictors are compared with those obtained using selected subsets. The subset selection procedures investigated include two stopping rules, C$\sb{\rm p}$ and S$\sb{\rm p}$, each combined with an 'all possible subsets' or 'forward selection' of variables. The estimators of performance utilized include parametric (MSEP$\sb{\rm m}$) and non-parametric (PRESS) assessments in the entire sample, and two data splitting estimates restricted to a random or balanced (Snee's DUPLEX) 'validation' half sample. The simulations were performed as a designed experiment, with population correlation matrices representing a broad range of data structures.^ The techniques examined for subset selection do not generally result in improved predictions relative to the full model. Approaches using 'forward selection' result in slightly smaller prediction errors and less biased estimators of predictive accuracy than 'all possible subsets' approaches but no differences are detected between the performances of C$\sb{\rm p}$ and S$\sb{\rm p}$. In every case, prediction errors of models obtained by subset selection in either of the half splits exceed those obtained using all predictors and the entire sample.^ Only the random split estimator is conditionally (on $\\beta$) unbiased, however MSEP$\sb{\rm m}$ is unbiased on average and PRESS is nearly so in unselected (fixed form) models. When subset selection techniques are used, MSEP$\sb{\rm m}$ and PRESS always underestimate prediction errors, by as much as 27 percent (on average) in small samples. Despite their bias, the mean squared errors (MSE) of these estimators are at least 30 percent less than that of the unbiased random split estimator. The DUPLEX split estimator suffers from large MSE as well as bias, and seems of little value within the context of stochastic regressor variables.^ To maximize predictive accuracy while retaining a reliable estimate of that accuracy, it is recommended that the entire sample be used for model development, and a leave-one-out statistic (e.g. PRESS) be used for assessment. ^

Changes at the BCLA

As we welcome 2014 we say goodbye to 2013 and I must start with an apology to authors who have submitted papers to CLAE and seen a delay in either the review process or the hard copy publication of their proofed article. The delays were caused by a major hike in the number of submissions to the journal in 2012 that increased further in 2013. In the 12 months leading to the end of October 2011 we had 94 new paper submissions, and for the same period to the end of 2012 the journal had 116 new papers. In 2012 we were awarded an impact factor for the first time and following that the next 12 month period to the end of October 2013 saw a massive increase in submissions with 171 new manuscripts being submitted. This is nearly twice as many papers as 2 years ago and 3 times as many as when I took over as Editor-in-Chief. In addition to this the UK academics will know that 2014 is a REF year (Research Excellence Framework) where universities are judged on their research and one of the major components of this measure remains to be published papers so there is a push to publishing before the REF deadline for counting. The rejection rate at CLAE has gone up too and currently is around 50% (more than double the rejection rate when I took over as Editor-in-Chief). At CLAE the number of pages that we publish each year has remained the same since 2007. When compiling issue 1 for 2014 I chose the papers to be included from the papers that were proofed and ready to go and there were around 200 proofed pages ready, which is enough to fill 3½ issues! At present Elsevier and the BCLA are preparing to increase the number the pages published per issue so that we can clear some of this backlog and remain up to date with the papers published in CLAE. I should add that on line publishing of papers is still available and there may have been review delays but there are no publishing online so authors can still get an epub on line final version of their paper with a DOI (digital object identifier) number enabling the paper to be cited. There are two awards that were made in 2013 that I would like to make special mention of. One was for my good friend Jan Bergmanson, who was awarded an honorary life fellowship of the College of Optometrists. Jan has served on the editorial board of CLAE for many years and in 2013 also celebrated 30 years of his annual ‘Texan Corneal and contact lens meeting’. The other award I wish to mention is Judith Morris, who was the BCLA Gold Medal Award winner in 2013. Judith has had many roles in her career and worked at Moorfields Eye Hospital, the Institute of Optometry and currently at City University. She has been the Europe Middle East and Africa President of IACLE (International Association of Contact Lens Educators) for many years and I think I am correct in saying that Judith is the only person who was President of both the BCLA (1983) and a few years later she was the President College of Optometrists (1989). Judith was also instrumental in introducing Vivien Freeman to the BCLA as they had been friends and Judith suggested that Vivien apply for an administrative job at the BCLA. Fast forward 29 years and in December 2013 Vivien stepped down as Secretary General of the BCLA. I would like to offer my own personal thanks to Vivien for her support of CLAE and of me over the years. The BCLA will not be the same and I wish you well in your future plans. But 2014 brings in a new position to the BCLA – Cheryl Donnelly has been given the new role of Chief Executive Officer. Cheryl was President of the BCLA in 2000 and has previously served on council. I look forward to working with Cheryl and envisage a bright future for the BCLA and CLAE. In this issue we have some great papers including some from authors who have not published with CLAE before. There is a nice paper on contact lens compliance in Nepal which brings home some familiar messages from an emerging market. A paper on how corneal curvature is affected by the use of hydrogel lenses is useful when advising patients how long they should leave their contact lenses out for to avoid seeing changes in refraction or curvature. This is useful information when refracting these patients or pre-laser surgery. There is a useful paper offering tips on fitting bitoric gas permeable lenses post corneal graft and a paper detailing surgery to implant piggyback multifocal intraocular lenses. One fact that I noted from the selection of papers in the current issue is where they were from. In this issue none of the corresponding authors are from the United Kingdom. There are two papers each from the United States, Spain and Iran, and one each from the Netherlands, Ireland, Republic of Korea, Australia and Hong Kong. This is an obvious reflection of the widening interest in CLAE and the BCLA and indicates the new research groups emerging in the field.