Fast cross-validation of kernel fisher discriminant classifiers

Given n training examples, the training of a Kernel Fisher Discriminant (KFD) classifier corresponds to solving a linear system of dimension n. In cross-validating KFD, the training examples are split into 2 distinct subsets for a number of times (L) wherein a subset of m examples is used for validation and the other subset of(n - m) examples is used for training the classifier. In this case L linear systems of dimension (n - m) need to be solved. We propose a novel method for cross-validation of KFD in which instead of solving L linear systems of dimension (n - m), we compute the inverse of an n × n matrix and solve L linear systems of dimension 2m, thereby reducing the complexity when L is large and/or m is small. For typical 10-fold and leave-one-out cross-validations, the proposed algorithm is approximately 4 and (^4/9n) times respectively as efficient as the naive implementations. Simulations are provided to demonstrate the efficiency of the proposed algorithms.

A fast kernel dimension reduction algorithm with applications to face recognition

This paper presents a novel dimensionality reduction algorithm for kernel based classification. In the feature space, the proposed algorithm maximizes the ratio of the squared between-class distance and the sum of the within-class variances of the training samples for a given reduced dimension. This algorithm has lower complexity than the recently reported kernel dimension reduction(KDR) for supervised learning. We conducted several simulations with large training datasets, which demonstrate that the proposed algorithm has similar performance or is marginally better compared with KDR whilst having the advantage of computational efficiency. Further, we applied the proposed dimension reduction algorithm to face recognition in which the number of training samples is very small. This proposed face recognition approach based on the new algorithm outperforms the eigenface approach based on the principle component analysis (PCA), when the training data is complete, that is, representative of the whole dataset.

Identification of nonlinear fMRI models using Auxiliary Particle Filter and kernel smoothing method

Hemodynamic models have a high potential in application to understanding the functional differences of the brain. However, full system identification with respect to model fitting to actual functional magnetic resonance imaging (fMRI) data is practically difficult and is still an active area of research. We present a simulation based Bayesian approach for nonlinear model based analysis of the fMRI data. The idea is to do a joint state and parameter estimation within a general filtering framework. One advantage of using Bayesian methods is that they provide a complete description of the posterior distribution, not just a single point estimate. We use an Auxiliary Particle Filter adjoined with a kernel smoothing approach to address this joint estimation problem.

Face recognition from face motion manifolds using robust kernel resistor-average distance

In this work we consider face recognition from face motion manifolds. An information-theoretic approach with Resistor-Average Distance (RAD) as a dissimilarity measure between distributions of face images is proposed. We introduce a kernel-based algorithm that retains the simplicity of the closed-form expression for the RAD between two normal distributions, while allowing for modelling of complex, nonlinear manifolds. Additionally, it is shown how errors in the face registration process can be modelled to significantly improve recognition. Recognition performance of our method is experimentally demonstrated and shown to outperform state-of-the-art algorithms. Recognition rates of 97–100% are consistently achieved on databases of 35– 90 people.

Topic model kernel : an empirical study towards probabilistically reduced features for classification

Probabilistic topic models have become a standard in modern machine learning with wide applications in organizing and summarizing ‘documents’ in high-dimensional data such as images, videos, texts, gene expression data, and so on. Representing data by dimensional reduction of mixture proportion extracted from topic models is not only richer in semantics than bag-of-word interpretation, but also more informative for classification tasks. This paper describes the Topic Model Kernel (TMK), a high dimensional mapping for Support Vector Machine classification of data generated from probabilistic topic models. The applicability of our proposed kernel is demonstrated in several classification tasks from real world datasets. We outperform existing kernels on the distributional features and give the comparative results on non-probabilistic data types.

The use of kernel density estimation to examine associations between neighborhood destination intensity and walking and physical activity

BACKGROUND: Local destinations have previously been shown to be associated with higher levels of both physical activity and walking, but little is known about how the distribution of destinations is related to activity. Kernel density estimation is a spatial analysis technique that accounts for the location of features relative to each other. Using kernel density estimation, this study sought to investigate whether individuals who live near destinations (shops and service facilities) that are more intensely distributed rather than dispersed: 1) have higher odds of being sufficiently active; 2) engage in more frequent walking for transport and recreation. METHODS: The sample consisted of 2349 residents of 50 urban areas in metropolitan Melbourne, Australia. Destinations within these areas were geocoded and kernel density estimates of destination intensity were created using kernels of 400m (meters), 800m and 1200m. Using multilevel logistic regression, the association between destination intensity (classified in quintiles Q1(least)-Q5(most)) and likelihood of: 1) being sufficiently active (compared to insufficiently active); 2) walking≥4/week (at least 4 times per week, compared to walking less), was estimated in models that were adjusted for potential confounders. RESULTS: For all kernel distances, there was a significantly greater likelihood of walking≥4/week, among respondents living in areas of greatest destinations intensity compared to areas with least destination intensity: 400m (Q4 OR 1.41 95%CI 1.02-1.96; Q5 OR 1.49 95%CI 1.06-2.09), 800m (Q4 OR 1.55, 95%CI 1.09-2.21; Q5, OR 1.71, 95%CI 1.18-2.48) and 1200m (Q4, OR 1.7, 95%CI 1.18-2.45; Q5, OR 1.86 95%CI 1.28-2.71). There was also evidence of associations between destination intensity and sufficient physical activity, however these associations were markedly attenuated when walking was included in the models. CONCLUSIONS: This study, conducted within urban Melbourne, found that those who lived in areas of greater destination intensity walked more frequently, and showed higher odds of being sufficiently physically active-an effect that was largely explained by levels of walking. The results suggest that increasing the intensity of destinations in areas where they are more dispersed; and or planning neighborhoods with greater destination intensity, may increase residents' likelihood of being sufficiently active for health.

Topic model kernel classification with probabilistically reduced features

Probabilistic topic models have become a standard in modern machine learning to deal with a wide range of applications. Representing data by dimensional reduction of mixture proportion extracted from topic models is not only richer in semantics interpretation, but could also be informative for classification tasks. In this paper, we describe the Topic Model Kernel (TMK), a topicbased kernel for Support Vector Machine classification on data being processed by probabilistic topic models. The applicability of our proposed kernel is demonstrated in several classification tasks with real world datasets. TMK outperforms existing kernels on the distributional features and give comparative results on nonprobabilistic data types.

Derivation of dynamic material properties of steel-fibre-reinforced concrete using kernel regression

The reliable and efficient design of steel-fibre-reinforced concrete (SFRC) structures requires clear knowledge of material properties. Since the locations and orientations of aggregates and fibres in concrete are intrinsically random, testing results from different specimens vary, and it needs hundreds or even thousands of specimens and tests to derive the unbiased statistical distributions of material properties by using traditional statistical techniques. Therefore, few statistical studies on the SFRC material properties can be found in literature. In this study, high-rate impact test results on SFRC using split Hopkinson pressure bar are further analysed. The influences of different strain rates and various volume fractions of fibres on compressive strength of SFRC specimens under dynamic loadings will be quantified, by using kernel regression, a kernel-based nonparametric statistical method. Several kernel estimators and functions will be compared. This technique allows one to derive an unbiased statistical estimation from limited testing data. Therefore it is especially useful when the testing data is limited.

Using kernel density estimation to understand the influence of neighbourhood destinations on BMI

OBJECTIVES: Little is known about how the distribution of destinations in the local neighbourhood is related to body mass index (BMI). Kernel density estimation (KDE) is a spatial analysis technique that accounts for the location of features relative to each other. Using KDE, this study investigated whether individuals living near destinations (shops and service facilities) that are more intensely distributed rather than dispersed, have lower BMIs.

STUDY DESIGN AND SETTING: A cross-sectional study of 2349 residents of 50 urban areas in metropolitan Melbourne, Australia.

METHODS: Destinations were geocoded, and kernel density estimates of destination intensity were created using kernels of 400, 800 and 1200 m. Using multilevel linear regression, the association between destination intensity (classified in quintiles Q1(least)-Q5(most)) and BMI was estimated in models that adjusted for the following confounders: age, sex, country of birth, education, dominant household occupation, household type, disability/injury and area disadvantage. Separate models included a physical activity variable.

RESULTS: For kernels of 800 and 1200 m, there was an inverse relationship between BMI and more intensely distributed destinations (compared to areas with least destination intensity). Effects were significant at 1200 m: Q4, β -0.86, 95% CI -1.58 to -0.13, p=0.022; Q5, β -1.03 95% CI -1.65 to -0.41, p=0.001. Inclusion of physical activity in the models attenuated effects, although effects remained marginally significant for Q5 at 1200 m: β -0.77 95% CI -1.52, -0.02, p=0.045.

CONCLUSIONS: This study conducted within urban Melbourne, Australia, found that participants living in areas of greater destination intensity within 1200 m of home had lower BMIs. Effects were partly explained by physical activity. The results suggest that increasing the intensity of destination distribution could reduce BMI levels by encouraging higher levels of physical activity.