Learning the Kernel Matrix with Semi-Definite Programming

Kernel-based learning algorithms work by embedding the data into a Euclidean space, and then searching for linear relations among the embedded data points. The embedding is performed implicitly, by specifying the inner products between each pair of points in the embedding space. This information is contained in the so-called kernel matrix, a symmetric and positive definite matrix that encodes the relative positions of all points. Specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space -- classical model selection problems in machine learning. In this paper we show how the kernel matrix can be learned from data via semi-definite programming (SDP) techniques. When applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm -- using the labelled part of the data one can learn an embedding also for the unlabelled part. The similarity between test points is inferred from training points and their labels. Importantly, these learning problems are convex, so we obtain a method for learning both the model class and the function without local minima. Furthermore, this approach leads directly to a convex method to learn the 2-norm soft margin parameter in support vector machines, solving another important open problem. Finally, the novel approach presented in the paper is supported by positive empirical results.

The rademacher complexity of coregularized kernel classes

In the multi-view approach to semisupervised learning, we choose one predictor from each of multiple hypothesis classes, and we co-regularize our choices by penalizing disagreement among the predictors on the unlabeled data. We examine the co-regularization method used in the co-regularized least squares (CoRLS) algorithm, in which the views are reproducing kernel Hilbert spaces (RKHS's), and the disagreement penalty is the average squared difference in predictions. The final predictor is the pointwise average of the predictors from each view. We call the set of predictors that can result from this procedure the co-regularized hypothesis class. Our main result is a tight bound on the Rademacher complexity of the co-regularized hypothesis class in terms of the kernel matrices of each RKHS. We find that the co-regularization reduces the Rademacher complexity by an amount that depends on the distance between the two views, as measured by a data dependent metric. We then use standard techniques to bound the gap between training error and test error for the CoRLS algorithm. Experimentally, we find that the amount of reduction in complexity introduced by co regularization correlates with the amount of improvement that co-regularization gives in the CoRLS algorithm.

The undirected feedback vertex set problem has a Poly(k) Kernel

Resolving a noted open problem, we show that the Undirected Feedback Vertex Set problem, parameterized by the size of the solution set of vertices, is in the parameterized complexity class Poly(k), that is, polynomial-time pre-processing is sufficient to reduce an initial problem instance (G, k) to a decision-equivalent simplified instance (G', k') where k' � k, and the number of vertices of G' is bounded by a polynomial function of k. Our main result shows an O(k11) kernelization bound.

Effects of cultivar on rodent damage in Australian macadamia orchards

The Black rat (Rattus rattus), a serious pest of Australian macadamia orchards has been estimated to cause up to 30% crop damage in Australian orchards. In recent years an increase in the number of commercially available cultivars has seen a change in orchard characteristics in Australia, primarily effecting fruiting and flowering patterns. This has been suggested to affect the feeding behaviour of rodents and in turn altered the damage process. In this study we compare the extent of damage in orchards containing one of three prevalent cultivars (A4/A16, A268 and HAES 344/741) and investigate the influence of these cultivars, particularly their distinctive fruiting traits, on rodent damage within the orchard. We demonstrate that the temporal pattern and extent of damage differs between cultivar types. Newer Australian macadamia cultivars tested in this study were found to be far more susceptible to rodent damage than the older Hawaiian developed cultivars, most likely due to an extended fruiting period and thinner shells. This has resulted in a more sustained period of crop damage than the patterns of crop damage observed in previous Australian studies. Crop damage caused by R. rattus is significantly higher in orchards that maintain high levels of canopy resources through the fruiting season and we postulate that this is due to the extended fruiting periods of the new cultivars used. The maintenance of canopy resource load in turn corresponds to high crop damage, in this study resulting in crop losses of up to 25%.

The spatial distribution of rodent damage in Australian macadamia (Macadamia integrifolia) orchards

The Black Rat (Rattus rattus), a global pest within the macadamia production industry, causes up to 30% crop damage in Australian orchards. During early stages of production in Australia, research demonstrated the importance of non crop adjacent habitats as significant in affecting the patterns of crop damage seen throughout orchards. Where once rodent damage was limited to the outside edges of orchard blocks, growers are now reporting finding crop damage throughout entire orchards. This study therefore aims to explore the spatial patterns of rodent distribution and damage now occurring in Australian macadamia orchards. We show that rodent damage and rodent distribution in these newer production regions differ from that shown in previous Australian research. Previous Australian research has shown damage patterns which were associated with the edges of orchard blocks however this study demonstrates a more widespread damage distribution. In the current study there is no relationship between rodent damage and the orchard edge. Arboreal rodent nests were identified within these newer orchard systems, suggesting rodents are residing within the tree component of the orchard system and not dependent on adjacent non-crop habitat for shelter. Results from this study confirm that rodents have modified their nesting and foraging behaviour in newer orchards systems in Australia. We suggest that this is a response of increased and prolonged availability of macadamia nuts in newer production regions enabling populations to be maintained throughout the year. Management strategies will require modification if control is to be achieved.

Kernel analysis on Grassmann manifolds for action recognition

Modelling video sequences by subspaces has recently shown promise for recognising human actions. Subspaces are able to accommodate the effects of various image variations and can capture the dynamic properties of actions. Subspaces form a non-Euclidean and curved Riemannian manifold known as a Grassmann manifold. Inference on manifold spaces usually is achieved by embedding the manifolds in higher dimensional Euclidean spaces. In this paper, we instead propose to embed the Grassmann manifolds into reproducing kernel Hilbert spaces and then tackle the problem of discriminant analysis on such manifolds. To achieve efficient machinery, we propose graph-based local discriminant analysis that utilises within-class and between-class similarity graphs to characterise intra-class compactness and inter-class separability, respectively. Experiments on KTH, UCF Sports, and Ballet datasets show that the proposed approach obtains marked improvements in discrimination accuracy in comparison to several state-of-the-art methods, such as the kernel version of affine hull image-set distance, tensor canonical correlation analysis, spatial-temporal words and hierarchy of discriminative space-time neighbourhood features.

Isoenergetic replacement of dietary saturated with monounsaturated fat via macadamia nuts enhances endothelial function in overweight subjects

Background & aims: - Excess adiposity (overweight) is one of numerous risk factors for cardiometabolic disease. Most risk reduction strategies for overweight rely on weight loss through dietary energy restriction. However, since the evidence base for long-term successful weight loss interventions is scant, it is important to identify strategies for risk reduction independent of weight loss. The aim of this study was to compare the effects of isoenergetic substitution of dietary saturated fat (SFA) with monounsaturated fat (MUFA) via macadamia nuts on coronary risk compared to usual diet in overweight adults. Methods: - A randomised controlled trial design, maintaining usual energy intake, but manipulating dietary lipid profile in a group of 64 (54 female, 10 male) overweight (BMI > 25), otherwise healthy, subjects. For the intervention group, energy intakes of usual (baseline) diets were calculated from multiple 3 day diet diaries, and SFA was replaced with MUFA (target: 50%E from fat as MUFA) by altering dietary SFA sources and adding macadamia nuts to the diet. Both control and intervention groups received advice on national guidelines for physical activity and adhered to the same protocol for diet diary record keeping and trial consultations. Anthropometric and clinical measures were taken at baseline and at 10 weeks. Results: A significant increase in brachial artery flow-mediated dilation (p < 0.05) was seen in the monounsaturated diet group at week 10 compared to baseline. This corresponded to significant decreases in waist circumference, total cholesterol (p < 0.05), plasma leptin and ICAM-1 (p < 0.01). Conclusions: - In patient subgroups where adherence to dietary energy-reduction is poor, isoenergetic interventions may improve endothelial function and other coronary risk factors without changes in body weight. This trial was registered with the Australia New Zealand Clinical Trial Registry (ACTRN12607000106437).

A basis for instrusion detection in distributed systems using kernel-level data tainting

This project was a step forward in developing intrusion detection systems in distributed environments such as web services. It investigates a new approach of detection based on so-called "taint-marking" techniques and introduces a theoretical framework along with its implementation in the Linux kernel.

Traversability estimation for a planetary rover via experimental kernel learning in a Gaussian process framework

A critical requirement for safe autonomous navigation of a planetary rover is the ability to accurately estimate the traversability of the terrain. This work considers the problem of predicting the attitude and configuration angles of the platform from terrain representations that are often incomplete due to occlusions and sensor limitations. Using Gaussian Processes (GP) and exteroceptive data as training input, we can provide a continuous and complete representation of terrain traversability, with uncertainty in the output estimates. In this paper, we propose a novel method that focuses on exploiting the explicit correlation in vehicle attitude and configuration during operation by learning a kernel function from vehicle experience to perform GP regression. We provide an extensive experimental validation of the proposed method on a planetary rover. We show significant improvement in the accuracy of our estimation compared with results obtained using standard kernels (Squared Exponential and Neural Network), and compared to traversability estimation made over terrain models built using state-of-the-art GP techniques.

Reliable Fault Diagnosis for Low-Speed Bearings Using Individually Trained Support Vector Machines With Kernel Discriminative Feature Analysis

This paper proposes a highly reliable fault diagnosis approach for low-speed bearings. The proposed approach first extracts wavelet-based fault features that represent diverse symptoms of multiple low-speed bearing defects. The most useful fault features for diagnosis are then selected by utilizing a genetic algorithm (GA)-based kernel discriminative feature analysis cooperating with one-against-all multicategory support vector machines (OAA MCSVMs). Finally, each support vector machine is individually trained with its own feature vector that includes the most discriminative fault features, offering the highest classification performance. In this study, the effectiveness of the proposed GA-based kernel discriminative feature analysis and the classification ability of individually trained OAA MCSVMs are addressed in terms of average classification accuracy. In addition, the proposedGA- based kernel discriminative feature analysis is compared with four other state-of-the-art feature analysis approaches. Experimental results indicate that the proposed approach is superior to other feature analysis methodologies, yielding an average classification accuracy of 98.06% and 94.49% under rotational speeds of 50 revolutions-per-minute (RPM) and 80 RPM, respectively. Furthermore, the individually trained MCSVMs with their own optimal fault features based on the proposed GA-based kernel discriminative feature analysis outperform the standard OAA MCSVMs, showing an average accuracy of 98.66% and 95.01% for bearings under rotational speeds of 50 RPM and 80 RPM, respectively.

Secondary structure element alignment kernel method for prediction of protein structural classes

In this paper, we aim at predicting protein structural classes for low-homology data sets based on predicted secondary structures. We propose a new and simple kernel method, named as SSEAKSVM, to predict protein structural classes. The secondary structures of all protein sequences are obtained by using the tool PSIPRED and then a linear kernel on the basis of secondary structure element alignment scores is constructed for training a support vector machine classifier without parameter adjusting. Our method SSEAKSVM was evaluated on two low-homology datasets 25PDB and 1189 with sequence homology being 25% and 40%, respectively. The jackknife test is used to test and compare our method with other existing methods. The overall accuracies on these two data sets are 86.3% and 84.5%, respectively, which are higher than those obtained by other existing methods. Especially, our method achieves higher accuracies (88.1% and 88.5%) for differentiating the α + β class and the α/β class compared to other methods. This suggests that our method is valuable to predict protein structural classes particularly for low-homology protein sequences. The source code of the method in this paper can be downloaded at http://math.xtu.edu.cn/myphp/math/research/source/SSEAK_source_code.rar.

Mesh-free approximations via the error reproducing kernel method and applications to nonlinear systems developing shocks

Error estimates for the error reproducing kernel method (ERKM) are provided. The ERKM is a mesh-free functional approximation scheme [A. Shaw, D. Roy, A NURBS-based error reproducing kernel method with applications in solid mechanics, Computational Mechanics (2006), to appear (available online)], wherein a targeted function and its derivatives are first approximated via non-uniform rational B-splines (NURBS) basis function. Errors in the NURBS approximation are then reproduced via a family of non-NURBS basis functions, constructed using a polynomial reproduction condition, and added to the NURBS approximation of the function obtained in the first step. In addition to the derivation of error estimates, convergence studies are undertaken for a couple of test boundary value problems with known exact solutions. The ERKM is next applied to a one-dimensional Burgers equation where, time evolution leads to a breakdown of the continuous solution and the appearance of a shock. Many available mesh-free schemes appear to be unable to capture this shock without numerical instability. However, given that any desired order of continuity is achievable through NURBS approximations, the ERKM can even accurately approximate functions with discontinuous derivatives. Moreover, due to the variation diminishing property of NURBS, it has advantages in representing sharp changes in gradients. This paper is focused on demonstrating this ability of ERKM via some numerical examples. Comparisons of some of the results with those via the standard form of the reproducing kernel particle method (RKPM) demonstrate the relative numerical advantages and accuracy of the ERKM.

Assessment of quality defects in macadamia kernels using NIR spectroscopy

Spectral data were collected of intact and ground kernels using 3 instruments (using Si-PbS, Si, and InGaAs detectors), operating over different areas of the spectrum (between 400 and 2500 nm) and employing transmittance, interactance, and reflectance sample presentation strategies. Kernels were assessed on the basis of oil and water content, and with respect to the defect categories of insect damage, rancidity, discoloration, mould growth, germination, and decomposition. Predictive model performance statistics for oil content models were acceptable on all instruments (R2 > 0.98; RMSECV < 2.5%, which is similar to reference analysis error), although that for the instrument employing reflectance optics was inferior to models developed for the instruments employing transmission optics. The spectral positions for calibration coefficients were consistent with absorbance due to the third overtones of CH2 stretching. Calibration models for moisture content in ground samples were acceptable on all instruments (R2 > 0.97; RMSECV < 0.2%), whereas calibration models for intact kernels were relatively poor. Calibration coefficients were more highly weighted around 1360, 740 and 840 nm, consistent with absorbance due to overtones of O-H stretching and combination. Intact kernels with brown centres or rancidity could be discriminated from each other and from sound kernels using principal component analysis. Part kernels affected by insect damage, discoloration, mould growth, germination, and decomposition could be discriminated from sound kernels. However, discrimination among these defect categories was not distinct and could not be validated on an independent set. It is concluded that there is good potential for a low cost Si photodiode array instrument to be employed to identify some quality defects of intact macadamia kernels and to quantify oil and moisture content of kernels in the process laboratory and for oil content in-line. Further work is required to examine the robustness of predictive models across different populations, including growing districts, cultivars and times of harvest.

Timing of fungicide applications for control of husk spot caused by Pseudocercospora macadamiae in macadamia

Pseudocercospora macadamiae is an important pathogen of macadamia in Australia, causing a disease known as husk spot. Growers strive to control the disease with a number of carbendazim and copper treatments. The aim of this study was to consider the macadamia fruit developmental stage at which fungicide application is most effective against husk spot, and whether application of copper-only applications at full-size fruit developmental stage toward the end of the season contributed to effective disease control. Fungicides were applied to macadamia trees at four developmental stages in three orchards in two subsequent production seasons. The effects of the treatments on disease incidence and severity were quantified using area under disease progress curve (AUDPC) and logistic regression models. Although disease incidence varied between cultivars, incidence and severity on cv. A16 showed consistent differences between the treatments. Most significant reduction in husk spot incidence occurred when spraying commenced at match-head sized-fruit developmental stage. All treatments significantly reduced husk spot incidence and severity compared with the untreated controls, and a significant positive linear relationship (R2 = 73%) between AUDPC and severity showed that timing of the first fungicide application is important for effective disease control. Application of fungicide at full-size fruit stage reduced disease incidence but had no impact on premature fruit drop.

Infection, colonisation and sporulation by Pseudocercospora macadamiae on macadamia fruit

The structures and manner with which Pseudocercospora macadamiae penetrates, colonises and proliferates from the pericarp of macadamia fruit was studied using scanning electron microscopy and fluorescence light microscopy. Germ tubes arising from conidia penetrated open stomata within 20 h of inoculation, without observation of specialised infection structures such as appressoria. Colonisation of the pericarp was intercellular, without observation of specialised intracellular infection structures such as haustoria, and was complete from the epidermis to the mesocarp. The fungus proliferated at the epidermis by the formation of conidiophores and conidia on substomatal and protuberant subepidermal stromata. These structures were not observed on the mesocarp surface. The onset of visual husk spot symptoms coincided with an increase in pathogen biomass on the pericarp surface. The progression of symptoms from tan-coloured spots to larger red-brown lesions coincided with the production of conidiophores from substomatal and protuberant subepidermal stromata. The darker the colour of the husk spot lesion, the more frequently protuberant subepidermal stromata were observed. These findings are discussed in the context of observation of other cercosporoid fungi.