On classical and other methods of discriminant analysis and estimation of log-odds

For two multinormal populations with equal covariance matrices the likelihood ratio discriminant function, an alternative allocation rule to the sample linear discriminant function when n1 ≠ n2 ,is studied analytically. With the assumption of a known covariance matrix its distribution is derived and the expectation of its actual and apparent error rates evaluated and compared with those of the sample linear discriminant function. This comparison indicates that the likelihood ratio allocation rule is robust to unequal sample sizes. The quadratic discriminant function is studied, its distribution reviewed and evaluation of its probabilities of misclassification discussed. For known covariance matrices the distribution of the sample quadratic discriminant function is derived. When the known covariance matrices are proportional exact expressions for the expectation of its actual and apparent error rates are obtained and evaluated. The effectiveness of the sample linear discriminant function for this case is also considered. Estimation of true log-odds for two multinormal populations with equal or unequal covariance matrices is studied. The estimative, Bayesian predictive and a kernel method are compared by evaluating their biases and mean square errors. Some algebraic expressions for these quantities are derived. With equal covariance matrices the predictive method is preferable. Where it derives this superiority is investigated by considering its performance for various levels of fixed true log-odds. It is also shown that the predictive method is sensitive to n1 ≠ n2. For unequal but proportional covariance matrices the unbiased estimative method is preferred. Product Normal kernel density estimates are used to give a kernel estimator of true log-odds. The effect of correlation in the variables with product kernels is considered. With equal covariance matrices the kernel and parametric estimators are compared by simulation. For moderately correlated variables and large dimension sizes the product kernel method is a good estimator of true log-odds.

Communications inspired linear discriminant analysis

We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label. By harnessing a recent theoretical result on the gradient of mutual information, the above optimization problem can be solved directly using gradient descent, without requiring simplification of the objective function. Theoretical analysis and empirical comparison are made between the proposed method and two closely related methods, and comparisons are also made with a method in which Rényi entropy is used to define the mutual information (in this case the gradient may be computed simply, under a special parameter setting). Relative to these alternative approaches, the proposed method achieves promising results on real datasets. Copyright 2012 by the author(s)/owner(s).

Geostatistical Fisher discriminant analysis

A geostatistical version of the classical Fisher rule (linear discriminant analysis) is presented.This method is applicable when a large dataset of multivariate observations is available within a domain split in several known subdomains, and it assumes that the variograms (or covariance functions) are comparable between subdomains, which only differ in the mean values of the available variables. The method consists on finding the eigen-decomposition of the matrix W-1B, where W is the matrix of sills of all direct- and cross-variograms, and B is the covariance matrix of the vectors of weighted means within each subdomain, obtained by generalized least squares. The method is used to map peat blanket occurrence in Northern Ireland, with data from the Tellus
survey, which requires a minimal change to the general recipe: to use compositionally-compliant variogram tools and models, and work with log-ratio transformed data.

Brief report : The defense mechanisms of homophobic adolescent males : a descriptive discriminant analysis

The study examined the role of defense mechanisms in homophobic attitudes of older male adolescents aged 17e18 years. A cross-sectional survey collected data from final year high school students (N ¼ 86) attending an all male school in a regional centre in Victoria, Australia. The school was identified by teachers as having a problematic culture of homophobic intolerance. Participants were divided into homophobic and non-homophobic groups based on their scores on the Homophobia Scale Questionnaire. Discriminant analysis was conducted to identify the predictors that would best categorise students into those two groups on the basis of defense styles derived from the Defense Style Questionnaire-40 (DSQ-40). The strongest predictors of homophobia amongst defense styles were idealisation, denial, somatisation and devaluation accounting for 18.31%, 17.64%, 13.10% and 11.35% of the variance, respectively. Results generally supported the larger contribution of more immature defenses to higher levels of homophobia.

Boosting performance for 2D linear discriminant analysis via regression

Two Dimensional Linear Discriminant Analysis (2DLDA) has received much interest in recent years. However, 2DLDA could make pairwise distances between any two classes become significantly unbalanced, which may affect its performance. Moreover 2DLDA could also suffer from the small sample size problem. Based on these observations, we propose two novel algorithms called Regularized 2DLDA and Ridge Regression for 2DLDA (RR-2DLDA). Regularized 2DLDA is an extension of 2DLDA with the introduction of a regularization parameter to deal with the small sample size problem. RR-2DLDA integrates ridge regression into Regularized 2DLDA to balance the distances among different classes after the transformation. These proposed algorithms overcome the limitations of 2DLDA and boost recognition accuracy. The experimental results on the Yale, PIE and FERET databases showed that RR-2DLDA is superior not only to 2DLDA but also other state-of-the-art algorithms.

DDoS discrimination by linear discriminant analysis (LDA)

In this paper, we propose an effective approach with a supervised learning system based on Linear Discriminant Analysis (LDA) to discriminate legitimate traffic from DDoS attack traffic. Currently there is a wide outbreak of DDoS attacks that remain risky for the entire Internet. Different attack methods and strategies are trying to challenge defence systems. Among the behaviours of attack sources, repeatable and predictable features differ from source of legitimate traffic. In addition, the DDoS defence systems lack the learning ability to fine-tune their accuracy. This paper analyses real trace traffic from publicly available datasets. Pearson's correlation coefficient and Shannon's entropy are deployed for extracting dependency and predictability of traffic data respectively. Then, LDA is used to train and classify legitimate and attack traffic flows. From the results of our experiment, we can confirm that the proposed discrimination system can differentiate DDoS attacks from legitimate traffic with a high rate of accuracy.