Robust formulations for clustering-based large-scale classification

Chebyshev-inequality-based convex relaxations of Chance-Constrained Programs (CCPs) are shown to be useful for learning classifiers on massive datasets. In particular, an algorithm that integrates efficient clustering procedures and CCP approaches for computing classifiers on large datasets is proposed. The key idea is to identify high density regions or clusters from individual class conditional densities and then use a CCP formulation to learn a classifier on the clusters. The CCP formulation ensures that most of the data points in a cluster are correctly classified by employing a Chebyshev-inequality-based convex relaxation. This relaxation is heavily dependent on the second-order statistics. However, this formulation and in general such relaxations that depend on the second-order moments are susceptible to moment estimation errors. One of the contributions of the paper is to propose several formulations that are robust to such errors. In particular a generic way of making such formulations robust to moment estimation errors is illustrated using two novel confidence sets. An important contribution is to show that when either of the confidence sets is employed, for the special case of a spherical normal distribution of clusters, the robust variant of the formulation can be posed as a second-order cone program. Empirical results show that the robust formulations achieve accuracies comparable to that with true moments, even when moment estimates are erroneous. Results also illustrate the benefits of employing the proposed methodology for robust classification of large-scale datasets.

Extension of TSVM to multi-class and hierarchical text classification problems With general losses

Transductive SVM (TSVM) is a well known semi-supervised large margin learning method for binary text classification. In this paper we extend this method to multi-class and hierarchical classification problems. We point out that the determination of labels of unlabeled examples with fixed classifier weights is a linear programming problem. We devise an efficient technique for solving it. The method is applicable to general loss functions. We demonstrate the value of the new method using large margin loss on a number of multi-class and hierarchical classification datasets. For maxent loss we show empirically that our method is better than expectation regularization/constraint and posterior regularization methods, and competitive with the version of entropy regularization method which uses label constraints.

Generative Maximum Entropy Learning for Multiclass Classification

Maximum entropy approach to classification is very well studied in applied statistics and machine learning and almost all the methods that exists in literature are discriminative in nature. In this paper, we introduce a maximum entropy classification method with feature selection for large dimensional data such as text datasets that is generative in nature. To tackle the curse of dimensionality of large data sets, we employ conditional independence assumption (Naive Bayes) and we perform feature selection simultaneously, by enforcing a `maximum discrimination' between estimated class conditional densities. For two class problems, in the proposed method, we use Jeffreys (J) divergence to discriminate the class conditional densities. To extend our method to the multi-class case, we propose a completely new approach by considering a multi-distribution divergence: we replace Jeffreys divergence by Jensen-Shannon (JS) divergence to discriminate conditional densities of multiple classes. In order to reduce computational complexity, we employ a modified Jensen-Shannon divergence (JS(GM)), based on AM-GM inequality. We show that the resulting divergence is a natural generalization of Jeffreys divergence to a multiple distributions case. As far as the theoretical justifications are concerned we show that when one intends to select the best features in a generative maximum entropy approach, maximum discrimination using J-divergence emerges naturally in binary classification. Performance and comparative study of the proposed algorithms have been demonstrated on large dimensional text and gene expression datasets that show our methods scale up very well with large dimensional datasets.

Damage Detection Sensitivity, Specificity and Classification Data Analysis for SHM Systems Design, Verification and Validation

In this paper, we consider applying derived knowledge base regarding the sensitivity and specificity of damage(s) to be detected by an SHM system being designed and qualified. These efforts are necessary toward developing capabilities in SHM system to classify reliably various probable damages through sequence of monitoring, i.e., damage precursor identification, detection of damage and monitoring its progression. We consider the particular problem of visual and ultrasonic NDE based SHM system design requirements, where the damage detection sensitivity and specificity data definitions for a class of structural components are established. Methodologies for SHM system specification creation are discussed in details. Examples are shown to illustrate how the physics of damage detection scheme limits particular damage detection sensitivity and specificity and further how these information can be used in algorithms to combine various different NDE schemes in an SHM system to enhance efficiency and effectiveness. Statistical and data driven models to determine the sensitivity and probability of damage detection (POD) has been demonstrated for plate with varying one-sided line crack using optical and ultrasonic based inspection techniques.

Advantages and Versatility of Fluorescence-Based Methodology to Characterize the Functionality of LDLR and Class Mutation Assignment

Familial hypercholesterolemia (FH) is a common autosomal codominant disease with a frequency of 1:500 individuals in its heterozygous form. The genetic basis of FH is most commonly mutations within the LDLR gene. Assessing the pathogenicity of LDLR variants is particularly important to give a patient a definitive diagnosis of FH. Current studies of LDLR activity ex vivo are based on the analysis of I-125-labeled lipoproteins (reference method) or fluorescent-labelled LDL. The main purpose of this study was to compare the effectiveness of these two methods to assess LDLR functionality in order to validate a functional assay to analyse LDLR mutations. LDLR activity of different variants has been studied by flow cytometry using FITC-labelled LDL and compared with studies performed previously with I-125-labeled lipoproteins. Flow cytometry results are in full agreement with the data obtained by the I-125 methodology. Additionally confocal microscopy allowed the assignment of different class mutation to the variants assayed. Use of fluorescence yielded similar results than I-125-labeled lipoproteins concerning LDLR activity determination, and also allows class mutation classification. The use of FITC-labelled LDL is easier in handling and disposal, cheaper than radioactivity and can be routinely performed by any group doing LDLR functional validations.

Some Extensions of the K-Means Algorithm for Image Segmentation and Pattern Classification

In this paper we present some extensions to the k-means algorithm for vector quantization that permit its efficient use in image segmentation and pattern classification tasks. It is shown that by introducing state variables that correspond to certain statistics of the dynamic behavior of the algorithm, it is possible to find the representative centers fo the lower dimensional maniforlds that define the boundaries between classes, for clouds of multi-dimensional, mult-class data; this permits one, for example, to find class boundaries directly from sparse data (e.g., in image segmentation tasks) or to efficiently place centers for pattern classification (e.g., with local Gaussian classifiers). The same state variables can be used to define algorithms for determining adaptively the optimal number of centers for clouds of data with space-varying density. Some examples of the applicatin of these extensions are also given.

Newly resolved relationships in an early land plant lineage: Bryophyta class Sphagnopsida (peat mosses).

UNLABELLED: PREMISE OF THE STUDY: The Sphagnopsida, an early-diverging lineage of mosses (phylum Bryophyta), are morphologically and ecologically unique and have profound impacts on global climate. The Sphagnopsida are currently classified in two genera, Sphagnum (peat mosses) with some 350-500 species and Ambuchanania with one species. An analysis of phylogenetic relationships among species and genera in the Sphagnopsida were conducted to resolve major lineages and relationships among species within the Sphagnopsida. • METHODS: Phylogenetic analyses of nucleotide sequences from the nuclear, plastid, and mitochondrial genomes (11 704 nucleotides total) were conducted and analyzed using maximum likelihood and Bayesian inference employing seven different substitution models of varying complexity. • KEY RESULTS: Phylogenetic analyses resolved three lineages within the Sphagnopsida: (1) Sphagnum sericeum, (2) S. inretortum plus Ambuchanania leucobryoides, and (3) all remaining species of Sphagnum. Sister group relationships among these three clades could not be resolved, but the phylogenetic results indicate that the highly divergent morphology of A. leucobryoides is derived within the Sphagnopsida rather than plesiomorphic. A new classification is proposed for class Sphagnopsida, with one order (Sphagnales), three families, and four genera. • CONCLUSIONS: The Sphagnopsida are an old lineage within the phylum Bryophyta, but the extant species of Sphagnum represent a relatively recent radiation. It is likely that additional species critical to understanding the evolution of peat mosses await discovery, especially in the southern hemisphere.

An exploration of the experience of implementing the second step violence prevention curriculum in one primary general learning disabilities class /

Violence has always been a part of the human experience, and therefore, a popular topic for research. It is a controversial issue, mostly because the possible sources of violent behaviour are so varied, encompassing both biological and environmental factors. However, very little disagreement is found regarding the severity of this societal problem. Most researchers agree that the number and intensity of aggressive acts among adults and children is growing. Not surprisingly, many educational policies, programs, and curricula have been developed to address this concern. The research favours programs which address the root causes of violence and seek to prevent rather than provide consequences for the undesirable behaviour. But what makes a violence prevention program effective? How should educators choose among the many curricula on the market? After reviewing the literature surrounding violence prevention programs and their effectiveness, The Second Step Violence Prevention Curriculum surfaced as unique in many ways. It was designed to address the root causes of violence in an active, student-centred way. Empathy training, anger management, interpersonal cognitive problem solving, and behavioural social skills form the basis of this program. Published in 1992, the program has been the topic of limited research, almost entirely carried out using quantitative methodologies.The purpose of this study was to understand what happens when the Second Step Violence Prevention Curriculum is implemented with a group of students and teachers. I was not seeking a statistical correlation between the frequency of violence and program delivery, as in most prior research. Rather, I wished to gain a deeper understanding of the impact ofthe program through the eyes of the participants. The Second Step Program was taught to a small, primary level, general learning disabilities class by a teacher and student teacher. Data were gathered using interviews with the teachers, personal observations, staff reports, and my own journal. Common themes across the four types of data collection emerged during the study, and these themes were isolated and explored for meaning. Findings indicate that the program does not offer a "quick fix" to this serious problem. However, several important discoveries were made. The teachers feU that the program was effective despite a lack of concrete evidence to support this claim. They used the Second Step strategies outside their actual instructional time and felt it made them better educators and disciplinarians. The students did not display a marked change in their behaviour during or after the program implementation, but they were better able to speak about their actions, the source of their aggression, and the alternatives which were available. Although they were not yet transferring their knowledge into positive action,a heightened awareness was evident. Finally, staff reports and my own journal led me to a deeper understanding ofhow perception frames reality. The perception that the program was working led everyone to feel more empowered when a violent incident occurred, and efforts were made to address the cause rather than merely to offer consequences. A general feeling that we were addressing the problem in a productive way was prevalent among the staff and students involved. The findings from this investigation have many implications for research and practice. Further study into the realm of violence prevention is greatly needed, using a balance of quantitative and qualitative methodologies. Such a serious problem can only be effectively addressed with a greater understanding of its complexities. This study also demonstrates the overall positive impact of the Second Step Violence Prevention Curriculum and, therefore, supports its continued use in our schools.

Classification analytique de systèmes différentiels linéaires déployant une singularité irrégulière de rang de Poincaré 1

Cette thèse traite de la classification analytique du déploiement de systèmes différentiels linéaires ayant une singularité irrégulière. Elle est composée de deux articles sur le sujet: le premier présente des résultats obtenus lors de l'étude de la confluence de l'équation hypergéométrique et peut être considéré comme un cas particulier du second; le deuxième contient les théorèmes et résultats principaux. Dans les deux articles, nous considérons la confluence de deux points singuliers réguliers en un point singulier irrégulier et nous étudions les conséquences de la divergence des solutions au point singulier irrégulier sur le comportement des solutions du système déployé. Pour ce faire, nous recouvrons un voisinage de l'origine (de manière ramifiée) dans l'espace du paramètre de déploiement $\epsilon$. La monodromie d'une base de solutions bien choisie est directement reliée aux matrices de Stokes déployées. Ces dernières donnent une interprétation géométrique aux matrices de Stokes, incluant le lien (existant au moins pour les cas génériques) entre la divergence des solutions à $\epsilon=0$ et la présence de solutions logarithmiques autour des points singuliers réguliers lors de la résonance. La monodromie d'intégrales premières de systèmes de Riccati correspondants est aussi interprétée en fonction des éléments des matrices de Stokes déployées. De plus, dans le second article, nous donnons le système complet d'invariants analytiques pour le déploiement de systèmes différentiels linéaires $x^2y'=A(x)y$ ayant une singularité irrégulière de rang de Poincaré $1$ à l'origine au-dessus d'un voisinage fixé $\mathbb{D}_r$ dans la variable $x$. Ce système est constitué d'une partie formelle, donnée par des polynômes, et d'une partie analytique, donnée par une classe d'équivalence de matrices de Stokes déployées. Pour chaque valeur du paramètre $\epsilon$ dans un secteur pointé à l'origine d'ouverture plus grande que $2\pi$, nous recouvrons l'espace de la variable, $\mathbb{D}_r$, avec deux secteurs et, au-dessus de chacun, nous choisissons une base de solutions du système déployé. Cette base sert à définir les matrices de Stokes déployées. Finalement, nous prouvons un théorème de réalisation des invariants qui satisfont une condition nécessaire et suffisante, identifiant ainsi l'ensemble des modules.

L’utilisation de la polarimétrie radar et de la décomposition de Touzi pour la caractérisation et la classification des physionomies végétales des milieux humides : le cas du Lac Saint-Pierre.

Les milieux humides remplissent plusieurs fonctions écologiques d’importance et contribuent à la biodiversité de la faune et de la flore. Même s’il existe une reconnaissance croissante sur l’importante de protéger ces milieux, il n’en demeure pas moins que leur intégrité est encore menacée par la pression des activités humaines. L’inventaire et le suivi systématique des milieux humides constituent une nécessité et la télédétection est le seul moyen réaliste d’atteindre ce but. L’objectif de cette thèse consiste à contribuer et à améliorer la caractérisation des milieux humides en utilisant des données satellites acquises par des radars polarimétriques en bande L (ALOS-PALSAR) et C (RADARSAT-2). Cette thèse se fonde sur deux hypothèses (chap. 1). La première hypothèse stipule que les classes de physionomies végétales, basées sur la structure des végétaux, sont plus appropriées que les classes d’espèces végétales car mieux adaptées au contenu informationnel des images radar polarimétriques. La seconde hypothèse stipule que les algorithmes de décompositions polarimétriques permettent une extraction optimale de l’information polarimétrique comparativement à une approche multipolarisée basée sur les canaux de polarisation HH, HV et VV (chap. 3). En particulier, l’apport de la décomposition incohérente de Touzi pour l’inventaire et le suivi de milieux humides est examiné en détail. Cette décomposition permet de caractériser le type de diffusion, la phase, l’orientation, la symétrie, le degré de polarisation et la puissance rétrodiffusée d’une cible à l’aide d’une série de paramètres extraits d’une analyse des vecteurs et des valeurs propres de la matrice de cohérence. La région du lac Saint-Pierre a été sélectionnée comme site d’étude étant donné la grande diversité de ses milieux humides qui y couvrent plus de 20 000 ha. L’un des défis posés par cette thèse consiste au fait qu’il n’existe pas de système standard énumérant l’ensemble possible des classes physionomiques ni d’indications précises quant à leurs caractéristiques et dimensions. Une grande attention a donc été portée à la création de ces classes par recoupement de sources de données diverses et plus de 50 espèces végétales ont été regroupées en 9 classes physionomiques (chap. 7, 8 et 9). Plusieurs analyses sont proposées pour valider les hypothèses de cette thèse (chap. 9). Des analyses de sensibilité par diffusiogramme sont utilisées pour étudier les caractéristiques et la dispersion des physionomies végétales dans différents espaces constitués de paramètres polarimétriques ou canaux de polarisation (chap. 10 et 12). Des séries temporelles d’images RADARSAT-2 sont utilisées pour approfondir la compréhension de l’évolution saisonnière des physionomies végétales (chap. 12). L’algorithme de la divergence transformée est utilisé pour quantifier la séparabilité entre les classes physionomiques et pour identifier le ou les paramètres ayant le plus contribué(s) à leur séparabilité (chap. 11 et 13). Des classifications sont aussi proposées et les résultats comparés à une carte existante des milieux humide du lac Saint-Pierre (14). Finalement, une analyse du potentiel des paramètres polarimétrique en bande C et L est proposé pour le suivi de l’hydrologie des tourbières (chap. 15 et 16). Les analyses de sensibilité montrent que les paramètres de la 1re composante, relatifs à la portion dominante (polarisée) du signal, sont suffisants pour une caractérisation générale des physionomies végétales. Les paramètres des 2e et 3e composantes sont cependant nécessaires pour obtenir de meilleures séparabilités entre les classes (chap. 11 et 13) et une meilleure discrimination entre milieux humides et milieux secs (chap. 14). Cette thèse montre qu’il est préférable de considérer individuellement les paramètres des 1re, 2e et 3e composantes plutôt que leur somme pondérée par leurs valeurs propres respectives (chap. 10 et 12). Cette thèse examine également la complémentarité entre les paramètres de structure et ceux relatifs à la puissance rétrodiffusée, souvent ignorée et normalisée par la plupart des décompositions polarimétriques. La dimension temporelle (saisonnière) est essentielle pour la caractérisation et la classification des physionomies végétales (chap. 12, 13 et 14). Des images acquises au printemps (avril et mai) sont nécessaires pour discriminer les milieux secs des milieux humides alors que des images acquises en été (juillet et août) sont nécessaires pour raffiner la classification des physionomies végétales. Un arbre hiérarchique de classification développé dans cette thèse constitue une synthèse des connaissances acquises (chap. 14). À l’aide d’un nombre relativement réduit de paramètres polarimétriques et de règles de décisions simples, il est possible d’identifier, entre autres, trois classes de bas marais et de discriminer avec succès les hauts marais herbacés des autres classes physionomiques sans avoir recours à des sources de données auxiliaires. Les résultats obtenus sont comparables à ceux provenant d’une classification supervisée utilisant deux images Landsat-5 avec une exactitude globale de 77.3% et 79.0% respectivement. Diverses classifications utilisant la machine à vecteurs de support (SVM) permettent de reproduire les résultats obtenus avec l’arbre hiérarchique de classification. L’exploitation d’une plus forte dimensionalitée par le SVM, avec une précision globale maximale de 79.1%, ne permet cependant pas d’obtenir des résultats significativement meilleurs. Finalement, la phase de la décomposition de Touzi apparaît être le seul paramètre (en bande L) sensible aux variations du niveau d’eau sous la surface des tourbières ouvertes (chap. 16). Ce paramètre offre donc un grand potentiel pour le suivi de l’hydrologie des tourbières comparativement à la différence de phase entre les canaux HH et VV. Cette thèse démontre que les paramètres de la décomposition de Touzi permettent une meilleure caractérisation, de meilleures séparabilités et de meilleures classifications des physionomies végétales des milieux humides que les canaux de polarisation HH, HV et VV. Le regroupement des espèces végétales en classes physionomiques est un concept valable. Mais certaines espèces végétales partageant une physionomie similaire, mais occupant un milieu différent (haut vs bas marais), ont cependant présenté des différences significatives quant aux propriétés de leur rétrodiffusion.

Non invasive classification system of scoliosis curve types using least-squares support vector machines

Objective To determine scoliosis curve types using non invasive surface acquisition, without prior knowledge from X-ray data. Methods Classification of scoliosis deformities according to curve type is used in the clinical management of scoliotic patients. In this work, we propose a robust system that can determine the scoliosis curve type from non invasive acquisition of the 3D back surface of the patients. The 3D image of the surface of the trunk is divided into patches and local geometric descriptors characterizing the back surface are computed from each patch and constitute the features. We reduce the dimensionality by using principal component analysis and retain 53 components using an overlap criterion combined with the total variance in the observed variables. In this work, a multi-class classifier is built with least-squares support vector machines (LS-SVM). The original LS-SVM formulation was modified by weighting the positive and negative samples differently and a new kernel was designed in order to achieve a robust classifier. The proposed system is validated using data from 165 patients with different scoliosis curve types. The results of our non invasive classification were compared with those obtained by an expert using X-ray images. Results The average rate of successful classification was computed using a leave-one-out cross-validation procedure. The overall accuracy of the system was 95%. As for the correct classification rates per class, we obtained 96%, 84% and 97% for the thoracic, double major and lumbar/thoracolumbar curve types, respectively. Conclusion This study shows that it is possible to find a relationship between the internal deformity and the back surface deformity in scoliosis with machine learning methods. The proposed system uses non invasive surface acquisition, which is safe for the patient as it involves no radiation. Also, the design of a specific kernel improved classification performance.

Improving Multi-class Text Classification with Naive Bayes

There are numerous text documents available in electronic form. More and more are becoming available every day. Such documents represent a massive amount of information that is easily accessible. Seeking value in this huge collection requires organization; much of the work of organizing documents can be automated through text classification. The accuracy and our understanding of such systems greatly influences their usefulness. In this paper, we seek 1) to advance the understanding of commonly used text classification techniques, and 2) through that understanding, improve the tools that are available for text classification. We begin by clarifying the assumptions made in the derivation of Naive Bayes, noting basic properties and proposing ways for its extension and improvement. Next, we investigate the quality of Naive Bayes parameter estimates and their impact on classification. Our analysis leads to a theorem which gives an explanation for the improvements that can be found in multiclass classification with Naive Bayes using Error-Correcting Output Codes. We use experimental evidence on two commonly-used data sets to exhibit an application of the theorem. Finally, we show fundamental flaws in a commonly-used feature selection algorithm and develop a statistics-based framework for text feature selection. Greater understanding of Naive Bayes and the properties of text allows us to make better use of it in text classification.

Classification of textures seen from different distances and under varying illumination direction

Changes in the angle of illumination incident upon a 3D surface texture can significantly alter its appearance, implying variations in the image texture. These texture variations produce displacements of class members in the feature space, increasing the failure rates of texture classifiers. To avoid this problem, a model-based texture recognition system which classifies textures seen from different distances and under different illumination directions is presented in this paper. The system works on the basis of a surface model obtained by means of 4-source colour photometric stereo, used to generate 2D image textures under different illumination directions. The recognition system combines coocurrence matrices for feature extraction with a Nearest Neighbour classifier. Moreover, the recognition allows one to guess the approximate direction of the illumination used to capture the test image

Kernel classifier construction using orthogonal forward selection and boosting with Fisher ratio class separability measure

A greedy technique is proposed to construct parsimonious kernel classifiers using the orthogonal forward selection method and boosting based on Fisher ratio for class separability measure. Unlike most kernel classification methods, which restrict kernel means to the training input data and use a fixed common variance for all the kernel terms, the proposed technique can tune both the mean vector and diagonal covariance matrix of individual kernel by incrementally maximizing Fisher ratio for class separability measure. An efficient weighted optimization method is developed based on boosting to append kernels one by one in an orthogonal forward selection procedure. Experimental results obtained using this construction technique demonstrate that it offers a viable alternative to the existing state-of-the-art kernel modeling methods for constructing sparse Gaussian radial basis function network classifiers. that generalize well.