On the influence of imputation in classification: practical issues

The substitution of missing values, also called imputation, is an important data preparation task for many domains. Ideally, the substitution of missing values should not insert biases into the dataset. This aspect has been usually assessed by some measures of the prediction capability of imputation methods. Such measures assume the simulation of missing entries for some attributes whose values are actually known. These artificially missing values are imputed and then compared with the original values. Although this evaluation is useful, it does not allow the influence of imputed values in the ultimate modelling task (e.g. in classification) to be inferred. We argue that imputation cannot be properly evaluated apart from the modelling task. Thus, alternative approaches are needed. This article elaborates on the influence of imputed values in classification. In particular, a practical procedure for estimating the inserted bias is described. As an additional contribution, we have used such a procedure to empirically illustrate the performance of three imputation methods (majority, naive Bayes and Bayesian networks) in three datasets. Three classifiers (decision tree, naive Bayes and nearest neighbours) have been used as modelling tools in our experiments. The achieved results illustrate a variety of situations that can take place in the data preparation practice.

Evolutionary tuning of SVM parameter values in multiclass problems

Support vector machines (SVMs) were originally formulated for the solution of binary classification problems. In multiclass problems, a decomposition approach is often employed, in which the multiclass problem is divided into multiple binary subproblems, whose results are combined. Generally, the performance of SVM classifiers is affected by the selection of values for their parameters. This paper investigates the use of genetic algorithms (GAs) to tune the parameters of the binary SVMs in common multiclass decompositions. The developed GA may search for a set of parameter values common to all binary classifiers or for differentiated values for each binary classifier. (C) 2008 Elsevier B.V. All rights reserved.

HIERARCHICAL DECOMPOSITION OF MULTICLASS PROBLEMS

Several popular Machine Learning techniques are originally designed for the solution of two-class problems. However, several classification problems have more than two classes. One approach to deal with multiclass problems using binary classifiers is to decompose the multiclass problem into multiple binary sub-problems disposed in a binary tree. This approach requires a binary partition of the classes for each node of the tree, which defines the tree structure. This paper presents two algorithms to determine the tree structure taking into account information collected from the used dataset. This approach allows the tree structure to be determined automatically for any multiclass dataset.

A hybrid approach to learn with imbalanced classes using evolutionary algorithms

There is an increasing interest in the application of Evolutionary Algorithms (EAs) to induce classification rules. This hybrid approach can benefit areas where classical methods for rule induction have not been very successful. One example is the induction of classification rules in imbalanced domains. Imbalanced data occur when one or more classes heavily outnumber other classes. Frequently, classical machine learning (ML) classifiers are not able to learn in the presence of imbalanced data sets, inducing classification models that always predict the most numerous classes. In this work, we propose a novel hybrid approach to deal with this problem. We create several balanced data sets with all minority class cases and a random sample of majority class cases. These balanced data sets are fed to classical ML systems that produce rule sets. The rule sets are combined creating a pool of rules and an EA is used to build a classifier from this pool of rules. This hybrid approach has some advantages over undersampling, since it reduces the amount of discarded information, and some advantages over oversampling, since it avoids overfitting. The proposed approach was experimentally analysed and the experimental results show an improvement in the classification performance measured as the area under the receiver operating characteristics (ROC) curve.

Poly-bagging predictors for classification modelling for credit scoring

Credit scoring modelling comprises one of the leading formal tools for supporting the granting of credit. Its core objective consists of the generation of a score by means of which potential clients can be listed in the order of the probability of default. A critical factor is whether a credit scoring model is accurate enough in order to provide correct classification of the client as a good or bad payer. In this context the concept of bootstraping aggregating (bagging) arises. The basic idea is to generate multiple classifiers by obtaining the predicted values from the fitted models to several replicated datasets and then combining them into a single predictive classification in order to improve the classification accuracy. In this paper we propose a new bagging-type variant procedure, which we call poly-bagging, consisting of combining predictors over a succession of resamplings. The study is derived by credit scoring modelling. The proposed poly-bagging procedure was applied to some different artificial datasets and to a real granting of credit dataset up to three successions of resamplings. We observed better classification accuracy for the two-bagged and the three-bagged models for all considered setups. These results lead to a strong indication that the poly-bagging approach may promote improvement on the modelling performance measures, while keeping a flexible and straightforward bagging-type structure easy to implement. (C) 2011 Elsevier Ltd. All rights reserved.

A Road Following Approach Using Artificial Neural Networks Combinations

Navigation is a broad topic that has been receiving considerable attention from the mobile robotic community over the years. In order to execute autonomous driving in outdoor urban environments it is necessary to identify parts of the terrain that can be traversed and parts that should be avoided. This paper describes an analyses of terrain identification based on different visual information using a MLP artificial neural network and combining responses of many classifiers. Experimental tests using a vehicle and a video camera have been conducted in real scenarios to evaluate the proposed approach.

Photoassociative ionization of cold Na atoms: repulsive levels effects on the ion production rate

We have revisited photoassociative ionization (PAI) in a cold sample of Na atmos. A two-color experiment was performed ina magneto-optical trap through the addition of aprobe laser. The observation of a marked change in the PAI rate for a definite frequency range can be attributed to the influence of repuisive levels and a possible avoided crossing between long-range molecular levels. (c) 2009 by Astro Ltd. Published exclusively by WLLEY-VCH Verlag GmbH & Co. KGaA

Estimating complex cortical networks via surface recordings-A critical note

We discuss potential caveats when estimating topologies of 3D brain networks from surface recordings. It is virtually impossible to record activity from all single neurons in the brain and one has to rely on techniques that measure average activity at sparsely located (non-invasive) recording sites Effects of this spatial sampling in relation to structural network measures like centrality and assortativity were analyzed using multivariate classifiers A simplified model of 3D brain connectivity incorporating both short- and long-range connections served for testing. To mimic M/EEG recordings we sampled this model via non-overlapping regions and weighted nodes and connections according to their proximity to the recording sites We used various complex network models for reference and tried to classify sampled versions of the ""brain-like"" network as one of these archetypes It was found that sampled networks may substantially deviate in topology from the respective original networks for small sample sizes For experimental studies this may imply that surface recordings can yield network structures that might not agree with its generating 3D network. (C) 2010 Elsevier Inc All rights reserved

A novel MAP-MRF approach for multispectral image contextual classification using combination of suboptimal iterative algorithms

In this paper we present a novel approach for multispectral image contextual classification by combining iterative combinatorial optimization algorithms. The pixel-wise decision rule is defined using a Bayesian approach to combine two MRF models: a Gaussian Markov Random Field (GMRF) for the observations (likelihood) and a Potts model for the a priori knowledge, to regularize the solution in the presence of noisy data. Hence, the classification problem is stated according to a Maximum a Posteriori (MAP) framework. In order to approximate the MAP solution we apply several combinatorial optimization methods using multiple simultaneous initializations, making the solution less sensitive to the initial conditions and reducing both computational cost and time in comparison to Simulated Annealing, often unfeasible in many real image processing applications. Markov Random Field model parameters are estimated by Maximum Pseudo-Likelihood (MPL) approach, avoiding manual adjustments in the choice of the regularization parameters. Asymptotic evaluations assess the accuracy of the proposed parameter estimation procedure. To test and evaluate the proposed classification method, we adopt metrics for quantitative performance assessment (Cohen`s Kappa coefficient), allowing a robust and accurate statistical analysis. The obtained results clearly show that combining sub-optimal contextual algorithms significantly improves the classification performance, indicating the effectiveness of the proposed methodology. (C) 2010 Elsevier B.V. All rights reserved.

Involutions of RA Loops

Let L be an RA loop, that is, a loop whose loop ring over any coefficient ring R is an alternative, but not associative, ring. Let l bar right arrow l(theta) denote an involution on L and extend it linearly to the loop ring RL. An element alpha is an element of RL is symmetric if alpha(theta) = alpha and skew-symmetric if alpha(theta) = -alpha. In this paper, we show that there exists an involution making the symmetric elements of RL commute if and only if the characteristic of R is 2 or theta is the canonical involution on L, and an involution making the skew-symmetric elements of RL commute if and only if the characteristic of R is 2 or 4.

COMMUTATIVE FINITE-DIMENSIONAL ALGEBRAS SATISFYING x(x(xy))=0 ARE NILPOTENT

We investigate the structure of commutative non-associative algebras satisfying the identity x(x(xy)) = 0. Recently, Correa and Hentzel proved that every commutative algebra satisfying above identity over a field of characteristic not equal 2 is solvable. We prove that every commutative finite-dimensional algebra u over a field F of characteristic not equal 2, 3 which satisfies the identity x(x(xy)) = 0 is nilpotent. Furthermore, we obtain new identities and properties for this class of algebras.

Polynomial identities for the ternary cyclic sum

We simplify the results of Bremner and Hentzel [J. Algebra 231 (2000) 387-405] on polynomial identities of degree 9 in two variables satisfied by the ternary cyclic sum [a, b, c] abc + bca + cab in every totally associative ternary algebra. We also obtain new identities of degree 9 in three variables which do not follow from the identities in two variables. Our results depend on (i) the LLL algorithm for lattice basis reduction, and (ii) linearization operators in the group algebra of the symmetric group which permit efficient computation of the representation matrices for a non-linear identity. Our computational methods can be applied to polynomial identities for other algebraic structures.

Examples of Self-Iterating Lie Algebras, 2

We study properties of self-iterating Lie algebras in positive characteristic. Let R = K[t(i)vertical bar i is an element of N]/(t(i)(p)vertical bar i is an element of N) be the truncated polynomial ring. Let partial derivative(i) = partial derivative/partial derivative t(i), i is an element of N, denote the respective derivations. Consider the operators v(1) = partial derivative(1) + t(0)(partial derivative(2) + t(1)(partial derivative(3) + t(2)(partial derivative(4) + t(3)(partial derivative(5) + t(4)(partial derivative(6) + ...))))); v(2) = partial derivative(2) + t(1)(partial derivative(3) + t(2)(partial derivative(4) + t(3)(partial derivative(5) + t(4)(partial derivative(6) + ...)))). Let L = Lie(p)(v(1), v(2)) subset of Der R be the restricted Lie algebra generated by these derivations. We establish the following properties of this algebra in case p = 2, 3. a) L has a polynomial growth with Gelfand-Kirillov dimension lnp/ln((1+root 5)/2). b) the associative envelope A = Alg(v(1), v(2)) of L has Gelfand-Kirillov dimension 2 lnp/ln((1+root 5)/2). c) L has a nil-p-mapping. d) L, A and the augmentation ideal of the restricted enveloping algebra u = u(0)(L) are direct sums of two locally nilpotent subalgebras. The question whether u is a nil-algebra remains open. e) the restricted enveloping algebra u(L) is of intermediate growth. These properties resemble those of Grigorchuk and Gupta-Sidki groups.

Crossed products by twisted partial actions and graded algebras

For a twisted partial action e of a group G on an (associative non-necessarily unital) algebra A over a commutative unital ring k, the crossed product A x(Theta) G is proved to be associative. Given a G-graded k-algebra B = circle plus(g is an element of G) B-g with the mild restriction of homogeneous non-degeneracy, a criteria is established for B to be isomorphic to the crossed product B-1 x(Theta) G for some twisted partial action of G on B-1. The equality BgBg-1 B-g = B-g (for all g is an element of G) is one of the ingredients of the criteria, and if it holds and, moreover, B has enough local units, then it is shown that B is stably isomorphic to a crossed product by a twisted partial action of G. (c) 2008 Elsevier Inc. All rights reserved.

Torsion theories induced from commutative subalgebras

We begin a study of torsion theories for representations of finitely generated algebras U over a field containing a finitely generated commutative Harish-Chandra subalgebra Gamma. This is an important class of associative algebras, which includes all finite W-algebras of type A over an algebraically closed field of characteristic zero, in particular, the universal enveloping algebra of gl(n) (or sl(n)) for all n. We show that any Gamma-torsion theory defined by the coheight of the prime ideals of Gamma is liftable to U. Moreover, for any simple U-module M, all associated prime ideals of M in Spec Gamma have the same coheight. Hence, the coheight of these associated prime ideals is an invariant of a given simple U-module. This implies the stratification of the category of U-modules controlled by the coheight of the associated prime ideals of Gamma. Our approach can be viewed as a generalization of the classical paper by Block (1981) [4]; it allows, in particular, to study representations of gl(n) beyond the classical category of weight or generalized weight modules. (C) 2011 Elsevier B.V. All rights reserved.