Fully complex-valued radial basis function networks: orthogonal least squares regression and classification

We consider a fully complex-valued radial basis function (RBF) network for regression and classification applications. For regression problems, the locally regularised orthogonal least squares (LROLS) algorithm aided with the D-optimality experimental design, originally derived for constructing parsimonious real-valued RBF models, is extended to the fully complex-valued RBF (CVRBF) network. Like its real-valued counterpart, the proposed algorithm aims to achieve maximised model robustness and sparsity by combining two effective and complementary approaches. The LROLS algorithm alone is capable of producing a very parsimonious model with excellent generalisation performance while the D-optimality design criterion further enhances the model efficiency and robustness. By specifying an appropriate weighting for the D-optimality cost in the combined model selecting criterion, the entire model construction procedure becomes automatic. An example of identifying a complex-valued nonlinear channel is used to illustrate the regression application of the proposed fully CVRBF network. The proposed fully CVRBF network is also applied to four-class classification problems that are typically encountered in communication systems. A complex-valued orthogonal forward selection algorithm based on the multi-class Fisher ratio of class separability measure is derived for constructing sparse CVRBF classifiers that generalise well. The effectiveness of the proposed algorithm is demonstrated using the example of nonlinear beamforming for multiple-antenna aided communication systems that employ complex-valued quadrature phase shift keying modulation scheme. (C) 2007 Elsevier B.V. All rights reserved.

Construction of tunable radial basis function networks using orthogonal forward selection

An orthogonal forward selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO misclassification rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstrated using examples taken from both regression and classification applications.

On improvement of classification accuracy for stochastic discrimination

Stochastic discrimination (SD) depends on a discriminant function for classification. In this paper, an improved SD is introduced to reduce the error rate of the standard SD in the context of a two-class classification problem. The learning procedure of the improved SD consists of two stages. Initially a standard SD, but with shorter learning period is carried out to identify an important space where all the misclassified samples are located. Then the standard SD is modified by 1) restricting sampling in the important space, and 2) introducing a new discriminant function for samples in the important space. It is shown by mathematical derivation that the new discriminant function has the same mean, but with a smaller variance than that of the standard SD for samples in the important space. It is also analyzed that the smaller the variance of the discriminant function, the lower the error rate of the classifier. Consequently, the proposed improved SD improves standard SD by its capability of achieving higher classification accuracy. Illustrative examples are provided to demonstrate the effectiveness of the proposed improved SD.

Particle swarm optimization aided orthogonal forward regression for unified data modeling

We propose a unified data modeling approach that is equally applicable to supervised regression and classification applications, as well as to unsupervised probability density function estimation. A particle swarm optimization (PSO) aided orthogonal forward regression (OFR) algorithm based on leave-one-out (LOO) criteria is developed to construct parsimonious radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines the center vector and diagonal covariance matrix of one RBF node by minimizing the LOO statistics. For regression applications, the LOO criterion is chosen to be the LOO mean square error, while the LOO misclassification rate is adopted in two-class classification applications. By adopting the Parzen window estimate as the desired response, the unsupervised density estimation problem is transformed into a constrained regression problem. This PSO aided OFR algorithm for tunable-node RBF networks is capable of constructing very parsimonious RBF models that generalize well, and our analysis and experimental results demonstrate that the algorithm is computationally even simpler than the efficient regularization assisted orthogonal least square algorithm based on LOO criteria for selecting fixed-node RBF models. Another significant advantage of the proposed learning procedure is that it does not have learning hyperparameters that have to be tuned using costly cross validation. The effectiveness of the proposed PSO aided OFR construction procedure is illustrated using several examples taken from regression and classification, as well as density estimation applications.

A multi-filter enhanced genetic ensemble system for gene selection and sample classification of microarray data

Background: Feature selection techniques are critical to the analysis of high dimensional datasets. This is especially true in gene selection from microarray data which are commonly with extremely high feature-to-sample ratio. In addition to the essential objectives such as to reduce data noise, to reduce data redundancy, to improve sample classification accuracy, and to improve model generalization property, feature selection also helps biologists to focus on the selected genes to further validate their biological hypotheses.
Results: In this paper we describe an improved hybrid system for gene selection. It is based on a recently proposed genetic ensemble (GE) system. To enhance the generalization property of the selected genes or gene subsets and to overcome the overfitting problem of the GE system, we devised a mapping strategy to fuse the goodness information of each gene provided by multiple filtering algorithms. This information is then used for initialization and mutation operation of the genetic ensemble system.
Conclusion: We used four benchmark microarray datasets (including both binary-class and multi-class classification problems) for concept proving and model evaluation. The experimental results indicate that the proposed multi-filter enhanced genetic ensemble (MF-GE) system is able to improve sample classification accuracy, generate more compact gene subset, and converge to the selection results more quickly. The MF-GE system is very flexible as various combinations of multiple filters and classifiers can be incorporated based on the data characteristics and the user preferences.

Uso de confiabilidade na rotulação de exemplos em problemas de classificação multirrótulo com aprendizado semissupervisionado

The techniques of Machine Learning are applied in classification tasks to acquire knowledge through a set of data or information. Some learning methods proposed in literature are methods based on semissupervised learning; this is represented by small percentage of labeled data (supervised learning) combined with a quantity of label and non-labeled examples (unsupervised learning) during the training phase, which reduces, therefore, the need for a large quantity of labeled instances when only small dataset of labeled instances is available for training. A commom problem in semi-supervised learning is as random selection of instances, since most of paper use a random selection technique which can cause a negative impact. Much of machine learning methods treat single-label problems, in other words, problems where a given set of data are associated with a single class; however, through the requirement existent to classify data in a lot of domain, or more than one class, this classification as called multi-label classification. This work presents an experimental analysis of the results obtained using semissupervised learning in troubles of multi-label classification using reliability parameter as an aid in the classification data. Thus, the use of techniques of semissupervised learning and besides methods of multi-label classification, were essential to show the results

Investigando a combinação de técnicas de aprendizado semissupervisionado e classificação hierárquica multirrótulo

Data classification is a task with high applicability in a lot of areas. Most methods for treating classification problems found in the literature dealing with single-label or traditional problems. In recent years has been identified a series of classification tasks in which the samples can be labeled at more than one class simultaneously (multi-label classification). Additionally, these classes can be hierarchically organized (hierarchical classification and hierarchical multi-label classification). On the other hand, we have also studied a new category of learning, called semi-supervised learning, combining labeled data (supervised learning) and non-labeled data (unsupervised learning) during the training phase, thus reducing the need for a large amount of labeled data when only a small set of labeled samples is available. Thus, since both the techniques of multi-label and hierarchical multi-label classification as semi-supervised learning has shown favorable results with its use, this work is proposed and used to apply semi-supervised learning in hierarchical multi-label classication tasks, so eciently take advantage of the main advantages of the two areas. An experimental analysis of the proposed methods found that the use of semi-supervised learning in hierarchical multi-label methods presented satisfactory results, since the two approaches were statistically similar results

Morphometric analysis of treatment effects of the Balters bionator in growing Class II patients

Objective: To investigate the effects of the standard (Class II) Balters bionator in growing patients with Class II malocclusion with mandibular retrusion by using morphometrics (thin-plate spline [TPS] analysis). Materials and Methods: Thirty-one Class II patients (17 male and 14 female) were treated with the Balters bionator (bionator group). Mean age at the start of treatment (T0) was 10.3 years, while it was 13 years at the end of treatment (T1). Mean treatment time was 2 years and 2 months. The control group consisted of 22 subjects (14 male and 8 female) with untreated Class II malocclusion. Mean age at T0 was 10.2 years, while it was 12.2 years at T1. The observation period lasted 2 years on average. TPS analysis evaluated statistical (permutation tests) differences in the craniofacial shape and size between the bionator and control groups. Results: Through TPS analysis (deformation grids) the bionator group showed significant shape changes in the mandible that could be described as a mandibular forward and downward displacement. The control group showed no statistically significant differences in the correction of Class II malocclusion. Conclusions: Bionator appliance is able to induce significant mandibular shape changes that lead to the correction of Class II dentoskeletal disharmony. © 2013 by The EH Angle Education and Research Foundation, Inc.

Explorando abordagens de múltiplos rótulos por floresta de caminhos ótimos

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Enamel matrix protein derivative plus synthetic bone substitute for the treatment of mandibular Class II furcation defects: a case series

Objectives: The aim of this study is to report on the treatment of mandibular Class II furcation defects with enamel matrix protein derivative (EMD) combined with a beta TCP/HA (beta-tricalcium phosphate/hydroxyapatite) alloplastic material. Method and Materials: Thirteen patients were selected. All patients were nonsmokers, systemically healthy, and diagnosed with chronic periodontitis; had not taken medications known to interfere with periodontal tissue health and healing; presented one Class II mandibular furcation defect with horizontal probing equal to or greater than 4 mm at buccal site. The clinical parameters evaluated were probing depth (PD), relative gingival margin position (RGMP), relative vertical clinical attachment level (RVCAL), and relative horizontal clinical attachment level (RHCAL). A paired Student t test was used to detect differences between the baseline and 6-month measurements, with the level of significance of .05. Results: After 6 months, the treatment produced a statistically significant reduction in PD and a significant gain in RVCAL and RHCAL, but no observable change in RGMP. RVCAL ranged from 13.77 (+/- 1.31) at baseline to 12.15 (+/- 1.29) after 6 months, with a mean change of -1.62 +/- 1.00 mm (P<.05). RHCAL ranged from 5.54 (+/- 0.75) to 2.92 (+/- 0.92), with a mean change of -2.62 +/- 0.63 mm (P<.05). After 6 months, 76.92% of the patients improved their diagnosis to Class I furcation defects while 23.08% remained as Class II. Conclusion: The present study has shown that positive clinical results may be expected from the combined treatment of Class II furcation defects with EMD and beta TCP/HA, especially considering the gain of horizontal attachment level. Despite this result, controlled clinical studies are needed to confirm our outcomes.

Ctp1 and the MRN-complex are required for endonucleolytic Rec12 removal with release of a single class of oligonucleotides in fission yeast

DNA double-strand breaks (DSBs) are formed during meiosis by the action of the topoisomerase-like Spo11/Rec12 protein, which remains covalently bound to the 5' ends of the broken DNA. Spo11/Rec12 removal is required for resection and initiation of strand invasion for DSB repair. It was previously shown that budding yeast Spo11, the homolog of fission yeast Rec12, is removed from DNA by endonucleolytic cleavage. The release of two Spo11 bound oligonucleotide classes, heterogeneous in length, led to the conjecture of asymmetric cleavage. In fission yeast, we found only one class of oligonucleotides bound to Rec12 ranging in length from 17 to 27 nucleotides. Ctp1, Rad50, and the nuclease activity of Rad32, the fission yeast homolog of Mre11, are required for endonucleolytic Rec12 removal. Further, we detected no Rec12 removal in a rad50S mutant. However, strains with additional loss of components localizing to the linear elements, Hop1 or Mek1, showed some Rec12 removal, a restoration depending on Ctp1 and Rad32 nuclease activity. But, deletion of hop1 or mek1 did not suppress the phenotypes of ctp1Delta and the nuclease dead mutant (rad32-D65N). We discuss what consequences for subsequent repair a single class of Rec12-oligonucleotides may have during meiotic recombination in fission yeast in comparison to two classes of Spo11-oligonucleotides in budding yeast. Furthermore, we hypothesize on the participation of Hop1 and Mek1 in Rec12 removal.

Markov blanket-based approach for learning multi-dimensional Bayesian network classifiers: An application to predict the European Quality of Life-5 Dimensions (EQ-5D) from the 39-item Parkinson's Disease Questionnaire (PDQ-39)

Multi-dimensional Bayesian network classifiers (MBCs) are probabilistic graphical models recently proposed to deal with multi-dimensional classification problems, where each instance in the data set has to be assigned to more than one class variable. In this paper, we propose a Markov blanket-based approach for learning MBCs from data. Basically, it consists of determining the Markov blanket around each class variable using the HITON algorithm, then specifying the directionality over the MBC subgraphs. Our approach is applied to the prediction problem of the European Quality of Life-5 Dimensions (EQ-5D) from the 39-item Parkinson’s Disease Questionnaire (PDQ-39) in order to estimate the health-related quality of life of Parkinson’s patients. Fivefold cross-validation experiments were carried out on randomly generated synthetic data sets, Yeast data set, as well as on a real-world Parkinson’s disease data set containing 488 patients. The experimental study, including comparison with additional Bayesian network-based approaches, back propagation for multi-label learning, multi-label k-nearest neighbor, multinomial logistic regression, ordinary least squares, and censored least absolute deviations, shows encouraging results in terms of predictive accuracy as well as the identification of dependence relationships among class and feature variables.

New definitions of margin for multi-class Boosting algorithms

La familia de algoritmos de Boosting son un tipo de técnicas de clasificación y regresión que han demostrado ser muy eficaces en problemas de Visión Computacional. Tal es el caso de los problemas de detección, de seguimiento o bien de reconocimiento de caras, personas, objetos deformables y acciones. El primer y más popular algoritmo de Boosting, AdaBoost, fue concebido para problemas binarios. Desde entonces, muchas han sido las propuestas que han aparecido con objeto de trasladarlo a otros dominios más generales: multiclase, multilabel, con costes, etc. Nuestro interés se centra en extender AdaBoost al terreno de la clasificación multiclase, considerándolo como un primer paso para posteriores ampliaciones. En la presente tesis proponemos dos algoritmos de Boosting para problemas multiclase basados en nuevas derivaciones del concepto margen. El primero de ellos, PIBoost, está concebido para abordar el problema descomponiéndolo en subproblemas binarios. Por un lado, usamos una codificación vectorial para representar etiquetas y, por otro, utilizamos la función de pérdida exponencial multiclase para evaluar las respuestas. Esta codificación produce un conjunto de valores margen que conllevan un rango de penalizaciones en caso de fallo y recompensas en caso de acierto. La optimización iterativa del modelo genera un proceso de Boosting asimétrico cuyos costes dependen del número de etiquetas separadas por cada clasificador débil. De este modo nuestro algoritmo de Boosting tiene en cuenta el desbalanceo debido a las clases a la hora de construir el clasificador. El resultado es un método bien fundamentado que extiende de manera canónica al AdaBoost original. El segundo algoritmo propuesto, BAdaCost, está concebido para problemas multiclase dotados de una matriz de costes. Motivados por los escasos trabajos dedicados a generalizar AdaBoost al terreno multiclase con costes, hemos propuesto un nuevo concepto de margen que, a su vez, permite derivar una función de pérdida adecuada para evaluar costes. Consideramos nuestro algoritmo como la extensión más canónica de AdaBoost para este tipo de problemas, ya que generaliza a los algoritmos SAMME, Cost-Sensitive AdaBoost y PIBoost. Por otro lado, sugerimos un simple procedimiento para calcular matrices de coste adecuadas para mejorar el rendimiento de Boosting a la hora de abordar problemas estándar y problemas con datos desbalanceados. Una serie de experimentos nos sirven para demostrar la efectividad de ambos métodos frente a otros conocidos algoritmos de Boosting multiclase en sus respectivas áreas. En dichos experimentos se usan bases de datos de referencia en el área de Machine Learning, en primer lugar para minimizar errores y en segundo lugar para minimizar costes. Además, hemos podido aplicar BAdaCost con éxito a un proceso de segmentación, un caso particular de problema con datos desbalanceados. Concluimos justificando el horizonte de futuro que encierra el marco de trabajo que presentamos, tanto por su aplicabilidad como por su flexibilidad teórica. Abstract The family of Boosting algorithms represents a type of classification and regression approach that has shown to be very effective in Computer Vision problems. Such is the case of detection, tracking and recognition of faces, people, deformable objects and actions. The first and most popular algorithm, AdaBoost, was introduced in the context of binary classification. Since then, many works have been proposed to extend it to the more general multi-class, multi-label, costsensitive, etc... domains. Our interest is centered in extending AdaBoost to two problems in the multi-class field, considering it a first step for upcoming generalizations. In this dissertation we propose two Boosting algorithms for multi-class classification based on new generalizations of the concept of margin. The first of them, PIBoost, is conceived to tackle the multi-class problem by solving many binary sub-problems. We use a vectorial codification to represent class labels and a multi-class exponential loss function to evaluate classifier responses. This representation produces a set of margin values that provide a range of penalties for failures and rewards for successes. The stagewise optimization of this model introduces an asymmetric Boosting procedure whose costs depend on the number of classes separated by each weak-learner. In this way the Boosting procedure takes into account class imbalances when building the ensemble. The resulting algorithm is a well grounded method that canonically extends the original AdaBoost. The second algorithm proposed, BAdaCost, is conceived for multi-class problems endowed with a cost matrix. Motivated by the few cost-sensitive extensions of AdaBoost to the multi-class field, we propose a new margin that, in turn, yields a new loss function appropriate for evaluating costs. Since BAdaCost generalizes SAMME, Cost-Sensitive AdaBoost and PIBoost algorithms, we consider our algorithm as a canonical extension of AdaBoost to this kind of problems. We additionally suggest a simple procedure to compute cost matrices that improve the performance of Boosting in standard and unbalanced problems. A set of experiments is carried out to demonstrate the effectiveness of both methods against other relevant Boosting algorithms in their respective areas. In the experiments we resort to benchmark data sets used in the Machine Learning community, firstly for minimizing classification errors and secondly for minimizing costs. In addition, we successfully applied BAdaCost to a segmentation task, a particular problem in presence of imbalanced data. We conclude the thesis justifying the horizon of future improvements encompassed in our framework, due to its applicability and theoretical flexibility.

Three essays on dual-class stock structure

Dual-class stock structure is characterized by the separation of voting rights and cash flow rights. The departure from a common "one share-one vote" configuration creates ideal conditions for conflicts of interest and agency problems between controlling insiders (the holders of voting rights) and remaining shareholders. The owners of voting rights have the opportunity to extract private benefits and act in their personal interest; as a result, dual-class firms are often perceived to have low transparency and high information asymmetry. This dissertation investigates the quality of information and the information environment of firms with two classes of stock. The first essay examines the quality of information by studying accruals in dual-class firms in comparison to firms with only one class of stock. The results suggest that the quality of accruals is better in dual-class firms than in single-class firms. In addition, the difference in the quality of accruals between firms that abolish their dual-class share structure by unification and singe-class firms disappears in the post-unification period. The second essay investigates the earnings informativeness of dual-class firms by examining the explanatory power of earnings for returns. The results indicate that the earnings informativeness is lower for dual-class firms as compared to single-class firms. Earnings informativeness improves in firms that unify their shares. The third essay compares the level of information asymmetry between dual-class firms and single-class firms. It is documented that the information environment for dual-class firms is worse than for single-class firms. Also, the finding suggests that the difference in information environment between dual-class firms and single-class firms disappears after dual-class stock unification.

Three Essays on Dual-Class Stock Structure

Dual-class stock structure is characterized by the separation of voting rights and cash flow rights. The departure from a common “one share-one vote” configuration creates ideal conditions for conflicts of interest and agency problems between controlling insiders (the holders of voting rights) and remaining shareholders. The owners of voting rights have the opportunity to extract private benefits and act in their personal interest; as a result, dual-class firms are often perceived to have low transparency and high information asymmetry. This dissertation investigates the quality of information and the information environment of firms with two classes of stock. The first essay examines the quality of information by studying accruals in dual-class firms in comparison to firms with only one class of stock. The results suggest that the quality of accruals is better in dual-class firms than in single-class firms. In addition, the difference in the quality of accruals between firms that abolish their dual-class share structure by unification and singe-class firms disappears in the post-unification period. The second essay investigates the earnings informativeness of dual-class firms by examining the explanatory power of earnings for returns. The results indicate that the earnings informativeness is lower for dual-class firms as compared to single-class firms. Earnings informativeness improves in firms that unify their shares. The third essay compares the level of information asymmetry between dual-class firms and single-class firms. It is documented that the information environment for dual-class firms is worse than for single-class firms. Also, the finding suggests that the difference in information environment between dual-class firms and single-class firms disappears after dual-class stock unification.