Evolutionary structure learning algorithm for Bayesian network and penalized mutual information metric

This paper formulates the problem of learning Bayesian network structures from data as determining the structure that best approximates the probability distribution indicated by the data. A new metric, Penalized Mutual Information metric, is proposed, and a evolutionary algorithm is designed to search for the best structure among alternatives. The experimental results show that this approach is reliable and promising.

Adaptive blind source separation using constant modulus criterion and signal mutual information

We address the problem of adaptive blind source separation (BSS) from instantaneous multi-input multi-output (MIMO) channels. It is known that the constant modulus (CM) criterion can be used to extract unknown source signals. However, the existing CM based algorithms normally extract the source signals in a serial manner. Consequently, the accuracy in extracting each source signal, except for the first one, depends on the accuracy of previous source extraction. This estimation error propagation (accumulation) causes severe performance degradation. In this paper, we propose a new adaptive separation algorithm that can separate all source signals simultaneously by directly updating the separation matrix. The superior performance of the new algorithm is demonstrated by simulation examples

Comments on 'Information measure for performance of image fusion'

The unsuitability of using classic mutual information measure as a performance measure for image fusion is discussed. Analytical proof that classic mutual information cannot be considered a measure for image fusion performance is provided.

An ensemble of classifiers with genetic algorithmBased Feature Selection

Different data classification algorithms have been developed and applied in various areas to analyze and extract valuable information and patterns from large datasets with noise and missing values. However, none of them could consistently perform well over all datasets. To this end, ensemble methods have been suggested as the promising measures. This paper proposes a novel hybrid algorithm, which is the combination of a multi-objective Genetic Algorithm (GA) and an ensemble classifier. While the ensemble classifier, which consists of a decision tree classifier, an Artificial Neural Network (ANN) classifier, and a Support Vector Machine (SVM) classifier, is used as the classification committee, the multi-objective Genetic Algorithm is employed as the feature selector to facilitate the ensemble classifier to improve the overall sample classification accuracy while also identifying the most important features in the dataset of interest. The proposed GA-Ensemble method is tested on three benchmark datasets, and compared with each individual classifier as well as the methods based on mutual information theory, bagging and boosting. The results suggest that this GA-Ensemble method outperform other algorithms in comparison, and be a useful method for classification and feature selection problems.

Measuring the capacity of image fusion

Multisource image fusion is usually achieved by repeatedly fusing source images in pairs. However, there is no guarantee on the delivered quality considering the amount of information to be squeezed into the same spatial dimension. This paper presents a fusion capacity measure and examines the limits at which fusing more images will not add further information. The fusion capacity index employs Mutual Information (MI) to measure how far the histogram of the examined image is from a uniformly distributed histogram of a saturated image.

Quantitative experimental comparison of single-beam, sidescan, and multibeam benthic habitat maps

Map comparison is a relatively uncommon practice in acoustic seabed classification to date, contrary to the field of land remote sensing, where it has been developed extensively over recent decades. The aim here is to illustrate the benefits of map comparison in the underwater realm with a case study of three maps independently describing the seabed habitats of the Te Matuku Marine Reserve (Hauraki Gulf, New Zealand). The maps are obtained from a QTC View classification of a single-beam echosounder (SBES) dataset, manual segmentation of a sidescan sonar (SSS) mosaic, and automatic classification of a backscatter dataset from a multibeam echosounder (MBES). The maps are compared using pixel-to-pixel similarity measures derived from the literature in land remote sensing. All measures agree in presenting the MBES and SSS maps as the most similar, and the SBES and SSS maps as the least similar. The results are discussed with reference to the potential of MBES backscatter as an alternative to SSS mosaic for imagery segmentation and to the potential of joint SBES–SSS survey for improved habitat mapping. Other applications of map-similarity measures in acoustic classification of the seabed are suggested.

Classification of healthcare data using genetic fuzzy logic system and wavelets

Healthcare plays an important role in promoting the general health and well-being of people around the world. The difficulty in healthcare data classification arises from the uncertainty and the high-dimensional nature of the medical data collected. This paper proposes an integration of fuzzy standard additive model (SAM) with genetic algorithm (GA), called GSAM, to deal with uncertainty and computational challenges. GSAM learning process comprises three continual steps: rule initialization by unsupervised learning using the adaptive vector quantization clustering, evolutionary rule optimization by GA and parameter tuning by the gradient descent supervised learning. Wavelet transformation is employed to extract discriminative features for high-dimensional datasets. GSAM becomes highly capable when deployed with small number of wavelet features as its computational burden is remarkably reduced. The proposed method is evaluated using two frequently-used medical datasets: the Wisconsin breast cancer and Cleveland heart disease from the UCI Repository for machine learning. Experiments are organized with a five-fold cross validation and performance of classification techniques are measured by a number of important metrics: accuracy, F-measure, mutual information and area under the receiver operating characteristic curve. Results demonstrate the superiority of the GSAM compared to other machine learning methods including probabilistic neural network, support vector machine, fuzzy ARTMAP, and adaptive neuro-fuzzy inference system. The proposed approach is thus helpful as a decision support system for medical practitioners in the healthcare practice.

EEG data classification using wavelet features selected by Wilcoxon statistics

This paper introduces a method to classify EEG signals using features extracted by an integration of wavelet transform and the nonparametric Wilcoxon test. Orthogonal Haar wavelet coefficients are ranked based on the Wilcoxon test’s statistics. The most prominent discriminant wavelets are assembled to form a feature set that serves as inputs to the naïve Bayes classifier. Two benchmark datasets, named Ia and Ib, downloaded from the brain–computer interface (BCI) competition II are employed for the experiments. Classification performance is evaluated using accuracy, mutual information, Gini coefficient and F-measure. Widely used classifiers, including feedforward neural network, support vector machine, k-nearest neighbours, ensemble learning Adaboost and adaptive neuro-fuzzy inference system, are also implemented for comparisons. The proposed combination of Haar wavelet features and naïve Bayes classifier considerably dominates the competitive classification approaches and outperforms the best performance on the Ia and Ib datasets reported in the BCI competition II. Application of naïve Bayes also provides a low computational cost approach that promotes the implementation of a potential real-time BCI system.

Blind estimation of MIMO relay channels

In this paper, we integrate two blind source separation (BSS) methods to estimate the individual channel state information (CSI) for the source-relay and relay-destination links of three-node two-hop multiple-input multiple-output (MIMO) relay systems. In particular, we propose a first-order Z-domain precoding technique for the blind estimation of the relay-destination channel matrix, while an algorithm based on the constant modulus and mutual information properties is developed to estimate the source-relay channel matrix. Compared with training-based MIMO relay channel estimation approaches, our algorithm has a better bandwidth efficiency as no bandwidth is wasted for sending the training sequences. Numerical examples are shown to demonstrate the performance of the proposed algorithm. © 2014 IEEE.

Fuzzy system with tabu search learning for classification of motor imagery data

This paper introduces an approach to classify EEG signals using wavelet transform and a fuzzy standard additive model (FSAM) with tabu search learning mechanism. Wavelet coefficients are ranked based on statistics of the Wilcoxon test. The most informative coefficients are assembled to form a feature set that serves as inputs to the tabu-FSAM. Two benchmark datasets, named Ia and Ib, downloaded from the brain-computer interface (BCI) competition II are employed for the experiments. Classification performance is evaluated using accuracy, mutual information, Gini coefficient and F-measure. Widely-used classifiers, including feedforward neural network, support vector machine, k-nearest neighbours, ensemble learning Adaboost and adaptive neuro-fuzzy inference system, are also implemented for comparisons. The proposed tabu-FSAM method considerably dominates the competitive classifiers, and outperforms the best performance on the Ia and Ib datasets reported in the BCI competition II.

Expected shannon entropy and shannon differentiation between subpopulations for neutral genes under the finite island model

Shannon entropy H and related measures are increasingly used in molecular ecology and population genetics because (1) unlike measures based on heterozygosity or allele number, these measures weigh alleles in proportion to their population fraction, thus capturing a previously-ignored aspect of allele frequency distributions that may be important in many applications; (2) these measures connect directly to the rich predictive mathematics of information theory; (3) Shannon entropy is completely additive and has an explicitly hierarchical nature; and (4) Shannon entropy-based differentiation measures obey strong monotonicity properties that heterozygosity-based measures lack. We derive simple new expressions for the expected values of the Shannon entropy of the equilibrium allele distribution at a neutral locus in a single isolated population under two models of mutation: the infinite allele model and the stepwise mutation model. Surprisingly, this complex stochastic system for each model has an entropy expressable as a simple combination of well-known mathematical functions. Moreover, entropy- and heterozygosity-based measures for each model are linked by simple relationships that are shown by simulations to be approximately valid even far from equilibrium. We also identify a bridge between the two models of mutation. We apply our approach to subdivided populations which follow the finite island model, obtaining the Shannon entropy of the equilibrium allele distributions of the subpopulations and of the total population. We also derive the expected mutual information and normalized mutual information ("Shannon differentiation") between subpopulations at equilibrium, and identify the model parameters that determine them. We apply our measures to data from the common starling (Sturnus vulgaris) in Australia. Our measures provide a test for neutrality that is robust to violations of equilibrium assumptions, as verified on real world data from starlings.

On the security of compressed sensing-based signal cryptosystem

With the development of the cyber-physical systems (CPS), the security analysis of the data therein becomes more and more important. Recently, due to the advantage of joint encryption and compression for data transmission in CPS, the emerging compressed sensing (CS)-based cryptosystem has attracted much attention, where security is of extreme importance. The existing methods only analyze the security of the plaintext under the assumption that the key is absolutely safe. However, for sparse plaintext, the prior sparsity knowledge of the plaintext could be exploited to partly retrieve the key, and then the plaintext, from the ciphertext. So, the existing methods do not provide a satisfactory security analysis. In this paper, it is conducted in the information theory frame, where the plaintext sparsity feature and the mutual information of the ciphertext, key, and plaintext are involved. In addition, the perfect secrecy criteria (Shannon-sense and Wyner-sense) are extended to measure the security. While the security level is given, the illegal access risk is also discussed. It is shown that the CS-based cryptosystem achieves the extended Wyner-sense perfect secrecy, but when the key is used repeatedly, both the plaintext and the key could be conditionally accessed.

Blind channel estimation and signal retrieving for MIMO relay systems

In this paper, we propose a blind channel estimation and signal retrieving algorithm for two-hop multiple-input multiple-output (MIMO) relay systems. This new algorithm integrates two blind source separation (BSS) methods to estimate the individual channel state information (CSI) of the source-relay and relay-destination links. In particular, a first-order Z-domain precoding technique is developed for the blind estimation of the relay-destination channel matrix, where the signals received at the relay node are pre-processed by a set of precoders before being transmitted to the destination node. With the estimated signals at the relay node, we propose an algorithm based on the constant modulus and signal mutual information properties to estimate the source-relay channel matrix. Compared with training-based MIMO relay channel estimation approaches, the proposed algorithm has a better bandwidth efficiency as no bandwidth is wasted for sending the training sequences. Numerical examples are shown to demonstrate the performance of the proposed algorithm.