Information theory and one-dimensional disordered conductors

A novel method is proposed to treat the problem of the random resistance of a strictly one-dimensional conductor with static disorder. It is suggested, for the probability distribution of the transfer matrix of the conductor, the distribution of maximum information-entropy, constrained by the following physical requirements: 1) flux conservation, 2) time-reversal invariance and 3) scaling, with the length of the conductor, of the two lowest cumulants of ζ, where = sh2ζ. The preliminary results discussed in the text are in qualitative agreement with those obtained by sophisticated microscopic theories.

Information theory and resistance fluctuations in one-dimensional disordered conductors

A novel method is proposed to treat the problem of the random resistance of a strictly one-dimensional conductor with static disorder. For the probability distribution of the transfer matrix R of the conductor we propose a distribution of maximum information entropy, constrained by the following physical requirements: (1) flux conservation, (2) time-reversal invariance, and (3) scaling with the length of the conductor of the two lowest cumulants of ω, where R=exp(iω→⋅Jbhat). The preliminary results discussed in the text are in qualitative agreement with those obtained by sophisticated microscopic theories.

Information Ambiguity

We propose a quantity called information ambiguity that plays the same role in the worst-case information-theoretic nalyses as the well-known notion of information entropy performs in the corresponding average-case analyses. We prove various properties of information ambiguity and illustrate its usefulness in performing the worst-case analysis of a variant of distributed source coding problem.

Information content of molecular graph and prediction of gas phase thermal entropy of organic compounds

Entropy is a fundamental thermodynamic property that has attracted a wide attention across domains, including chemistry. Inference of entropy of chemical compounds using various approaches has been a widely studied topic. However, many aspects of entropy in chemical compounds remain unexplained. In the present work, we propose two new information-theoretical molecular descriptors for the prediction of gas phase thermal entropy of organic compounds. The descriptors reflect the bulk and size of the compounds as well as the gross topological symmetry in their structures, all of which are believed to determine entropy. A high correlation () between the entropy values and our information-theoretical indices have been found and the predicted entropy values, obtained from the corresponding statistically significant regression model, have been found to be within acceptable approximation. We provide additional mathematical result in the form of a theorem and proof that might further help in assessing changes in gas phase thermal entropy values with the changes in molecular structures. The proposed information-theoretical molecular descriptors, regression model and the mathematical result are expected to augment predictions of gas phase thermal entropy for a large number of chemical compounds.

On the axiomatic approach to the maximum entropy principle of inference

Recent axiomatic derivations of the maximum entropy principle from consistency conditions are critically examined. We show that proper application of consistency conditions alone allows a wider class of functionals, essentially of the form ∝ dx p(x)[p(x)/g(x)] s , for some real numbers, to be used for inductive inference and the commonly used form − ∝ dx p(x)ln[p(x)/g(x)] is only a particular case. The role of the prior densityg(x) is clarified. It is possible to regard it as a geometric factor, describing the coordinate system used and it does not represent information of the same kind as obtained by measurements on the system in the form of expectation values.

Discussion of enthalpy, entropy and free energy of formation of GaN

Presented in this letter is a critical discussion of a recent paper on experimental investigation of the enthalpy, entropy and free energy of formation of gallium nitride (GaN) published in this journal [T.J. Peshek, J.C. Angus, K. Kash, J. Cryst. Growth 311 (2008) 185-189]. It is shown that the experimental technique employed detects neither the equilibrium partial pressure of N-2 corresponding to the equilibrium between Ga and GaN at fixed temperatures nor the equilibrium temperature at constant pressure of N-2. The results of Peshek et al. are discussed in the light of other information on the Gibbs energy of formation available in the literature. Entropy of GaN is derived from heat-capacity measurements. Based on a critical analysis of all thermodynamic information now available, a set of optimized parameters is identified and a table of thermodynamic data for GaN developed from 298.15 to 1400 K.

Maximum-entropy calculation of the electron density at 4 A resolution of Pf1 filamentous bacteriophage

A 4 A electron-density map of Pf1 filamentous bacterial virus has been calculated from x-ray fiber diffraction data by using the maximum-entropy method. This method produces a map that is free of features due to noise in the data and enables incomplete isomorphous-derivative phase information to be supplemented by information about the nature of the solution. The map shows gently curved (banana-shaped) rods of density about 70 A long, oriented roughly parallel to the virion axis but slewing by about 1/6th turn while running from a radius of 28 A to one of 13 A. Within these rods, there is a helical periodicity with a pitch of 5 to 6 A. We interpret these rods to be the helical subunits of the virion. The position of strongly diffracted intensity on the x-ray fiber pattern shows that the basic helix of the virion is right handed and that neighboring nearly parallel protein helices cross one another in an unusual negative sense.

The standard enthalpy and entropy of formation of Rh2O3 - A third-law optimization

The standard Gibbs energy of formation of Rh203 at high temperature has been determined recently with high precision. The new data are significantly different from those given in thermodynamic compilations.Accurate values for enthalpy and entropy of formation at 298.15 K could not be evaluated from the new data,because reliable values for heat capacity of Rh2O3 were not available. In this article, a new measurement of the high temperature heat capacity of Rh2O3 using differential scanning calorimetry (DSC) is presented.The new values for heat capacity also differ significantly from those given in compilations. The information on heat capacity is coupled with standard Gibbs energy of formation to evaluate values for standard enthalpy and entropy of formation at 289.15 K using a multivariate analysis. The results suggest a major revision in thermodynamic data for Rh2O3. For example, it is recommended that the standard entropy of Rh203 at 298.15 K be changed from 106.27 J mol-' K-'given in the compilations of Barin and Knacke et al. to 75.69 J mol-' K". The recommended revision in the standard enthalpy of formation is from -355.64 kJ mol-'to -405.53 kJ mol".

Transport in nanoporous zeolites: Relationships between sorbate size, entropy, and diffusivity

Molecular dynamics simulations have been performed on monatomic sorbates confined within zeolite NaY to obtain the dependence of entropy and self-diffusivity on the sorbate diameter. Previously, molecular dynamics simulations by Santikary and Yashonath J. Phys. Chem. 98, 6368 (1994)], theoretical analysis by Derouane J. Catal. 110, 58 (1988)] as well as experiments by Kemball Adv. Catal. 2, 233 (1950)] found that certain sorbates in certain adsorbents exhibit unusually high self-diffusivity. Experiments showed that the loss of entropy for certain sorbates in specific adsorbents was minimum. Kemball suggested that such sorbates will have high self-diffusivity in these adsorbents. Entropy of the adsorbed phase has been evaluated from the trajectory information by two alternative methods: two-phase and multiparticle expansion. The results show that anomalous maximum in entropy is also seen as a function of the sorbate diameter. Further, the experimental observation of Kemball that minimum loss of entropy is associated with maximum in self-diffusivity is found to be true for the system studied here. A suitably scaled dimensionless self-diffusivity shows an exponential dependence on the excess entropy of the adsorbed phase, analogous to excess entropy scaling rules seen in many bulk and confined fluids. The two trajectory-based estimators for the entropy show good semiquantitative agreement and provide some interesting microscopic insights into entropy changes associated with confinement.

Unsupervised feature selection for outlier detection in categorical data using mutual information

Outlier detection in high dimensional categorical data has been a problem of much interest due to the extensive use of qualitative features for describing the data across various application areas. Though there exist various established methods for dealing with the dimensionality aspect through feature selection on numerical data, the categorical domain is actively being explored. As outlier detection is generally considered as an unsupervised learning problem due to lack of knowledge about the nature of various types of outliers, the related feature selection task also needs to be handled in a similar manner. This motivates the need to develop an unsupervised feature selection algorithm for efficient detection of outliers in categorical data. Addressing this aspect, we propose a novel feature selection algorithm based on the mutual information measure and the entropy computation. The redundancy among the features is characterized using the mutual information measure for identifying a suitable feature subset with less redundancy. The performance of the proposed algorithm in comparison with the information gain based feature selection shows its effectiveness for outlier detection. The efficacy of the proposed algorithm is demonstrated on various high-dimensional benchmark data sets employing two existing outlier detection methods.

Minimization Problems Based on Relative alpha-Entropy I: Forward Projection

Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative alpha-entropies (denoted I-alpha), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative alpha-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimizers of these relative alpha-entropies on closed and convex sets are shown to exist. Such minimizations generalize the maximum Renyi or Tsallis entropy principle. The minimizing probability distribution (termed forward I-alpha-projection) for a linear family is shown to obey a power-law. Other results in connection with statistical inference, namely subspace transitivity and iterated projections, are also established. In a companion paper, a related minimization problem of interest in robust statistics that leads to a reverse I-alpha-projection is studied.

Minimization Problems Based on Relative alpha-Entropy II: Reverse Projection

In part I of this two-part work, certain minimization problems based on a parametric family of relative entropies (denoted I-alpha) were studied. Such minimizers were called forward I-alpha-projections. Here, a complementary class of minimization problems leading to the so-called reverse I-alpha-projections are studied. Reverse I-alpha-projections, particularly on log-convex or power-law families, are of interest in robust estimation problems (alpha > 1) and in constrained compression settings (alpha < 1). Orthogonality of the power-law family with an associated linear family is first established and is then exploited to turn a reverse I-alpha-projection into a forward I-alpha-projection. The transformed problem is a simpler quasi-convex minimization subject to linear constraints.

ELISA: An Estimated Load Information Scheduling Algorithm for Distributed Computing Systems

In this paper, we present a decentralized dynamic load scheduling/balancing algorithm called ELISA (Estimated Load Information Scheduling Algorithm) for general purpose distributed computing systems. ELISA uses estimated state information based upon periodic exchange of exact state information between neighbouring nodes to perform load scheduling. The primary objective of the algorithm is to cut down on the communication and load transfer overheads by minimizing the frequency of status exchange and by restricting the load transfer and status exchange within the buddy set of a processor. It is shown that the resulting algorithm performs almost as well as a perfect information algorithm and is superior to other load balancing schemes based on the random sharing and Ni-Hwang algorithms. A sensitivity analysis to study the effect of various design parameters on the effectiveness of load balancing is also carried out. Finally, the algorithm's performance is tested on large dimensional hypercubes in the presence of time-varying load arrival process and is shown to perform well in comparison to other algorithms. This makes ELISA a viable and implementable load balancing algorithm for use in general purpose distributed computing systems.

Efficient use of correlation entropy for analysing time series data

The correlation dimension D 2 and correlation entropy K 2 are both important quantifiers in nonlinear time series analysis. However, use of D 2 has been more common compared to K 2 as a discriminating measure. One reason for this is that D 2 is a static measure and can be easily evaluated from a time series. However, in many cases, especially those involving coloured noise, K 2 is regarded as a more useful measure. Here we present an efficient algorithmic scheme to compute K 2 directly from a time series data and show that K 2 can be used as a more effective measure compared to D 2 for analysing practical time series involving coloured noise.

G(its)(2) VSR: An Information Theoretic Secure Verifiable Secret Redistribution Protocol for Long-term Archival Storage

Protocols for secure archival storage are becoming increasingly important as the use of digital storage for sensitive documents is gaining wider practice. Wong et al.[8] combined verifiable secret sharing with proactive secret sharing without reconstruction and proposed a verifiable secret redistribution protocol for long term storage. However their protocol requires that each of the receivers is honest during redistribution. We proposed[3] an extension to their protocol wherein we relaxed the requirement that all the recipients should be honest to the condition that only a simple majority amongst the recipients need to be honest during the re(distribution) processes. Further, both of these protocols make use of Feldman's approach for achieving integrity during the (redistribution processes. In this paper, we present a revised version of our earlier protocol, and its adaptation to incorporate Pedersen's approach instead of Feldman's thereby achieving information theoretic secrecy while retaining integrity guarantees.