Groups, information theory, and Einstein's likelihood principle

We propose a unifying picture where the notion of generalized entropy is related to information theory by means of a group-theoretical approach. The group structure comes from the requirement that an entropy be well defined with respect to the composition of independent systems, in the context of a recently proposed generalization of the Shannon-Khinchin axioms. We associate to each member of a large class of entropies a generalized information measure, satisfying the additivity property on a set of independent systems as a consequence of the underlying group law. At the same time, we also show that Einstein's likelihood function naturally emerges as a byproduct of our informational interpretation of (generally nonadditive) entropies. These results confirm the adequacy of composable entropies both in physical and social science contexts.

A novel information theory method for filter feature selection

In this paper, we propose a novel filter for feature selection. Such filter relies on the estimation of the mutual information between features and classes. We bypass the estimation of the probability density function with the aid of the entropic-graphs approximation of Rényi entropy, and the subsequent approximation of the Shannon one. The complexity of such bypassing process does not depend on the number of dimensions but on the number of patterns/samples, and thus the curse of dimensionality is circumvented. We show that it is then possible to outperform a greedy algorithm based on the maximal relevance and minimal redundancy criterion. We successfully test our method both in the contexts of image classification and microarray data classification.

Information theory and curricular synthesis,

Mode of access: Internet.

Science and information theory.

Mode of access: Internet.

Principles of coding, filtering, and information theory.

Includes bibliography.

Signal or Noise? Information Theory and the Novel

In the context of a hostile funding environment, universities are increasingly asked to justify their output in narrowly defined economic terms, and this can be difficult in Humanities or Arts faculties where productivity is rarely reducible to a simple financial indicator. This can lead to a number of immediate consequences that I have no need to rehearse here, but can also result in some interesting tensions within the academic community itself. First is that which has become known as the ‘Science Wars’: the increasingly acrimonious exchanges between scientists and scientific academics and cultural critics or theorists about who has the right to describe the world. Much has already been said—and much remains to be said—about this issue, but it is not my intention to discuss it here. Rather, I will look at a second area of contestation: the incorporation of scientific theory into literary or cultural criticism. Much of this work comes from a genuine commitment to interdisciplinarity, and an appreciation of insights that a fresh perspective can bring to a familiar object. However, some can be seen as cynical attempts to lend literary studies the sort of empirical legitimacy of the sciences. In particular, I want to look at a number of critics who have applied information theory to the literary work. In this paper, I will examine several instances of this sort of criticism, and then, through an analysis of a novel by American author Richard Powers, Three Farmers on Their Way to a Dance, show how this sort of criticism merely reduces the meaningful analysis of a complex literary text.

Simple Gallager codes for BSC

We investigate the performance of Gallager type error- correcting codes for Binary Symmetric Channels, where the code word comprises products of K bits selected from the original message and decoding is carried out utilizing a connectivity tensor with C connections per index. Shannon's bound for the channel capacity is recovered for large K and zero temperature when the code rate K/C is finite. Close to optimal error-correcting capability, with improved decoding properties is obtained for finite K and C.

Statistical physics of irregular low-density parity-check codes

Low-density parity-check codes with irregular constructions have recently been shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon's coding bound when appropriate parameters are chosen. The decoding by belief propagation is also studied using statistical physics arguments; the theoretical solutions obtained are in good agreement with simulation results. We compare the performance of irregular codes with that of regular codes and discuss the factors that contribute to the improvement in performance.

Transmission of image data over digital networks involving mobile terminals

There is a growing demand for data transmission over digital networks involving mobile terminals. An important class of data required for transmission over mobile terminals is image information such as street maps, floor plans and identikit images. This sort of transmission is of particular interest to the service industries such as the Police force, Fire brigade, medical services and other services. These services cannot be applied directly to mobile terminals because of the limited capacity of the mobile channels and the transmission errors caused by the multipath (Rayleigh) fading. In this research, transmission of line diagram images such as floor plans and street maps, over digital networks involving mobile terminals at transmission rates of 2400 bits/s and 4800 bits/s have been studied. A low bit-rate source encoding technique using geometric codes is found to be suitable to represent line diagram images. In geometric encoding, the amount of data required to represent or store the line diagram images is proportional to the image detail. Thus a simple line diagram image would require a small amount of data. To study the effect of transmission errors due to mobile channels on the transmitted images, error sources (error files), which represent mobile channels under different conditions, have been produced using channel modelling techniques. Satisfactory models of the mobile channel have been obtained when compared to the field test measurements. Subjective performance tests have been carried out to evaluate the quality and usefulness of the received line diagram images under various mobile channel conditions. The effect of mobile transmission errors on the quality of the received images has been determined. To improve the quality of the received images under various mobile channel conditions, forward error correcting codes (FEC) with interleaving and automatic repeat request (ARQ) schemes have been proposed. The performance of the error control codes have been evaluated under various mobile channel conditions. It has been shown that a FEC code with interleaving can be used effectively to improve the quality of the received images under normal and severe mobile channel conditions. Under normal channel conditions, similar results have been obtained when using ARQ schemes. However, under severe mobile channel conditions, the FEC code with interleaving shows better performance.

Some new Results for Additive Self-Dual Codes over GF(4)

* Supported by COMBSTRU Research Training Network HPRN-CT-2002-00278 and the Bulgarian National Science Foundation under Grant MM-1304/03.

Basic Structure of the General Information Theory

The basic structure of the General Information Theory (GIT) is presented in the paper. The main divisions of the GIT are outlined. Some new results are pointed.

Information theory analysis of regenerative channels

In this paper we summarize our recently proposed work on the information theory analysis of regenerative channels. We discuss how the design and the transfer function properties of the regenerator affect the noise statistics and enable Shannon capacities higher than that of the corresponding linear channels (in the absence of regeneration).

Capital structure, risk and asymmetric information : theory and evidence

This paper proposes a principal-agent model between banks and firms with risk and asymmetric information. A mixed form of finance to firms is assumed. The capital structure of firms is a relevant cause for the final aggregate level of investment in the economy. In the model analyzed, there may be a separating equilibrium, which is not economically efficient, because aggregate investments fall short of the first-best level. Based on European firm-level data, an empirical model is presented which validates the result of the relevance of the capital structure of firms. The relative magnitude of equity in the capital structure makes a real difference to the profits obtained by firms in the economy.