Minimization Problems Based on Relative alpha-Entropy I: Forward Projection
Data(s) |
2015
|
---|---|
Resumo |
Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative alpha-entropies (denoted I-alpha), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative alpha-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimizers of these relative alpha-entropies on closed and convex sets are shown to exist. Such minimizations generalize the maximum Renyi or Tsallis entropy principle. The minimizing probability distribution (termed forward I-alpha-projection) for a linear family is shown to obey a power-law. Other results in connection with statistical inference, namely subspace transitivity and iterated projections, are also established. In a companion paper, a related minimization problem of interest in robust statistics that leads to a reverse I-alpha-projection is studied. |
Formato |
application/pdf |
Identificador |
http://eprints.iisc.ernet.in/52389/1/IEEE_Tra_on_Inf_The_61-9_5063_2015.pdf Kumar, Ashok M and Sundaresan, Rajesh (2015) Minimization Problems Based on Relative alpha-Entropy I: Forward Projection. In: IEEE TRANSACTIONS ON INFORMATION THEORY, 61 (9). pp. 5063-5080. |
Publicador |
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
Relação |
http://dx.doi.org/10.1109/TIT.2015.2449311 http://eprints.iisc.ernet.in/52389/ |
Palavras-Chave | #Electrical Communication Engineering |
Tipo |
Journal Article PeerReviewed |