Inequalities for Csiszár f-Divergence in Information Theory
One of the important issues in many applications of Probability Theory is finding an appropriate measure of distance (or difference or discrimination) between two probability distributions. A number of divergence measures for this purpose have been proposed and extensively studied by Jeffreys, Kullback and Leibler, Renyi, Havrda and Charvat, Kapur, Sharma and Mittal, Burbea and Rao, Rao, Lin, Csiszár, Ali and Silvey, Vajda, Shioya and Da-te among others.
These measures have been applied in a variety of fields such as: anthropology, genetics, finance, economics, political science, biology, the analysis of contingency tables, approximation of probability distributions, signal processing and pattern recognition. A number of these measures of distance are specific cases of Csiszár f-divergence and so further exploration of this concept will have a flow on effect to other measures of distance and to areas in which they are applied.
The main aim of the present collection of papers written by some members of the RGMIA (Research Group in Mathematical Inequalities and Applications) is to prove the fact that the Modern Theory of Inequalities may have an important impact in other branches of Mathematical Sciences.
In the Chapter I, by using convexity arguments and recent developments on Jensen type inequalities, some new bounds for Csiszár f-divergence are given.
In Chapter II, the general Csiszár f-divergence is compared with some particular divergences which play a fundamental role in Information Theory, such as: the Kullback-Leibler distance, Hellinger discrimination, Variational distances and others.
Using Taylor's expansion and recent generalisations of Taylor's formula, in Chapter III, accurate approximations of Csiszár f-divergence for mappings which are not necessarily convex are given. This is a domain where the authors are churning out more results which will be communicated at a later stage.
Finally, in Chapter IV
and by the use of some Ostrowski type inequalities, the authors point
out some natural connections between Csiszár f-divergence and
other types of inequalities. It is the express intention of the present team to apply the other
fundamental Ostrowski type inequalities discovered recently (see for
example the book online:
Note: This book is not in final form. The Editor
invites researchers with comments to contact him for their results to be included in a new version.
To reference this book, please use the following:
S.S. DRAGOMIR (Ed.), Inequalities for Csiszár f-Divergence in Information Theory, RGMIA Monographs, Victoria University, 2000. (ONLINE: http://ajmaa.org/RGMIA/monographs.php/).