Computational information geometry deals with the study and design of efficient algorithms in information spaces
using the language of geometry (such as invariance, distance, projection, ball, etc).
Historically, the field was pioneered by C.R. Rao in 1945 who proposed to use the Fisher information metric as the Riemannian metric.
This seminal work gave birth to the geometrization of statistics (eg, statistical curvature and second-order efficiency).
In statistics, invariance (by non-singular 1-to-1 reparametrization and sufficient statistics) yield the class of f-divergences, including the celebrated Kullback-Leibler divergence.
The differential geometry of f-divergences can be analyzed using dual alpha-connections.
Common algorithms in machine learning (such as clustering, expectation-maximization, statistical estimating, regression, independent component analysis, boosting, etc) can be revisited and further
explored using those concepts.
Nowadays, the framework of computational information geometry opens up novel horizons in music, multimedia, radar, and finance/economy.
Generic algorithms: Design meta-algorithms that can handle any arbitrary parameterized distance or loss function.
(The usual class of parameterized distances are Bregman, Csiszar and Burbea-Rao divergences.)
Examples: K-means, expectation maximization (EM), Voronoi diagrams, barycenters, smallest enclosing balls, ball trees, etc.
Geometry of information.
Examples: Dually flat spaces of exponential/mixture families (VC-dimension of balls remains unchanged to d+1), Riemmanian geometry, Finsler geometry (HARDI datasets), etc.
Let us give some examples of information manifolds:
Statistical manifolds (parametric distributions),
Neural manifolds (Boltzmann machines with fixed topology, i.e., number of nodes),
ARMA(p,q) time-series manifolds (e-flat=-1-flat)
Strictly speaking, geometrizing information-theoretic problems does not provide a more powerful framework in theory.
This is because synthetical and analytical geometries are equivalent.
Informally, that means that we can do geometry by algebraic equations.
However, geometrizing problems help grab intuition on the problem at hand.
Geometry also yields novel notions to mathematical theories.
For example, let us cite the two curvature notions in statistics: exponential and mixture curvatures emanating from conjugate connections.
So although synthetical geometry provides the same power as analytical geometry, the third-order asymptotic theory of statistics has been obtained so far only from synthetical information geometry.
Dual differential geometries are also useful to tackle information-theoretic problems such as
Multiterminal problems met in information theory,
Linear programming problems (e.g., continuous Karmarkar inner method walking along the m-geodesic),
Clustering (negative entropy and dual Legendre log-normalizer conjugate for soft/hard clustering).
Feature in computer vision http://svr-www.eng.cam.ac.uk/~er258/work/fast.html
Information Theory and Applications Center
http://www.cis.upenn.edu/~cis610/ (Jean Gallier)
http://www.nec.co.jp/rd/en/ccil/ Keiji Yamada
SIAM. J. Matrix Anal. & Appl. (SIMAX)
Journal of Elasticity, Springer
International Journal of Computational Intelligence Research (1)
International Journal of Computer Mathematics (1)