info_theory.hpp

Tools to calculate Information Theory metrics.

Info-theory formulas: H(X) = -SUM(X: p[x] log2 p[x]) H(X|Y) = H(XY) - H(Y) I(X:Y) = H(X) - H(X|Y) H2(p) = -p log2(p) - (1-p)log2(1-p) = H({p, 1-p})

Developer notes:

• Input may come as WEIGHTS or as ELEMENTS (or both!). ELEMENTS need to be converted to WEIGHTS for calculations.

• Want basic info theory functions, as well as tools (for channels, error-correction, compression, etc.)

Note

Status: ALPHA

Functions

template<typename CONTAINER>
double Entropy(const CONTAINER &weights)

Convert a vector of weights to probabilities and return the entropy of the system.

template<typename CONTAINER, typename WEIGHT_FUN>
double Entropy(const CONTAINER &objs, WEIGHT_FUN fun, double total = 0.0)

Calculate the entropy in a container of arbitrary objects. Args are a container, a function to extract the weight of each member, and an (optional) total weight.

constexpr double Entropy2(const double p)

Calculate the entropy when their are two possibile states based on one state’s probability.