Fused Kullback-Leibler divergence for sets of distributions

Function calculating the Kullback-Leibler divergence between two sets of multivariate normal distributions. In other words, it calculates a weigthed mean of Kullback-Leibler divergences between multiple paired normal distributions.

KLdiv.fused(MtestList, MrefList, StestList, SrefList, ns, symmetric = FALSE)

Arguments

MtestList

A list of mean vectors of the approximating multivariate normal distribution for each class. Assumed to be zero vectors if not supplied.

MrefList

A list of mean vectors of the reference multivariate normal distribution for each class. Assumed to be zero vectors if not supplied.

StestList

A list of covariance matrices of the approximating multivariate normal distribtuion for each class. Usually a list of sample covariance matrices.

SrefList

A list of covariance matrices of the references multivariate normal distribtuion for each class. Usually a list of the population or reference covariance matrices.

ns

a numeric of the same length as the previous arguments giving the sample sizes. Used as weights in the weighted mean.

symmetric

a logical indicating if original symmetric version of KL divergence should be calculated.

Value

Function returns a numeric representing the (optionally symmetric) fused Kullback-Leibler divergence.

See also

KLdiv

Author

Anders Ellern Bilgrau, Wessel N. van Wieringen, Carel F.W. Peeters <carel.peeters@wur.nl>

Examples

# Create some toy data
n <- c(40, 60, 80)
p <- 10
Stest <- replicate(length(n), diag(p), simplify = FALSE)
Sref <- createS(n, p = p)

KLdiv.fused(StestList = Stest, SrefList = Sref, ns = n, symmetric = FALSE)
#> [1] 0.5379534
KLdiv.fused(StestList = Stest, SrefList = Sref, ns = n, symmetric = TRUE)
#> [1] 1.29974