Guaranteed Deterministic Bounds on the Total Variation Distance between
Univariate Mixtures
F. Nielsen, and K. Sun. (2018)cite arxiv:1806.11311Comment: 11 pages, 2 figures.
Abstract
The total variation distance is a core statistical distance between
probability measures that satisfies the metric axioms, with value always
falling in $0,1$. This distance plays a fundamental role in machine learning
and signal processing: It is a member of the broader class of $f$-divergences,
and it is related to the probability of error in Bayesian hypothesis testing.
Since the total variation distance does not admit closed-form expressions for
statistical mixtures (like Gaussian mixture models), one often has to rely in
practice on costly numerical integrations or on fast Monte Carlo approximations
that however do not guarantee deterministic lower and upper bounds. In this
work, we consider two methods for bounding the total variation of univariate
mixture models: The first method is based on the information monotonicity
property of the total variation to design guaranteed nested deterministic lower
bounds. The second method relies on computing the geometric lower and upper
envelopes of weighted mixture components to derive deterministic bounds based
on density ratio. We demonstrate the tightness of our bounds in a series of
experiments on Gaussian, Gamma and Rayleigh mixture models.
Description
[1806.11311] Guaranteed Deterministic Bounds on the Total Variation Distance between Univariate Mixtures
%0 Journal Article
%1 nielsen2018guaranteed
%A Nielsen, Frank
%A Sun, Ke
%D 2018
%K bounds divergences
%T Guaranteed Deterministic Bounds on the Total Variation Distance between
Univariate Mixtures
%U http://arxiv.org/abs/1806.11311
%X The total variation distance is a core statistical distance between
probability measures that satisfies the metric axioms, with value always
falling in $0,1$. This distance plays a fundamental role in machine learning
and signal processing: It is a member of the broader class of $f$-divergences,
and it is related to the probability of error in Bayesian hypothesis testing.
Since the total variation distance does not admit closed-form expressions for
statistical mixtures (like Gaussian mixture models), one often has to rely in
practice on costly numerical integrations or on fast Monte Carlo approximations
that however do not guarantee deterministic lower and upper bounds. In this
work, we consider two methods for bounding the total variation of univariate
mixture models: The first method is based on the information monotonicity
property of the total variation to design guaranteed nested deterministic lower
bounds. The second method relies on computing the geometric lower and upper
envelopes of weighted mixture components to derive deterministic bounds based
on density ratio. We demonstrate the tightness of our bounds in a series of
experiments on Gaussian, Gamma and Rayleigh mixture models.
@article{nielsen2018guaranteed,
abstract = {The total variation distance is a core statistical distance between
probability measures that satisfies the metric axioms, with value always
falling in $[0,1]$. This distance plays a fundamental role in machine learning
and signal processing: It is a member of the broader class of $f$-divergences,
and it is related to the probability of error in Bayesian hypothesis testing.
Since the total variation distance does not admit closed-form expressions for
statistical mixtures (like Gaussian mixture models), one often has to rely in
practice on costly numerical integrations or on fast Monte Carlo approximations
that however do not guarantee deterministic lower and upper bounds. In this
work, we consider two methods for bounding the total variation of univariate
mixture models: The first method is based on the information monotonicity
property of the total variation to design guaranteed nested deterministic lower
bounds. The second method relies on computing the geometric lower and upper
envelopes of weighted mixture components to derive deterministic bounds based
on density ratio. We demonstrate the tightness of our bounds in a series of
experiments on Gaussian, Gamma and Rayleigh mixture models.},
added-at = {2019-12-11T14:23:19.000+0100},
author = {Nielsen, Frank and Sun, Ke},
biburl = {https://www.bibsonomy.org/bibtex/2d2298ece5d7a6516659bf6220dec2243/kirk86},
description = {[1806.11311] Guaranteed Deterministic Bounds on the Total Variation Distance between Univariate Mixtures},
interhash = {f211f871f3461be66da8855787d2e2ab},
intrahash = {d2298ece5d7a6516659bf6220dec2243},
keywords = {bounds divergences},
note = {cite arxiv:1806.11311Comment: 11 pages, 2 figures},
timestamp = {2019-12-11T14:23:19.000+0100},
title = {Guaranteed Deterministic Bounds on the Total Variation Distance between
Univariate Mixtures},
url = {http://arxiv.org/abs/1806.11311},
year = 2018
}