Аннотация
We present an efficient algorithm for maximum likelihood estimation (MLE) of
the general exponential family, even in cases when the energy function is
represented by a deep neural network. We consider the primal-dual view of the
MLE for the kinectics augmented model, which naturally introduces an
adversarial dual sampler. The sampler will be represented by a novel neural
network architectures, dynamics embeddings, mimicking the dynamical-based
samplers, e.g., Hamiltonian Monte-Carlo and its variants. The dynamics
embedding parametrization inherits the flexibility from HMC, and provides
tractable entropy estimation of the augmented model. Meanwhile, it couples the
adversarial dual samplers with the primal model, reducing memory and sample
complexity. We further show that several existing estimators, including
contrastive divergence (Hinton, 2002), score matching (Hyvärinen, 2005),
pseudo-likelihood (Besag, 1975), noise-contrastive estimation (Gutmann and
Hyvärinen, 2010), non-local contrastive objectives (Vickrey et al., 2010),
and minimum probability flow (Sohl-Dickstein et al., 2011), can be recast as
the special cases of the proposed method with different prefixed dual samplers.
Finally, we empirically demonstrate the superiority of the proposed estimator
against existing state-of-the-art methods on synthetic and real-world
benchmarks.
Описание
[1904.12083] Exponential Family Estimation via Adversarial Dynamics Embedding
Линки и ресурсы
тэги
сообщество