Hierarchical Dirichlet Processes
JOURNAL-AMERICAN STATISTICAL ASSOCIATION, 101 (476):
1566 (2006)Abstract
We consider problems involving groups of data, where each observation within a group is
a draw from a mixture model, and where it is desirable to share mixture components between
groups. We assume that the number of mixture components is unknown a priori and is to be
inferred from the data. In this setting it is natural to consider sets of Dirichlet processes, one
for each group, where the well-known clustering property of the Dirichlet process provides a
nonparametric prior for the number of mixture components within each group. Given our desire
to tie the mixture models in the various groups, we consider a hierarchical model, specifically
one in which the base measure for the child Dirichlet processes is itself distributed according to
a Dirichlet process. Such a base measure being discrete, the child Dirichlet processes necessar-
ily share atoms. Thus, as desired, the mixture models in the different groups necessarily share
mixture components. We discuss representations of hierarchical Dirichlet processes in terms of
a stick-breaking process, and a generalization of the Chinese restaurant process that we refer
to as the “Chinese restaurant franchise.” We present Markov chain Monte Carlo algorithms
for posterior inference in hierarchical Dirichlet process mixtures, and describe applications to
problems in information retrieval and text modelling.
