Abstract
We explore the use of semantic word embeddings in text segmentation
algorithms, including the C99 segmentation algorithm and new algorithms
inspired by the distributed word vector representation. By developing a general
framework for discussing a class of segmentation objectives, we study the
effectiveness of greedy versus exact optimization approaches and suggest a new
iterative refinement technique for improving the performance of greedy
strategies. We compare our results to known benchmarks, using known metrics. We
demonstrate state-of-the-art performance for an untrained method with our
Content Vector Segmentation (CVS) on the Choi test set. Finally, we apply the
segmentation procedure to an in-the-wild dataset consisting of text extracted
from scholarly articles in the arXiv.org database.
Users
Please
log in to take part in the discussion (add own reviews or comments).