Article,
Spectral Methods for Data Science: A Statistical Perspective
(2020)cite arxiv:2012.08496.
Abstract
Spectral methods have emerged as a simple yet surprisingly effective approach
for extracting information from massive, noisy and incomplete data. In a
nutshell, spectral methods refer to a collection of algorithms built upon the
eigenvalues (resp. singular values) and eigenvectors (resp. singular vectors)
of some properly designed matrices constructed from data. A diverse array of
applications have been found in machine learning, data science, and signal
processing. Due to their simplicity and effectiveness, spectral methods are not
only used as a stand-alone estimator, but also frequently employed to
initialize other more sophisticated algorithms to improve performance.
While the studies of spectral methods can be traced back to classical matrix
perturbation theory and methods of moments, the past decade has witnessed
tremendous theoretical advances in demystifying their efficacy through the lens
of statistical modeling, with the aid of non-asymptotic random matrix theory.
This monograph aims to present a systematic, comprehensive, yet accessible
introduction to spectral methods from a modern statistical perspective,
highlighting their algorithmic implications in diverse large-scale
applications. In particular, our exposition gravitates around several central
questions that span various applications: how to characterize the sample
efficiency of spectral methods in reaching a target level of statistical
accuracy, and how to assess their stability in the face of random noise,
missing data, and adversarial corruptions? In addition to conventional $\ell_2$
perturbation analysis, we present a systematic $\ell_ınfty$ and
$\ell_2,ınfty$ perturbation theory for eigenspace and singular subspaces,
which has only recently become available owing to a powerful "leave-one-out"
analysis framework.
