Abstract
In this article I describe a research agenda for securing machine learning
models against adversarial inputs at test time. This article does not present
results but instead shares some of my thoughts about where I think that the
field needs to go. Modern machine learning works very well on I.I.D. data: data
for which each example is drawn independently and for which the
distribution generating each example is identical. When these assumptions
are relaxed, modern machine learning can perform very poorly. When machine
learning is used in contexts where security is a concern, it is desirable to
design models that perform well even when the input is designed by a malicious
adversary. So far most research in this direction has focused on an adversary
who violates the identical assumption, and imposes some kind of
restricted worst-case distribution shift. I argue that machine learning
security researchers should also address the problem of relaxing the \em
independence assumption and that current strategies designed for robustness to
distribution shift will not do so. I recommend dynamic models that change
each time they are run as a potential solution path to this problem, and show
an example of a simple attack using correlated data that can be mitigated by a
simple dynamic defense. This is not intended as a real-world security measure,
but as a recommendation to explore this research direction and develop more
realistic defenses.
Users
Please
log in to take part in the discussion (add own reviews or comments).