We show that a recurrent neural network is able to learn a model to represent
sequences of communications between computers on a network and can be used to
identify outlier network traffic. Defending computer networks is a challenging
problem and is typically addressed by manually identifying known malicious
actor behavior and then specifying rules to recognize such behavior in network
communications. However, these rule-based approaches often generalize poorly
and identify only those patterns that are already known to researchers. An
alternative approach that does not rely on known malicious behavior patterns
can potentially also detect previously unseen patterns. We tokenize and
compress netflow into sequences of "words" that form "sentences" representative
of a conversation between computers. These sentences are then used to generate
a model that learns the semantic and syntactic grammar of the newly generated
language. We use Long-Short-Term Memory (LSTM) cell Recurrent Neural Networks
(RNN) to capture the complex relationships and nuances of this language. The
language model is then used predict the communications between two IPs and the
prediction error is used as a measurement of how typical or atyptical the
observed communication are. By learning a model that is specific to each
network, yet generalized to typical computer-to-computer traffic within and
outside the network, a language model is able to identify sequences of network
activity that are outliers with respect to the model. We demonstrate positive
unsupervised attack identification performance (AUC 0.84) on the ISCX IDS
dataset which contains seven days of network activity with normal traffic and
four distinct attack patterns.
Description
Network Traffic Anomaly Detection Using Recurrent Neural Networks
%0 Generic
%1 radford2018network
%A Radford, Benjamin J.
%A Apolonio, Leonardo M.
%A Trias, Antonio J.
%A Simpson, Jim A.
%D 2018
%K anomaly_detection deep_learning lstm netflow network nlp security
%T Network Traffic Anomaly Detection Using Recurrent Neural Networks
%U http://arxiv.org/abs/1803.10769
%X We show that a recurrent neural network is able to learn a model to represent
sequences of communications between computers on a network and can be used to
identify outlier network traffic. Defending computer networks is a challenging
problem and is typically addressed by manually identifying known malicious
actor behavior and then specifying rules to recognize such behavior in network
communications. However, these rule-based approaches often generalize poorly
and identify only those patterns that are already known to researchers. An
alternative approach that does not rely on known malicious behavior patterns
can potentially also detect previously unseen patterns. We tokenize and
compress netflow into sequences of "words" that form "sentences" representative
of a conversation between computers. These sentences are then used to generate
a model that learns the semantic and syntactic grammar of the newly generated
language. We use Long-Short-Term Memory (LSTM) cell Recurrent Neural Networks
(RNN) to capture the complex relationships and nuances of this language. The
language model is then used predict the communications between two IPs and the
prediction error is used as a measurement of how typical or atyptical the
observed communication are. By learning a model that is specific to each
network, yet generalized to typical computer-to-computer traffic within and
outside the network, a language model is able to identify sequences of network
activity that are outliers with respect to the model. We demonstrate positive
unsupervised attack identification performance (AUC 0.84) on the ISCX IDS
dataset which contains seven days of network activity with normal traffic and
four distinct attack patterns.
@misc{radford2018network,
abstract = {We show that a recurrent neural network is able to learn a model to represent
sequences of communications between computers on a network and can be used to
identify outlier network traffic. Defending computer networks is a challenging
problem and is typically addressed by manually identifying known malicious
actor behavior and then specifying rules to recognize such behavior in network
communications. However, these rule-based approaches often generalize poorly
and identify only those patterns that are already known to researchers. An
alternative approach that does not rely on known malicious behavior patterns
can potentially also detect previously unseen patterns. We tokenize and
compress netflow into sequences of "words" that form "sentences" representative
of a conversation between computers. These sentences are then used to generate
a model that learns the semantic and syntactic grammar of the newly generated
language. We use Long-Short-Term Memory (LSTM) cell Recurrent Neural Networks
(RNN) to capture the complex relationships and nuances of this language. The
language model is then used predict the communications between two IPs and the
prediction error is used as a measurement of how typical or atyptical the
observed communication are. By learning a model that is specific to each
network, yet generalized to typical computer-to-computer traffic within and
outside the network, a language model is able to identify sequences of network
activity that are outliers with respect to the model. We demonstrate positive
unsupervised attack identification performance (AUC 0.84) on the ISCX IDS
dataset which contains seven days of network activity with normal traffic and
four distinct attack patterns.},
added-at = {2019-05-17T08:47:44.000+0200},
author = {Radford, Benjamin J. and Apolonio, Leonardo M. and Trias, Antonio J. and Simpson, Jim A.},
biburl = {https://www.bibsonomy.org/bibtex/258ae48bd3218388ac375dfa5c96feb80/dallmann},
description = {Network Traffic Anomaly Detection Using Recurrent Neural Networks},
interhash = {329ccd783b3bf49be2becb69242a1354},
intrahash = {58ae48bd3218388ac375dfa5c96feb80},
keywords = {anomaly_detection deep_learning lstm netflow network nlp security},
note = {cite arxiv:1803.10769Comment: Prepared for the 2017 National Symposium on Sensor and Data Fusion},
timestamp = {2019-05-17T08:47:44.000+0200},
title = {Network Traffic Anomaly Detection Using Recurrent Neural Networks},
url = {http://arxiv.org/abs/1803.10769},
year = 2018
}