Abstract
Recent DNN pruning algorithms have succeeded in reducing the number of
parameters in fully connected layers, often with little or no drop in
classification accuracy. However, most of the existing pruning schemes either
have to be applied during training or require a costly retraining procedure
after pruning to regain classification accuracy. We start by proposing a cheap
pruning algorithm for fully connected DNN layers based on difference of convex
functions (DC) optimisation, that requires little or no retraining. We then
provide a theoretical analysis for the growth in the Generalization Error (GE)
of a DNN for the case of bounded perturbations to the hidden layers, of which
weight pruning is a special case. Our pruning method is orders of magnitude
faster than competing approaches, while our theoretical analysis sheds light to
previously observed problems in DNN pruning. Experiments on commnon feedforward
neural networks validate our results.
Users
Please
log in to take part in the discussion (add own reviews or comments).