Recently, convolutional neural networks with 3D kernels (3D CNNs) have been
very popular in computer vision community as a result of their superior ability
of extracting spatio-temporal features within video frames compared to 2D CNNs.
Although there has been great advances recently to build resource efficient 2D
CNN architectures considering memory and power budget, there is hardly any
similar resource efficient architectures for 3D CNNs. In this paper, we have
converted various well-known resource efficient 2D CNNs to 3D CNNs and
evaluated their performance on three major benchmarks in terms of
classification accuracy for different complexity levels. We have experimented
on (1) Kinetics-600 dataset to inspect their capacity to learn, (2) Jester
dataset to inspect their ability to capture motion patterns, and (3) UCF-101 to
inspect the applicability of transfer learning. We have evaluated the run-time
performance of each model on a single Titan XP GPU and a Jetson TX2 embedded
system. The results of this study show that these models can be utilized for
different types of real-world applications since they provide real-time
performance with considerable accuracies and memory usage. Our analysis on
different complexity levels shows that the resource efficient 3D CNNs should
not be designed too shallow or narrow in order to save complexity. The codes
and pretrained models used in this work are publicly available.
Description
[1904.02422] Resource Efficient 3D Convolutional Neural Networks
%0 Generic
%1 kopuklu2019resource
%A Köpüklü, Okan
%A Kose, Neslihan
%A Gunduz, Ahmet
%A Rigoll, Gerhard
%D 2019
%K 2019 3D cnn deep-learning iccv
%T Resource Efficient 3D Convolutional Neural Networks
%U http://arxiv.org/abs/1904.02422
%X Recently, convolutional neural networks with 3D kernels (3D CNNs) have been
very popular in computer vision community as a result of their superior ability
of extracting spatio-temporal features within video frames compared to 2D CNNs.
Although there has been great advances recently to build resource efficient 2D
CNN architectures considering memory and power budget, there is hardly any
similar resource efficient architectures for 3D CNNs. In this paper, we have
converted various well-known resource efficient 2D CNNs to 3D CNNs and
evaluated their performance on three major benchmarks in terms of
classification accuracy for different complexity levels. We have experimented
on (1) Kinetics-600 dataset to inspect their capacity to learn, (2) Jester
dataset to inspect their ability to capture motion patterns, and (3) UCF-101 to
inspect the applicability of transfer learning. We have evaluated the run-time
performance of each model on a single Titan XP GPU and a Jetson TX2 embedded
system. The results of this study show that these models can be utilized for
different types of real-world applications since they provide real-time
performance with considerable accuracies and memory usage. Our analysis on
different complexity levels shows that the resource efficient 3D CNNs should
not be designed too shallow or narrow in order to save complexity. The codes
and pretrained models used in this work are publicly available.
@misc{kopuklu2019resource,
abstract = {Recently, convolutional neural networks with 3D kernels (3D CNNs) have been
very popular in computer vision community as a result of their superior ability
of extracting spatio-temporal features within video frames compared to 2D CNNs.
Although there has been great advances recently to build resource efficient 2D
CNN architectures considering memory and power budget, there is hardly any
similar resource efficient architectures for 3D CNNs. In this paper, we have
converted various well-known resource efficient 2D CNNs to 3D CNNs and
evaluated their performance on three major benchmarks in terms of
classification accuracy for different complexity levels. We have experimented
on (1) Kinetics-600 dataset to inspect their capacity to learn, (2) Jester
dataset to inspect their ability to capture motion patterns, and (3) UCF-101 to
inspect the applicability of transfer learning. We have evaluated the run-time
performance of each model on a single Titan XP GPU and a Jetson TX2 embedded
system. The results of this study show that these models can be utilized for
different types of real-world applications since they provide real-time
performance with considerable accuracies and memory usage. Our analysis on
different complexity levels shows that the resource efficient 3D CNNs should
not be designed too shallow or narrow in order to save complexity. The codes
and pretrained models used in this work are publicly available.},
added-at = {2020-01-08T20:28:48.000+0100},
author = {Köpüklü, Okan and Kose, Neslihan and Gunduz, Ahmet and Rigoll, Gerhard},
biburl = {https://www.bibsonomy.org/bibtex/22fdff8a650416ff78c489a23fa24b544/analyst},
description = {[1904.02422] Resource Efficient 3D Convolutional Neural Networks},
interhash = {b2153530f60b9c733dc0136f65289b7c},
intrahash = {2fdff8a650416ff78c489a23fa24b544},
keywords = {2019 3D cnn deep-learning iccv},
note = {cite arxiv:1904.02422Comment: Accepted to ICCV 2019 workshop - Neural Architects},
timestamp = {2020-01-08T20:29:31.000+0100},
title = {Resource Efficient 3D Convolutional Neural Networks},
url = {http://arxiv.org/abs/1904.02422},
year = 2019
}