C. Gaudet, and A. Maida. (2017)cite arxiv:1712.04604Comment: 8 pages, 1 figure.
Abstract
The field of deep learning has seen significant advancement in recent years.
However, much of the existing work has been focused on real-valued numbers.
Recent work has shown that a deep learning system using the complex numbers can
be deeper for a fixed parameter budget compared to its real-valued counterpart.
In this work, we explore the benefits of generalizing one step further into the
hyper-complex numbers, quaternions specifically, and provide the architecture
components needed to build deep quaternion networks. We go over quaternion
convolutions, present a quaternion weight initialization scheme, and present
algorithms for quaternion batch-normalization. These pieces are tested in a
classification model by end-to-end training on the CIFAR-10 and CIFAR-100 data
sets and a segmentation model by end-to-end training on the KITTI Road
Segmentation data set. The quaternion networks show improved convergence
compared to real-valued and complex-valued networks, especially on the
segmentation task.
%0 Generic
%1 gaudet2017quaternion
%A Gaudet, Chase
%A Maida, Anthony
%D 2017
%K 2017 arxiv deep-learning paper quaternion
%T Deep Quaternion Networks
%U http://arxiv.org/abs/1712.04604
%X The field of deep learning has seen significant advancement in recent years.
However, much of the existing work has been focused on real-valued numbers.
Recent work has shown that a deep learning system using the complex numbers can
be deeper for a fixed parameter budget compared to its real-valued counterpart.
In this work, we explore the benefits of generalizing one step further into the
hyper-complex numbers, quaternions specifically, and provide the architecture
components needed to build deep quaternion networks. We go over quaternion
convolutions, present a quaternion weight initialization scheme, and present
algorithms for quaternion batch-normalization. These pieces are tested in a
classification model by end-to-end training on the CIFAR-10 and CIFAR-100 data
sets and a segmentation model by end-to-end training on the KITTI Road
Segmentation data set. The quaternion networks show improved convergence
compared to real-valued and complex-valued networks, especially on the
segmentation task.
@misc{gaudet2017quaternion,
abstract = {The field of deep learning has seen significant advancement in recent years.
However, much of the existing work has been focused on real-valued numbers.
Recent work has shown that a deep learning system using the complex numbers can
be deeper for a fixed parameter budget compared to its real-valued counterpart.
In this work, we explore the benefits of generalizing one step further into the
hyper-complex numbers, quaternions specifically, and provide the architecture
components needed to build deep quaternion networks. We go over quaternion
convolutions, present a quaternion weight initialization scheme, and present
algorithms for quaternion batch-normalization. These pieces are tested in a
classification model by end-to-end training on the CIFAR-10 and CIFAR-100 data
sets and a segmentation model by end-to-end training on the KITTI Road
Segmentation data set. The quaternion networks show improved convergence
compared to real-valued and complex-valued networks, especially on the
segmentation task.},
added-at = {2018-07-24T20:18:39.000+0200},
author = {Gaudet, Chase and Maida, Anthony},
biburl = {https://www.bibsonomy.org/bibtex/2d7eb05c84f0cc0837c4cbd980ab47ff4/analyst},
description = {[1712.04604] Deep Quaternion Networks},
interhash = {4d9b081a34ec2e21afd0834f6259a252},
intrahash = {d7eb05c84f0cc0837c4cbd980ab47ff4},
keywords = {2017 arxiv deep-learning paper quaternion},
note = {cite arxiv:1712.04604Comment: 8 pages, 1 figure},
timestamp = {2018-07-24T20:18:39.000+0200},
title = {Deep Quaternion Networks},
url = {http://arxiv.org/abs/1712.04604},
year = 2017
}