D. Blalock, and J. Guttag. (2021)cite arxiv:2106.10860Comment: To appear at ICML 2021.
Abstract
Multiplying matrices is among the most fundamental and compute-intensive
operations in machine learning. Consequently, there has been significant work
on efficiently approximating matrix multiplies. We introduce a learning-based
algorithm for this task that greatly outperforms existing methods. Experiments
using hundreds of matrices from diverse domains show that it often runs
$100\times$ faster than exact matrix products and $10\times$ faster than
current approximate methods. In the common case that one matrix is known ahead
of time, our method also has the interesting property that it requires zero
multiply-adds. These results suggest that a mixture of hashing, averaging, and
byte shuffling$-$the core operations of our method$-$could be a more promising
building block for machine learning than the sparsified, factorized, and/or
scalar quantized matrix products that have recently been the focus of
substantial research and hardware investment.
Description
[2106.10860] Multiplying Matrices Without Multiplying
%0 Generic
%1 blalock2021multiplying
%A Blalock, Davis
%A Guttag, John
%D 2021
%K 2021 matrix
%T Multiplying Matrices Without Multiplying
%U http://arxiv.org/abs/2106.10860
%X Multiplying matrices is among the most fundamental and compute-intensive
operations in machine learning. Consequently, there has been significant work
on efficiently approximating matrix multiplies. We introduce a learning-based
algorithm for this task that greatly outperforms existing methods. Experiments
using hundreds of matrices from diverse domains show that it often runs
$100\times$ faster than exact matrix products and $10\times$ faster than
current approximate methods. In the common case that one matrix is known ahead
of time, our method also has the interesting property that it requires zero
multiply-adds. These results suggest that a mixture of hashing, averaging, and
byte shuffling$-$the core operations of our method$-$could be a more promising
building block for machine learning than the sparsified, factorized, and/or
scalar quantized matrix products that have recently been the focus of
substantial research and hardware investment.
@misc{blalock2021multiplying,
abstract = {Multiplying matrices is among the most fundamental and compute-intensive
operations in machine learning. Consequently, there has been significant work
on efficiently approximating matrix multiplies. We introduce a learning-based
algorithm for this task that greatly outperforms existing methods. Experiments
using hundreds of matrices from diverse domains show that it often runs
$100\times$ faster than exact matrix products and $10\times$ faster than
current approximate methods. In the common case that one matrix is known ahead
of time, our method also has the interesting property that it requires zero
multiply-adds. These results suggest that a mixture of hashing, averaging, and
byte shuffling$-$the core operations of our method$-$could be a more promising
building block for machine learning than the sparsified, factorized, and/or
scalar quantized matrix products that have recently been the focus of
substantial research and hardware investment.},
added-at = {2021-09-04T13:47:43.000+0200},
author = {Blalock, Davis and Guttag, John},
biburl = {https://www.bibsonomy.org/bibtex/26c4b52435a2abfca7769233390a58be6/analyst},
description = {[2106.10860] Multiplying Matrices Without Multiplying},
interhash = {94ed2cd5ff9821fa265cfdedeb31fb47},
intrahash = {6c4b52435a2abfca7769233390a58be6},
keywords = {2021 matrix},
note = {cite arxiv:2106.10860Comment: To appear at ICML 2021},
timestamp = {2021-09-04T13:47:43.000+0200},
title = {Multiplying Matrices Without Multiplying},
url = {http://arxiv.org/abs/2106.10860},
year = 2021
}