Abstract
Unordered, variable-sized inputs arise in many settings across multiple
fields. The ability for set- and multiset- oriented neural networks to handle
this type of input has been the focus of much work in recent years. We propose
to represent multisets using complex-weighted multiset automata and show how
the multiset representations of certain existing neural architectures can be
viewed as special cases of ours. Namely, (1) we provide a new theoretical and
intuitive justification for the Transformer model's representation of positions
using sinusoidal functions, and (2) we extend the DeepSets model to use complex
numbers, enabling it to outperform the existing model on an extension of one of
their tasks.
Users
Please
log in to take part in the discussion (add own reviews or comments).