Misc,
Graph-Coupled Oscillator Networks
(2022)cite arxiv:2202.02296Comment: ICML 2022.
Abstract
We propose Graph-Coupled Oscillator Networks (GraphCON), a novel framework
for deep learning on graphs. It is based on discretizations of a second-order
system of ordinary differential equations (ODEs), which model a network of
nonlinear controlled and damped oscillators, coupled via the adjacency
structure of the underlying graph. The flexibility of our framework permits any
basic GNN layer (e.g. convolutional or attentional) as the coupling function,
from which a multi-layer deep neural network is built up via the dynamics of
the proposed ODEs. We relate the oversmoothing problem, commonly encountered in
GNNs, to the stability of steady states of the underlying ODE and show that
zero-Dirichlet energy steady states are not stable for our proposed ODEs. This
demonstrates that the proposed framework mitigates the oversmoothing problem.
Moreover, we prove that GraphCON mitigates the exploding and vanishing
gradients problem to facilitate training of deep multi-layer GNNs. Finally, we
show that our approach offers competitive performance with respect to the
state-of-the-art on a variety of graph-based learning tasks.
