Interacting Systems Are Prevalent In Nature, From Dynamical Systems In Physics To Complex Societal Dynamics. The Interplay Of Components Can Give Rise To Complex Behavior, Which Can Often Be Explained Using A Simple Model Of The System's Constituent Parts. In This Work, We Introduce The Neural Relational Inference (nri) Model: An Unsupervised Model That Learns To Infer Interactions While Simultaneously Learning The Dynamics Purely From Observational Data. Our Model Takes The Form Of A Variational Auto-encoder, In Which The Latent Code Represents The Underlying Interaction Graph And The Reconstruction Is Based On Graph Neural Networks. In Experiments On Simulated Physical Systems, We Show That Our Nri Model Can Accurately Recover Ground-truth Interactions In An Unsupervised Manner. We Further Demonstrate That We Can Find An Interpretable Structure And Predict Complex Dynamics In Real Motion Capture And Sports Tracking Data.
Proceedings of the 1st Annual Conference on Robot Learning on 13-15 November 2017 Published as Volume 78 by the Proceedings of Machine Learning Research on 18 October 2017. Volume Edited by: Sergey Levine Vincent Vanhoucke Ken Goldberg Series Editors: Neil D. Lawrence Mark Reid
C. Weng, B. Curless, and I. Kemelmacher-Shlizerman. (2018)cite arxiv:1812.02246Comment: The project page is at https://grail.cs.washington.edu/projects/wakeup/, and the supplementary video is at https://youtu.be/G63goXc5MyU.