The paper discusses the capabilities of large pre-trained language models and their limitations in accessing and manipulating knowledge. The authors introduce retrieval-augmented generation (RAG) models that combine pre-trained parametric and non-parametric memory for language generation. The study explores the effectiveness of RAG models in various NLP tasks and compares them with other architectures.
O. Khattab, und M. Zaharia. Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, Seite 39--48. (2020)
T. Fornaciari, A. Uma, S. Paun, B. Plank, D. Hovy, und M. Poesio. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seite 2591--2597. Online, Association for Computational Linguistics, (Juni 2021)
I. Ryazanov, und J. Björklund. 35th Annual Workshop of the Swedish Artificial Intelligence Society SAIS 2023, Volume 199 von SAIS 2023, Seite 94–99. Linköping University Electronic Press, (Juni 2023)