We observed that generally the embedding representation is very rich and information dense. For example, reducing the dimensionality of the inputs using SVD or PCA, even by 10%, generally results in worse downstream performance on specific tasks.
E. Nie, S. Liang, H. Schmid, und H. Schütze. Findings of the Association for Computational Linguistics: ACL 2023, Seite 8320--8340. Toronto, Canada, Association for Computational Linguistics, (Juli 2023)
X. Liu, T. Zhu, H. Tan, und R. Zhang. The Semantic Web--ISWC 2022: 21st International Semantic Web Conference, Virtual Event, October 23--27, 2022, Proceedings, Seite 284--302. Springer, (2022)