Author of the publication

SparseBERT: Rethinking the Importance Analysis in Self-attention.

, , , , , , and . ICML, volume 139 of Proceedings of Machine Learning Research, page 9547-9557. PMLR, (2021)

Please choose a person to relate this publication to

To differ between persons with the same name, the academic degree and the title of an important publication will be displayed. You can also use the button next to the name to display some publications already assigned to the person.

 

Other publications of authors with the same name

ScheMoE: An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling., , , , , , , , and . EuroSys, page 236-249. ACM, (2024)AutoBERT-Zero: Evolving BERT Backbone from Scratch., , , , , , , and . AAAI, page 10663-10671. AAAI Press, (2022)EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation., , , , , and . EMNLP (Findings), page 1424-1437. Association for Computational Linguistics, (2021)CAPE: Context-Adaptive Positional Encoding for Length Extrapolation., , , , , , , , , and 1 other author(s). CoRR, (2024)A Survey of Reasoning with Foundation Models., , , , , , , , , and 24 other author(s). CoRR, (2023)SparseBERT: Rethinking the Importance Analysis in Self-attention., , , , , , and . ICML, volume 139 of Proceedings of Machine Learning Research, page 9547-9557. PMLR, (2021)One Student Knows All Experts Know: From Sparse to Dense., , , , and . Tiny Papers @ ICLR, OpenReview.net, (2023)NumGPT: Improving Numeracy Ability of Generative Pre-trained Models., , , , , , and . CoRR, (2021)A Study on Transformer Configuration and Training Objective., , , , , , , , and . ICML, volume 202 of Proceedings of Machine Learning Research, page 38913-38925. PMLR, (2023)CAME: Confidence-guided Adaptive Memory Efficient Optimization., , , , , and . ACL (1), page 4442-4453. Association for Computational Linguistics, (2023)