There are generally two types of science: first, there’s the type that makes computers work, allows us to ride around in metal boxes propelled by continuous explosion, and makes it so that milk doesn’t taste all gross. Then there’s the fringe science, the stuff that shoots up your nose like mathematical horseradish and dances a jig on your brain
S. Merity. (2019)cite arxiv:1911.11423Comment: Addition of citations and contextual results (no attention head, single attention head, attention per layer), removal of wordpiece WikiText-103 numbers due to normalization issues, fix of SHA attention figure Q arrow, other minor fixes.