Skip to content

Latest commit

 

History

History
6 lines (4 loc) · 253 Bytes

README.md

File metadata and controls

6 lines (4 loc) · 253 Bytes

Synthesizer-attention

A Pytorch implementation of Dense Synthesizer attention and Random Synthesizer attention.

Reference

[1] Tay, Yi, et al. "Synthesizer: Rethinking Self-Attention in Transformer Models." arXiv preprint arXiv:2005.00743 (2020).