Sasha Rush (Cornell University) “Pretraining Without Attention”
January 17, 2023
When:
February 3, 2023 @ 12:00 pm – 1:15 pm
Where:
Hackerman Hall B17, 3400 N. Charles Street, Baltimore, MD 21218
Abstract Transformers are essential to pretraining. As we approach 5 years of BERT, the connection between attention as architecture and transfer learning remains key to this central thread in NLP. Other architectures such as CNNs and RNNs[…]
