Attention Is All You Need

Introduces the Transformer, a sequence model with no recurrence or convolution. Every token attends to every other token through stacked multi-head self-attention, so training runs in parallel across the whole sequence.

Why it matters

It first set records in machine translation, then became the base of nearly every modern language model and spread to vision, audio and beyond.

Read the paper: arXiv:1706.03762

Plain-language summary written by SolxLab. Read the full paper at the official link; all rights remain with the authors.