Skip to main content Link Menu Expand (external link) Document Search Copy Copied
Nguyen'Log

The Transformer Notes

Table of contents
  1. The Transformer Notes
    1. Introduction
    2. Positional Encoder
    3. Transformer Encoder
    4. Transformer Decoder
    5. Word Embedding
    6. Other Operators
    7. References

Introduction

Transformer architecture Figure 1: Transformer architecture

Main components include:

  • Positional Encoder
  • Transformer Encoder
  • Transformer Decoder
  • Word (Dictionary) Embedding

Positional Encoder

Transformer Encoder

Transformer Decoder

Word Embedding

Other Operators

References

  1. Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge University Press. https://doi.org/10.1017/9781108679930
  2. Klein, G., Kim, Y., Deng, Y., Senellart, J., & Rush, A. M. (2021). The Annotated Transformer. https://nlp.seas.harvard.edu/2018/04/03/attention.html
    [Online; accessed 2023-04-04]