Transformers Explained

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

Prepared and maintained by the Founder of Outcome School: Amit Shekhar

Note: This series will continue to grow as I write more blogs and create more videos on new topics. Keep learning.

Part 1: The Architecture

We start with the complete picture - every component of the Transformer and how data flows through it. The rest of this guide then zooms into each piece.

Decoding Transformer Architecture

In this blog, we will learn about the Transformer architecture by decoding it piece by piece - understanding what each component does, how they work together, and why this architecture powers every modern Large Language Model (LLM).

We will cover the following:

• Why the Transformer was needed
• The two halves of the architecture
• Tokenization, Embedding, and Positional Encoding
• The Attention Mechanism and Multi-Head Attention
• Feed-Forward Networks, Residual Connections, and Layer Normalization
• How the Encoder and Decoder work
• How data flows through the entire architecture
• The three variants of the Transformer
• Why the Transformer is so powerful

Let's get started: Decoding Transformer Architecture

Part 2: Attention

Attention is the heart of the Transformer. In this part, we go from the intuition to the exact math, and then through every attention variant used in modern models.

Self Attention in Transformers

In this blog, we will learn about Self Attention in Transformers. We will understand what it is, how it works step by step, and why it is the heart of modern Large Language Models like BERT and GPT.

We will cover the following:

• What is Self Attention?
• Why do we need Self Attention?
• Query, Key, and Value vectors
• Step-by-step working of Self Attention
• A simple example walk-through
• Why Self Attention works so well
• Multi-Head Self Attention
• Where Self Attention is used

Let's get started: Self Attention in Transformers

Math behind Attention - Q, K, and V

In this blog, we will learn about the math behind Attention - Query(Q),