为什么值得关注

该仓库由经验丰富的作者精心编写，提供结构清晰的教程系列，既讲解直觉概念又包含数学推导，并覆盖 GQA、Flash Attention、RoPE、混合专家等现代变体。

适合谁使用

机器学习工程师和深度学习从业者
学习 Transformer 模型的学生和研究人员
使用大型语言模型的开发者
准备 Transformer 相关技术面试的人员

典型使用场景

从头学习完整的 Transformer 架构
理解不同注意力变体及其权衡
掌握位置嵌入和归一化背后的数学原理
学习混合专家（MoE）和视觉 Transformer 等现代扩展

项目优势

涵盖从分词到输出层的每一个主要组件
同时提供概念解释和详细的数值示例
涉及最新进展（GQA、Flash Attention、RoPE、RMSNorm、MoE、ViT）
据 README 所述，会持续更新博客和视频

使用前须知

所有内容托管在外部网站（Outcome School 博客/Substack），仓库本身不包含内容
没有提供代码实现或可运行的示例
假设读者具备一定的机器学习基础，可能不适合完全零基础的初学者

README 快速开始

Transformers Explained

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

Prepared and maintained by the Founder of Outcome School: Amit Shekhar

Note: This series will continue to grow as I write more blogs and create more videos on new topics. Keep learning.

Part 1: The Architecture

We start with the complete picture - every component of the Transformer and how data flows through it. The rest of this guide then zooms into each piece.

Decoding Transformer Architecture

In this blog, we will learn about the Transformer architecture by decoding it piece by piece - understanding what each component does, how they work together, and why this architecture powers every modern Large Language Model (LLM).

We will cover the following:

Why the Transformer was needed
The two halves of the architecture
Tokenization, Embedding, and Positional Encoding
The Attention Mechanism and Multi-Head Attention
Feed-Forward Networks, Residual Connections, and Layer Normalization
How the Encoder and Decoder work
How data flows through the entire architecture
The three variants of the Transformer
Why the Transformer is so powerful

Let's get started: Decoding Transformer Architecture

Part 2: Attention

Attention is the heart of the Transformer. In this part, we go from the intuition to the exact math, and then through every attention variant used in modern models.

Self Attention in Transformers

In this blog, we will learn about Self Attention in Transformers. We will understand what it is, how it works step by step, and why it is the heart of modern Large Language Models like BERT and GPT.

We will cover the following:

What is Self Attention?
Why do we need Self Attention?
Query, Key, and Value vectors
Step-by-step working of Self Attention
A simple example walk-through
Why Self Attention works so well
Multi-Head Self Attention
Where Self Attention is used

Let's get started: Self Attention in Transformers

Math behind Attention - Q, K, and V

In this blog, we will learn about the math behind Attention - Query(Q),

项目描述

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

transformers-explained