Source directory

The best existing material on each concept — original papers, visual explainers, canonical blog deep-dives, and lectures — collected alongside the concept pages. 181 sources across 40 concepts and 15 candidate topics. Topics marked interactive exists already have an excellent interactive explainer; this site links to it rather than building its own.

Primitives

attention interactive exists

Lets every position look at every other position and mix in what's relevant. The breakthrough that replaced sequential processing with direct connections between any two positions.

feed-forward networks (MLP)

A small per-position neural net applied identically at every token position; where transformers store much of their factual knowledge and the natural target for sparsity in mixture-of-experts designs.

tokenization interactive exists

How raw text becomes the integer IDs models actually see — BPE, WordPiece, SentencePiece — and why tokenizer quirks explain so many otherwise-baffling LLM failures.

embeddings

Dense vectors that represent discrete tokens (words, image patches, users) as points in a continuous space, where geometric closeness reflects semantic similarity.

layer normalization

Rescales each token's activations to zero mean and unit variance at every layer, keeping training stable as networks grow deep.

residual connections

Adds a layer's input back to its output — y = x + F(x) — giving gradients a direct highway through deep networks and making 100-layer stacks trainable.

positional encoding

Injects token-order information into a transformer, because attention alone treats its input as an unordered set and cannot distinguish "the dog bit the man" from "the man bit the dog."

expert routing

Learned switches that decide what flows where — the gates controlling information inside an LSTM cell, and the router picking which experts process each token in a mixture-of-experts model.

Modules

transformer block interactive exists

The repeating unit of every transformer — attention then feed-forward, each wrapped in a residual connection and normalization. Stack N of them and you have an encoder or a decoder.

encoder

Reads the whole input at once — bidirectionally — and produces a contextual representation of it. The "understanding" half of the transformer recipe.

decoder

Generates output one token at a time, each step attending only to what came before — the "generation" half of the transformer recipe.

predictor

A lightweight module that predicts the representation of hidden content from visible context — the defining piece of JEPA and latent world models.

diffusion transformer (DiT) interactive exists

A transformer backbone used as the denoiser inside a diffusion model — the architecture that replaced the convolutional U-Net in nearly every frontier image and video generation system.

output head interactive exists

The final projection from hidden vectors to actual predictions — logits over a vocabulary, class scores, values, or actions.

Modifiers

Mixture of Experts (MoE)

Swaps the dense feed-forward block for many routed "expert" MLPs — only a few fire per token, giving a larger model for the same inference cost.

attention head sharing (GQA/MLA)

Shares key/value projections across query heads to shrink the KV cache at inference — a modifier on the attention sublayer, not a peer architecture.

Families

large language models (LLMs) interactive exists

A large model trained to predict the next token; the dominant form stacks decoder blocks at enormous scale (GPT, Claude, Llama). "LLM" is a category, not one architecture — the invariant is the autoregressive objective applied at scale.

BERT

A stack of bidirectional encoder blocks trained to fill in masked words — built to understand text, not to generate it. The architecture that cemented the pretrain-then-fine-tune paradigm in NLP.

encoder–decoder (seq2seq) interactive exists

An encoder reads the source sequence; a decoder generates the target while cross-attending to it — the original Transformer wiring for translation, summarization, and T5/BART. Decoding is autoregressive, but pretraining objectives vary.

Vision Transformer (ViT)

Chops an image into patches, embeds each like a token, and runs a standard encoder over them. The training objective is not fixed — ViT is a backbone used for supervised classification, masked autoencoding, and contrastive learning alike.

convolutional neural networks (CNNs) interactive exists

Stacks of sliding learned filters that detect local patterns and compose them into global understanding — the vision default before ViT (ResNet, EfficientNet). The training objective is not fixed; the same backbone is used for supervised, self-supervised, and contrastive learning.

recurrent neural networks (RNN/LSTM)

Processes sequences one step at a time, carrying a hidden state forward as a running memory; LSTM adds gates so memory survives long sequences. The pre-transformer default for language — and a backbone whose training objective is not fixed.

diffusion models interactive exists

Generate by reversing noise — start from static and repeatedly denoise until an image (or video) emerges. The denoiser inside is swappable, and flow matching is the straight-line successor that trains most modern systems.

generative adversarial networks (GANs) interactive exists

A generator forges samples while a discriminator learns to spot fakes; trained against each other until the forgeries get good (StyleGAN). Introduced by Goodfellow et al. in 2014.

GAN Lab (Polo Club of Data Science) explainer
Generative Adversarial Networks (Goodfellow et al., 2014) paper
From GAN to WGAN (Lilian Weng, 2017) blog

variational autoencoders (VAEs) interactive exists

An autoencoder whose bottleneck is a probability distribution — rebuild the input, but keep the latent space smooth enough to sample new data from (ELBO = reconstruction + KL). Introduced by Kingma & Welling in 2014.

CLIP (dual-encoder) interactive exists

Two separate encoders — one for images, one for text — trained so matching pairs land close together in a shared embedding space. Introduced by OpenAI (Radford et al., 2021).

autoencoders interactive exists

Encoder squeezes the input through a bottleneck; decoder rebuilds it. The bottleneck is forced to learn what matters — and is the parent of VAEs and sparse autoencoders.

VQ-VAE

An autoencoder with a discrete codebook bottleneck — images become grids of tokens, which is what lets transformers generate them (VQ-VAE, VQGAN, DALL-E 1).

JEPA (joint-embedding predictive architecture)

Predict the representation of hidden content rather than its pixels — LeCun's alternative to generative pretraining, using an encoder plus a predictor to learn in latent space (I-JEPA, V-JEPA).

masked image modeling (MAE)

BERT's fill-in-the-blank idea applied to pixels — hide most of an image's patches and train a vision transformer to reconstruct them. A simple, scalable way to pretrain vision encoders without labels (MAE, BEiT).

state space models (SSM/Mamba) interactive exists

Replaces attention with a selective state-space scan — recurrent memory you can train in parallel, sub-quadratic in sequence length. Mostly used autoregressively (Mamba, S4). The objective is not fixed.

graph neural networks (GNNs) interactive exists

Neural networks that operate on graph-structured data by passing messages between connected nodes. The objective varies — node classification, link prediction, or graph classification — making GNNs a backbone, not a single model.

Perceiver

Cross-attend a huge input into a small fixed-size latent array, then run the transformer there — one architecture for images, audio, point clouds, anything (Perceiver, Perceiver IO).

flow matching

A generative training objective that learns a velocity field transporting noise to data along straight paths — fewer sampling steps than diffusion, same backbone architectures (SD3, Flux).

rectified flow

Straighten the noise-to-data paths a flow model learns until sampling takes a handful of ODE steps — or just one. The flow-matching formulation behind Stable Diffusion 3.

energy-based models interactive exists

A framework that assigns a scalar energy to each configuration — low for plausible data, high for everything else. Generation and inference become search: descent on the energy landscape.

vision-language models (VLMs)

A composition of a vision encoder and an LLM decoder — not a new wiring, but a projector-bridged assembly that lets a language model process images alongside text (LLaVA, Flamingo).

vision-language-action models (VLAs)

A VLM extended for robotics — the model sees the scene, reads the instruction, and emits motor actions as tokens, inheriting web-scale knowledge without changing the architecture (RT-2, π0).

retrieval-augmented architectures (RAG)

Bolt a search step onto a generator — fetch relevant documents at inference time and condition on them (RAG, RETRO) instead of memorizing everything in weights.

world models interactive exists

A learned model of an environment's dynamics that lets an agent predict what happens next and plan ahead. A goal, not one wiring — the architecture and objective vary significantly across implementations.

Second ring — candidates

Topics queued for their own page. Their sources are collected ahead of time, so each page can be written link-first — and where an interactive explainer already exists, the page points there instead of this site building one.

denoiser no page yet

The network diffusion and flow models train to undo corruption — a convolutional U-Net in the first wave, now usually a transformer (DiT).

page-debt: already a map node and builder chip, but unclickable — write the page, set the module's slug, delete this entry

generator + discriminator no page yet interactive exists

The adversarial pair: a generator forging samples, a discriminator judging real from fake, each trained against the other.

page-debt: map node + builder chip with no page; GAN Lab covers the training dynamics interactively — link, don't rebuild

Generative Adversarial Networks (Goodfellow et al., 2014) paper
GAN Lab (Polo Club of Data Science) explainer
From GAN to WGAN (Lilian Weng, 2017) blog

U-Net no page yet

Encoder–decoder convolutions with skip connections at every scale — the denoiser of the first diffusion wave, still the default in segmentation.

currently a slugless denoiser variant on the map — a page gives that variant its missing link target

LoRA no page yet

Freeze the weights, learn low-rank deltas on top — fine-tune a huge model by training under 1% of its parameters.

training-efficiency trio; modifier-shaped like MoE/GQA (changes how primitives run, not which ones)

quantization no page yet

Store weights in fewer bits (8, 4, even 2) — most of the quality at a fraction of the memory.

training-efficiency trio; Grootendorst's visual guide is near-definitive — link, don't write deep

distillation no page yet

Train a small student to match a big teacher's output distribution, not just the labels — keeping most of the capability at a fraction of the size.

training-efficiency trio; Hinton 2015 plus DistilBERT as the worked example