Glossary · Math & training
Activation Checkpointing
A training-memory technique that saves only selected forward-pass activations and recomputes the omitted ones during backpropagation.
Why it matters
It lets you train larger models or sequences within a fixed memory budget by trading additional computation for lower activation storage.
In practice
Checkpoint the memory-heavy transformer blocks, measure the extra step time, and keep recovery checkpoints separate from activation-recomputation settings.
Common confusion
Activation checkpointing is not a durable training checkpoint. It helps one forward and backward pass fit in memory but cannot resume a crashed run.
Related terms
Sources
Browse the learning paths to see this term in context — every lesson is free to read.