Glossary · Math & training
Gradient Clipping
Limiting gradient values or their combined norm before an optimizer update when they exceed a chosen threshold.
Why it matters
It can prevent an unusually large gradient from destabilizing a training step and producing non-finite values.
In practice
Log unclipped norms, clip after unscaling mixed-precision gradients, and investigate repeated clipping instead of treating it as a substitute for diagnosing instability.
Common confusion
Clipping controls update magnitude; it does not repair invalid data, a broken loss, or a consistently unsuitable learning rate.
Related terms
Sources
Browse the learning paths to see this term in context — every lesson is free to read.