Glossary · Evaluation & safety
Distribution Shift
A difference between the data distribution used to build or evaluate a system and the distribution it encounters after deployment.
Why it matters
A model can pass held-out tests yet fail when users, tasks, language, tools, or operating conditions change.
In practice
Define expected deployment slices, monitor performance and input characteristics by slice, and add new failures to a versioned eval set.
Common confusion
Distribution shift is not always model drift. The model may be unchanged while its environment or user population changes.
Related terms
Sources
Browse the learning paths to see this term in context — every lesson is free to read.