Glossary · Evaluation & safety
Benchmark Contamination
Overlap or information leakage between evaluation examples and data used to pretrain, tune, prompt, select, or otherwise improve the evaluated system.
Why it matters
Contamination can make a benchmark score reflect prior exposure rather than the ability to generalize to unseen tasks.
In practice
Track dataset provenance, search training sources for exact and near duplicates, hold back private test cases, and refresh public evals with newly authored examples.
Common confusion
Contamination is broader than exact copying. Paraphrases, answer keys, benchmark metadata, and repeated prompt tuning can also leak evaluation information.
Related terms
Sources
Browse the learning paths to see this term in context — every lesson is free to read.