Leakage often enters through a field or a step that looks harmless. A churn model is trained with a column that is filled in only after a customer has already cancelled. A fraud model uses a review flag that is set by the same investigation it is meant to predict. A preprocessing step computes averages over the whole dataset before it is split, so the test rows leak into training.
Common sources:
- target leakage: a feature that is recorded after, or because of, the outcome
- train-test contamination: the same records, or near copies, on both sides of the split
- preprocessing fitted on all data before the split, such as scaling or imputation
- time leakage: training on records from after the period the model is meant to predict
The usual sign is a result that is better than expected, followed by a drop once the model meets new data.
How is data leakage different from overfitting?
- Overfitting means the model learned patterns specific to the training data. It shows up as a gap between training and test scores.
- Leakage means the test result is no longer a fair measure. Training and test scores can both look good, because the answer reached the model through the data.
- Overfitting is addressed with model and training choices, or with more varied training data. Leakage is addressed by fixing the data, the features and the order of preparation steps.
Where does data leakage break in AI work?
A team compares two versions of a training set and the prepared version scores far higher. The jump is real on paper, but one of the added features was derived from a field that is only filled in after the outcome. Most of the gain came from the leak, not from a real improvement.
Ruling leakage out takes the preparation records: which fields were used, when they are filled in, and whether every learned transformation was fitted on training data alone. A before-and-after comparison on the same evaluation set shows that results changed, and the preparation record is what lets a team check why. Data quality for AI lists leakage among the problems rules-based checks may not catch without task-specific context.