Labels are often produced by people and processes that were not designed for training a model. A category is chosen from a dropdown at the end of a busy shift. An outcome field is updated in one system and not in another. A rule for what counts as fraud changes halfway through the year, and older records keep the old meaning.
Where it comes from:
- manual labelling under time pressure or with unclear instructions
- outcome fields that are updated late or not at all
- definitions that changed over time while the history kept the old labels
- labels merged from sources that used different categories
How is label noise different from feature noise?
- Feature noise is error in the inputs, such as a mistyped amount or a wrong date. It blurs the signal the model uses.
- Label noise is error in the answers. It teaches the model a wrong mapping and also distorts the scores, because the evaluation labels may be wrong too.
- Feature noise is often found with validation rules. Label noise can be flagged by comparing sources or model confidence, but confirming it takes a review by the people who own the outcome.
Where does label noise break in AI work?
A claims model is trained on five years of outcomes. In year three, the business changed how partial payments were recorded, and from then on those claims were labelled approved instead of partially approved. The model learns two meanings for one label, and its scores on recent claims drift for reasons nobody can see in the code.
Catching it takes a sample review with the outcome owners and a record of when label definitions changed. In Syntitan, the consistency axis in Ready for Raw Data asks whether the same code carries the same meaning across sources and time, which applies to label values too, and the ground truth the evaluation set uses has to be checked the same way.