A data outlier can be a typo, such as a price entered with three extra zeros, or the most important record in the file, such as one fraudulent transfer among thousands of normal ones. Outlier detection finds the candidates; deciding which kind each one is takes knowledge of the process that produced the data.
Rare cases are often the point. In CUBIG’s UCI Shill Bidding experiment (October 2026), 675 of the 6,321 input records, about 10.7%, belong to the positive class the classifier is trained to find. The experiment did not test outlier removal, but a cleaning rule that drops unusual records should be checked against a class like this before it runs.
The scikit-learn guide to novelty and outlier detection describes outlier detection as unsupervised anomaly detection: the training data contains outliers, and the method tries to fit the regions where the data is most concentrated, ignoring the deviant observations. The same guide notes that outlier detection in high dimensions, or without assumptions about the distribution of normal data, is very challenging.
Common methods:
- statistical rules, such as flagging values more than a set number of standard deviations from the mean, or outside the interquartile range
- distance and density methods, such as Local Outlier Factor, which flag points with much lower local density than their neighbors
- isolation-based methods, such as Isolation Forest, which flag points that are easy to separate from the rest
- business rules, such as a negative quantity or an order dated before the account existed
How is outlier detection different from anomaly detection?
- Outlier detection usually refers to finding unusual points inside a dataset that may already contain them.
- Anomaly detection is the broader task of spotting abnormal observations or behavior, including in new data as it arrives.
- In practice the terms overlap, and scikit-learn treats outlier detection as one form of anomaly detection.
Where does outlier detection break in AI work?
Outlier detection breaks when every flagged point is removed by default. Clipping or deleting extreme values can make a dataset look cleaner and a quality score look better, while removing the rare cases a fraud, failure or churn model exists to catch.
A safer routine records each flagged group, the decision made about it (corrected, kept or removed), the reason, and who made that decision. Keeping the original values lets a team compare a model trained with and without the treatment on the same evaluation set, and see what the choice cost on the rare cases, not only on the overall score.