A retention team can call 100 customers a week. Lowering the model’s threshold finds more of the customers who will leave, but more of the people on the list turn out to be staying. A precision-recall curve shows that trade-off at every threshold, so the team can see what each extra churner costs in wasted calls.
The scikit-learn example on precision-recall describes precision-recall as a useful measure of prediction success when the classes are very imbalanced. Average precision summarizes the curve as a weighted mean of the precision at each threshold, weighted by the increase in recall from the previous threshold.
In CUBIG’s UCI Iranian Churn experiment (October 2026), 99 of the 630 evaluation customers churned, about 16%. Logistic regression’s average precision rose from 0.7563 to 0.8144 after data preparation on one train/test split, while precision at the fixed 0.5 threshold fell from 0.8400 to 0.6555 as recall rose.
How to read a precision-recall curve:
- the horizontal axis is recall, the share of actual positive cases the model finds
- the vertical axis is precision, the share of flagged cases that are actually positive
- each point is one threshold; moving right usually finds more positives at the cost of lower precision
- a model that ranks at random has a precision close to the share of positive cases, so the baseline depends on how rare the positives are
How is a precision-recall curve different from a ROC curve?
- A ROC curve plots the true positive rate against the false positive rate, which is measured against all negative cases.
- A precision-recall curve replaces the false positive rate with precision, so it shows directly what share of flagged cases are wrong.
- When negatives far outnumber positives, a 2015 study in PLOS ONE by Saito and Rehmsmeier argues that precision-recall plots are more informative than ROC plots.
Where does a precision-recall curve break in AI work?
A precision-recall curve breaks when the class balance of the evaluation data differs from production. Precision depends on how common the positive class is, so a curve drawn on a test set that was balanced by resampling will promise more precision than the model delivers on real traffic. The evaluation records should keep the real-world share of positives.
A single average precision number also hides the operating point. Two models with similar average precision can behave differently at the threshold a team actually uses, so the curve should be read next to the confusion matrix at that threshold. In CUBIG’s Syntitan, Ready for Model reports PR-AUC with F1, recall and precision for binary classification tasks; the value shown as PR-AUC is calculated as average precision, as described in how to evaluate a churn prediction model.