What is ROC Curve?

A fraud model gives every transaction a score. A low cutoff sends more transactions to review, catching more fraud along with more legitimate payments; a high cutoff shortens the queue and lets more fraud through. A ROC curve draws that trade-off for every cutoff at once.

The scikit-learn guide to model evaluation describes the ROC curve as a plot of the true positive rate against the false positive rate at various threshold settings. The area under the curve, ROC-AUC, summarizes the curve in one number. By the usual reading of the metric, 1.0 means the model ranks every positive case above every negative one, and 0.5 means its ranking is no better than chance.

A higher ROC-AUC does not tell you how the list changes at the cutoff a team actually uses. In CUBIG’s UCI Iranian Churn experiment (October 2026), logistic regression’s ROC-AUC rose from 0.9208 to 0.9399 after data preparation; at the 0.5 threshold, the model caught 78 of 99 churners instead of 42, while precision fell from 84.00% to 65.55%.

How to read a ROC curve:

• the vertical axis is the true positive rate, the share of actual positive cases the model flags
• the horizontal axis is the false positive rate, the share of actual negative cases flagged by mistake
• each point on the curve is one threshold; moving along the curve trades more catches for more false alarms
• the diagonal line is a model that ranks at random, and a curve bowing toward the top-left corner ranks better

How is a ROC curve different from a precision-recall curve?

• A ROC curve compares the true positive rate with the false positive rate, which is measured against all negative cases.
• A precision-recall curve compares precision and recall, so it shows what share of flagged cases are real.
• When positive cases are rare, many false positives can still look like a small false positive rate. A 2015 study in PLOS ONE by Saito and Rehmsmeier argues that precision-recall plots are more informative than ROC plots on imbalanced datasets.

Where does a ROC curve break in AI work?

A ROC curve breaks when ROC-AUC is reported alone. The number summarizes ranking across all thresholds, but a team acts at one threshold, and that threshold decides how long the review or contact list is and how many cases it misses.

A ROC comparison also holds only on the same evaluation set. Comparing two models, or two versions of training data, means scoring both on the same records and then reading the confusion matrix at one fixed threshold, ideally the one the business will use, as in how to evaluate a churn prediction model.