What is Confusion Matrix?

A confusion matrix is a table that counts how a classification model's predictions line up with the actual outcomes, showing true positives, false positives, false negatives and true negatives, so a team can see which kinds of errors the model makes instead of reading a single accuracy score.

Confusion Matrix in Machine Learning: Definition

A confusion matrix turns one score into four counts. For a churn model, the counts answer four questions: how many leaving customers were flagged, how many were missed, how many staying customers were flagged by mistake, and how many were correctly left alone.

CUBIG’s UCI Iranian Churn experiment (October 2026) reports the matrix for logistic regression on 630 evaluation customers, 99 of whom churned. Before data preparation, the model identified 42 churners, missed 57, flagged 8 non-churners and correctly classified 523. After preparation, with the same model settings and the same evaluation records, the counts were 78, 21, 41 and 490.

The four cells:

true positive: the model flagged the case, and the outcome happened
false positive: the model flagged the case, and the outcome did not happen (a false alarm)
false negative: the model did not flag the case, and the outcome happened (a miss)
true negative: the model did not flag the case, and the outcome did not happen

Many common classification metrics are built from these counts, including precision and recall and the F1 score.

How is a confusion matrix different from accuracy?

Accuracy is one number: the share of all predictions that were correct.
A confusion matrix keeps the errors apart, so a team can see whether the model is missing cases or raising false alarms.
When one outcome is rare, accuracy can stay high while most of the rare cases are missed. In the Iranian Churn experiment, accuracy was 0.8968 while the model missed 57 of 99 churners, and it moved only to 0.9016 when correctly identified churners rose from 42 to 78.

Where does a confusion matrix break in AI work?

A confusion matrix describes one threshold on one set of evaluation records. Change the decision threshold and the four counts change. Score the model on a different test set and the counts are no longer comparable with earlier ones.

Comparing two matrices is meaningful only when the model is scored on the same evaluation set at the same threshold. In Syntitan, CUBIG’s AI-ready data platform, Ready for Model compares results before and after data preparation under the same evaluation conditions, which is why the two Iranian Churn matrices can be read side by side. Whether 41 false alarms is acceptable for 78 identified churners depends on what a missed churner and an unneeded retention call cost the business.

Frequently asked questions

What are the four parts of a confusion matrix?

True positives, false positives, false negatives and true negatives. Together they show which predictions were correct and which kinds of errors the model made.

Why use a confusion matrix instead of accuracy?

Accuracy merges every error into one number. A confusion matrix shows misses and false alarms separately, which matters when one outcome is rare or when the two errors have different costs.

Does a confusion matrix change with the threshold?

Yes. A lower threshold usually flags more cases, which tends to raise true positives and false positives together, so a matrix should always be read with the threshold that produced it.