Read a Confusion Matrix With an Imbalanced-Class Example

Accuracy is the first metric everyone learns, and it is one of the first that misleads. The classic trap is the imbalanced dataset: if only 5% of your examples belong to the class you care about, a model that predicts "no" every time scores 95% accuracy and may be useless for its purpose. The confusion matrix is the small table that exposes this by showing exactly what your model got right and wrong, broken down by class. Once you can read one, a bare accuracy number tells you far less than it did before.

What a confusion matrix actually contains

For a binary problem, the matrix is a two-by-two grid. The rows are the true labels, the columns are the predicted labels, and each cell counts examples. True negatives sit top-left: the model said "no" and the truth was "no." True positives sit bottom-right: the model said "yes" and the truth was "yes." The off-diagonal cells are the mistakes: false positives (predicted "yes," truly "no") and false negatives (predicted "no," truly "yes"). Accuracy is the sum of the diagonal divided by everything. The matrix matters because it keeps the two kinds of mistakes separate, and in the real world those two mistakes almost always cost different amounts.

A worked example: when accuracy misleads

Imagine a fraud detector on 1,000 transactions, where 50 are fraudulent and 950 are legitimate. A model that predicts "legitimate" for every transaction produces this matrix: 950 true negatives, 0 true positives, 0 false positives, and 50 false negatives. Accuracy is 950 out of 1,000, or 95% — a number that looks strong until you realise the model caught zero fraud. The confusion matrix makes the failure obvious: the true-positive cell is empty. This is why accuracy alone is a weak summary on imbalanced data — it can reward a model for ignoring the rare class entirely. The official scikit-learn confusion_matrix documentation shows the exact layout and a worked call (scikit-learn: confusion_matrix).

The two questions the matrix forces you to ask

Once you see the four cells, two derived questions become unavoidable. Recall asks: of all the real fraud cases, how many did we catch? In the example, that is 0 out of 50, or 0% — the signal that the model is useless for its purpose. Precision asks: of all the cases we flagged as fraud, how many really were? Here it is undefined (we flagged none), but in a model that flags 30 cases of which 20 are real, precision is 20 out of 30, or about 67%. Recall and precision pull against each other: you can often raise recall by flagging more cases, but that usually lowers precision because you flag more innocent transactions. The confusion matrix is what lets you see that trade-off instead of guessing at it.

How lowering the threshold actually moves the cells

Many classifiers output a score rather than a hard label; you choose a threshold to convert scores into yes/no decisions. On fixed scores and data, lowering the threshold means positive predictions are non-decreasing: false negatives may drop or stay the same, and false positives may rise or stay the same. Recall is non-decreasing when defined, and the false-positive rate is non-decreasing. But accuracy is not guaranteed to fall — it may increase, decrease, or stay the same depending on the class balance and where the threshold sits. Precision is not guaranteed to be monotonic either. The scores themselves are not necessarily calibrated probabilities; they rank cases but may not correspond to true likelihoods. The official scikit-learn guide on tuning the decision threshold explains the distinction between the statistical problem of predicting scores and the decision problem of acting on them (scikit-learn: tuning the decision threshold).

Common mistakes when reading the matrix

One mistake is reporting accuracy on an imbalanced problem without ever looking at the matrix. Another is confusing which axis is which: some libraries put predictions on rows and truth on columns, so always check the labels before interpreting. A third is ignoring the cost asymmetry: in fraud, a false negative (missed fraud) may cost thousands, while a false positive (a flagged legitimate transaction) may delay or block a valid activity and cost a manual review. The "best" threshold depends on those costs, not on whichever metric happens to look good. A model with 40% recall but 99% precision may be more useful than one with 90% recall and 10% precision, depending on what a false alarm costs your team.

An exercise: find the hidden failure

Take any classifier on an imbalanced dataset and print its confusion matrix. Compute accuracy, then compute recall for the minority class. If accuracy is high but minority-class recall is near zero, you have found a model that looks successful while failing its actual purpose. Then try lowering the decision threshold — flag a case as fraud at a lower score — and watch how the matrix shifts: false negatives may drop, false positives may rise, and accuracy may move in either direction. Decide, given the real costs, which version you would actually ship.

Your next step

Reading a confusion matrix is the foundation; the next step is choosing the right metric when false positives and false negatives cost different things — precision, recall and F1. That is the subject of a later article in this series. For the broader learning path, browse the Applied AI program previews or the data analysis program.