ROC curve and AUC explained: formulas, example, and limits

A ROC curve shows how a binary classifier trades true positive rate against false positive rate as its threshold moves. AUC compresses that ranking behavior into one number, but it does not choose the operating threshold for you or replace precision–recall analysis when the positive class is rare.

What are a ROC curve and AUC?

A receiver operating characteristic (ROC) curve plots the true positive rate (TPR) against the false positive rate (FPR) for every classification threshold. The area under the ROC curve (AUC) is the area beneath that plot, usually between 0 and 1 for a useful score ranking.

The curve evaluates a model’s ordering of examples before you commit to one threshold. That makes it useful for comparing whether positive cases tend to receive higher scores than negative cases. It does not tell you whether a threshold is operationally affordable, whether predicted probabilities are calibrated, or how many positive alerts your team will actually receive.

What do the ROC axes measure?

Start with the four cells of a confusion matrix. At a chosen threshold, scores at or above the threshold become positive predictions and scores below it become negative predictions. The confusion matrix guide explains those cells in detail; ROC analysis uses two rates derived from them:

TPR answers, “Of the actual positives, how many did the model catch?” FPR answers, “Of the actual negatives, how many did the model incorrectly flag?” A point near the top-left corner has high TPR and low FPR. The diagonal from (0, 0) to (1, 1) represents random ranking: increasing the threshold policy along that diagonal catches positives at about the same rate as it flags negatives.

Each threshold produces one point. A very high threshold may predict almost nothing as positive, giving TPR and FPR near zero. A very low threshold may predict almost everything as positive, giving TPR and FPR near one. The curve connects the points as the threshold sweeps from high to low.

How do you calculate ROC-AUC by hand?

Use six cases with three actual positives and three actual negatives. The labels and scores are:

Sort by score from highest to lowest. Starting with a threshold above 0.90, the classifier predicts everything negative, so the curve begins at FPR 0 and TPR 0. As each score enters the positive prediction set, update TP or FP and divide by the number of actual positives or negatives:

The ROC points are therefore (0, 0), (0, 1/3), (0, 2/3), (1/3, 2/3), (1/3, 1), (2/3, 1), and (1, 1). Calculate the area with trapezoids. Vertical moves have zero width, so only the horizontal segments contribute:

The ROC-AUC is approximately 0.889. You can reach the same result another way: choose one positive and one negative at random. The positive scores beat all three negative scores for the first two positives, and the 0.40 positive beats two of the three negatives. That is 3 + 3 + 2 = 8 winning pairs out of 9, or 8/9. If scores tie, the pairwise interpretation conventionally gives a tied pair half credit.

How should you interpret AUC?

AUC is a ranking score. AUC 0.5 means the ranking is no better than random under the usual interpretation. AUC 1 means every positive receives a higher score than every negative, assuming no ties. AUC below 0.5 can mean the score direction is reversed; negating the score would reverse the ranking, though the model’s usefulness still depends on whether that reversal is valid and stable.

AUC does not mean “the model is correct 88.9% of the time” in the worked example. It means the score ranking wins about 88.9% of randomly selected positive-versus-negative comparisons. It also does not tell you whether scores of 0.8 represent an 80% probability. Calibration is a separate question.

Use AUC to compare ranking quality across thresholds or to assess whether a model separates classes at all. Then choose a threshold using the cost of FP and FN, required recall or specificity, alert capacity, and the prevalence of positives in deployment. A model with good AUC can still be unusable at the threshold your operation can support.

When is precision–recall analysis better than ROC-AUC?

ROC-AUC treats the false positive rate as FP divided by all actual negatives. When negatives are extremely common, that denominator can be very large. A model may produce many false positives while the FPR still looks small. The same false positives can make a positive alert queue mostly wrong, which is visible in precision.

Precision–recall analysis focuses on precision and recall: among predicted positives, how many are correct, and among actual positives, how many are found. It is often more informative when the positive class is rare and the practical question is whether an alert stream is worth reviewing. The condition matters: if the classes are reasonably balanced or the cost is explicitly tied to FPR, ROC analysis remains useful.

Always report the positive prevalence with the curve or AUC. Precision depends on prevalence even when the model’s TPR and FPR stay the same. This is why a model tested on a balanced sample can produce a different precision in production. For a direct count-based view, use the confusion matrix calculator at the threshold you expect to deploy.

How can ROC-AUC analysis go wrong?

Choosing a threshold from AUC alone. AUC summarizes all thresholds; it does not select one. Pick the operating point from the error costs and capacity of the real system, then inspect its confusion matrix.

Using accuracy as a substitute. Accuracy depends on one threshold and on class prevalence. AUC answers a ranking question. They are not interchangeable, and a model can have similar AUC to another model but very different accuracy at the chosen threshold.

Ignoring the evaluation split. AUC can look excellent when near-duplicate records, future features, or user identifiers leak across training and test data. Keep all preprocessing and threshold selection inside the training/validation process, and evaluate once on an untouched test set.

Comparing AUCs without uncertainty. AUC is an estimate from finite data. The difference between 0.81 and 0.83 may be noise, especially with few positive cases or correlated predictions. Use repeated splits or confidence intervals when the decision depends on a small difference.

Forgetting ties and score direction. Tied scores need a consistent convention, and the curve assumes higher scores mean more positive evidence. A reversed score can produce AUC below 0.5 even when the underlying ordering contains signal.

Using ROC-AUC when the product needs calibrated probabilities. AUC only cares about ordering. If the system must say “there is a 20% chance,” evaluate calibration as well as discrimination, and check reliability in the population where the model will operate.

What edge cases should you document?

ROC-AUC requires at least one positive and one negative actual label. If an evaluation slice contains only one class, TPR or FPR has no meaningful denominator and AUC is undefined. Report the support instead of forcing a value.

For multilabel or multiclass problems, decide whether you want one-vs-rest curves per class, macro averaging, micro averaging, or a weighted summary. A single “multiclass AUC” can hide important differences between classes unless the averaging method is stated.

When scores are identical for many cases, the curve may have larger jumps than expected. That is not a bug; it reflects limited ranking resolution. If a model emits only a few discrete scores, evaluate the actual threshold options rather than implying that every point along a smooth curve is deployable.

Use the calculator and compare related metrics

Try the ROC curve and AUC calculator with the six-case example above, then replace it with a representative holdout set. The calculator lists each unique threshold, TP, FP, TPR, and FPR so the AUC is auditable rather than a black-box number.

For threshold-level counts, read confusion matrix explained. For precision, recall, and F1 trade-offs, continue to accuracy vs precision vs recall vs F1. For numeric predictions rather than classes, use the companion MAE vs MSE vs RMSE vs R² guide.

FAQ