Deterministic vs Probabilistic Predictions

Some models output a hard label — "urgent" or "not urgent." Others output a score or probability — "72% likely urgent." That distinction matters because it changes how you set a decision threshold, whether you can trust the numbers as probabilities, and how you compare models. A hard label is a decision already made; a score is raw material for a decision you still get to make.

Hard labels vs scores

A model that outputs a hard class has already applied a threshold, usually 0.5 on an internal score. That is convenient but inflexible: you cannot adjust the threshold later without retraining or accessing the internal score. A model that outputs a score lets you choose the threshold at decision time, which matters when the cost of false positives and false negatives differs. The scikit-learn guide on classification threshold tuning explains this distinction (scikit-learn: tuning the decision threshold).

Scores are not always calibrated probabilities

A score that ranks cases — higher score means more likely positive — is useful even if the numbers are not true probabilities. But if you want to say "this case is 72% likely to be fraud," the scores must be calibrated: among cases scored 0.72, about 72% should actually be fraud. Many models produce well-ranked but poorly calibrated scores. Calibration can be improved with methods like Platt scaling or isotonic regression, but it must be checked, not assumed.

A worked example

Suppose a fraud model outputs a score from 0 to 1. At a 0.5 threshold, it flags 100 transactions, of which 30 are fraud — precision 30%. Lowering the threshold to 0.3 flags 200 transactions, of which 40 are fraud — precision 20% but higher recall. Raising it to 0.7 flags 40, of which 28 are fraud — precision 70% but lower recall. The score lets you trace this trade-off; a hard-label model would have hidden it. If the scores are calibrated, you can also say "cases scored above 0.7 are at least 70% likely fraud," which supports a cost-based decision.

Common mistakes

One mistake is treating a raw score as a calibrated probability without checking. Another is using a hard-label model where the threshold needs to change with business costs. A third is comparing models by their hard-label accuracy when their scores, evaluated across thresholds, would rank them differently.

An exercise

Take a model that outputs scores. Sweep the threshold from 0.1 to 0.9 and record precision and recall at each. Plot the trade-off. Then check calibration: bin the scores and compare the average predicted probability to the actual positive rate in each bin. For the broader learning path, see the Applied AI program previews or the data analysis program.