Build a Simple Baseline Before Training a Machine-learning Model

Before you train a model, answer one question: what is the simplest prediction that still does a reasonable job? That simple prediction is your baseline (modèle de référence), and it is one of the most important numbers in your project. A model that beats the baseline may be worth shipping; a model that fails to beat it may just be an expensive way of doing what a one-line rule already does. Building a baseline first keeps you from spending weeks on a model that turns out to add little over a trivial rule.

Why a baseline matters more than a clever model

A baseline anchors every later claim. When you say "our model achieves 88% accuracy," the number means little in isolation. If the baseline — always predicting the most common class — already gets 86%, your model's real contribution is two points, not eighty-eight. The baseline converts a vanity number into a more honest one. It also tells you where the ceiling of "easy" is, which keeps you from celebrating a result that required no intelligence to produce. Whether a two-point gain is worth deploying depends on the cost of errors and the uncertainty around the estimate, not on the headline number alone.

Three baselines worth knowing

For classification, the simplest baseline is the majority class: always predict whichever category is most common. If 70% of support tickets are non-urgent, "always predict non-urgent" gets 70% accuracy for free. For regression, two simple baselines use the training target's centre: the mean minimizes mean squared error, while the median minimizes mean absolute error. If the average 14-day stream count is 6,200, predicting 6,200 for every track gives a mean-squared-error baseline to beat; predicting the training median gives a mean-absolute-error baseline. A slightly stronger baseline is the "group mean": predict the average for the track's genre. These take minutes to compute and immediately expose whether your features add information. Always derive the baseline from the training data only, so the comparison is honest.

A worked example with consistent counts

Imagine 200 support tickets labelled urgent or not, with 40 urgent (20%) and 160 non-urgent (80%). The majority-class baseline predicts "non-urgent" for all 200 and scores 80% accuracy. Now you train a logistic regression and it reaches 84% accuracy. Without the baseline, 84% sounds impressive; with it, you see the gain is four points. Look closer: of the 40 truly urgent tickets, suppose the model caught 12 and missed 28. The counts must be consistent: 12 true positives, 28 false negatives, and the remaining 160 non-urgent predictions split into 156 true negatives and 4 false positives (156 + 4 = 160), which checks out (12 + 28 + 156 + 4 = 200). The baseline caught zero urgent tickets. So while overall accuracy barely moved, the model found 12 urgent cases the baseline could never find — which may matter far more than the headline number, depending on what a missed urgent ticket costs.

Common mistakes with baselines

The first mistake is computing the baseline on the wrong split — if you calculate "most common class" on the full dataset including the test set, you have leaked information and your comparison is dishonest. Always derive the baseline from the training data only. The second is choosing a baseline so weak it is uninformative: a random-label baseline can still be diagnostic — it tells you whether your evaluation harness and metric behave as expected — but it is rarely a fair yardstick for a real model. The third, and most common, is never building one at all and then presenting a model's raw accuracy as if it were meaningful. The official scikit-learn train_test_split documentation is a good reference for splitting data cleanly before you compute any baseline or model score (scikit-learn: train_test_split).

An exercise: compare baseline and model on validation

Pick a dataset with a clear target. Split off a test set first and set it aside. From the remainder, split off a validation set. Compute the majority-class (or training-median) baseline and record its score on the validation set. Now write the simplest non-trivial rule you can think of using one feature — for example, "predict urgent if the ticket subject contains the word 'broken'" — and measure that on validation too. Train a model and compare it to both on validation while you iterate. Only once you are satisfied should you run the chosen model once on the held-out test set for a final comparison. The exercise is about learning to ask, every time, "did I actually beat the simple version, and by enough to matter given the cost of errors?"

Your next step

Once you have a baseline and a model that beats it honestly, the next question is whether your test score is trustworthy — which depends entirely on how you split your data. That is the subject of the next article, on training, validation and test sets without data leakage. To see how baselines fit into a full applied AI workflow, browse the Applied AI program previews or the data analysis program.