Features vs Labels: Thinking Clearly About Inputs and Targets

In a supervised project, the single most important distinction is between features (the inputs you use to make a prediction) and the label (the thing you predict). It sounds obvious, yet a large share of leakage and "too good to be true" results come from a feature that is, directly or indirectly, the label in disguise. Thinking clearly about this line at the start saves weeks of debugging later.

What counts as a feature

A feature is any value available at prediction time that you feed to the model. The key constraint is "available at prediction time": if you would not know the value when making a real prediction, it is not a usable feature. For a track-release prediction made before release, the tempo and the artist's prior streams are features; the track's post-release stream count is not, because that is the label (or a function of it).

What counts as the label

The label is the target you train the model to predict. It is derived from outcomes that occur after (or independently of) the features. In the streaming example, the label is "did the track exceed 10,000 streams in its first 14 days?" — an outcome known only after 14 days. The label exists in historical data for training, but at prediction time for a new release it is unknown.

A worked example: leakage from a label-derived feature

Suppose you add a feature called "14-day stream count" to predict whether a track exceeds 10,000 streams in 14 days. That feature is the label itself. The model will achieve near-perfect accuracy and be completely useless, because at prediction time for a new release you do not have the 14-day count yet. A subtler version: a feature that is a ratio or rank computed from the label. The fix is to audit every feature and ask, "would I know this value at the moment of prediction?" If the answer is no, the feature must be removed or recomputed from information available at prediction time.

Common mistakes

One mistake is including an ID that encodes the label — for example, a track ID whose historical performance the model memorizes. Another is using a timestamp feature that leaks the future, like "month of release" when releases in certain months systematically perform better and the model sees the test period. A third is normalizing or encoding using statistics computed on data that includes the label column. The scikit-learn pipeline documentation helps structure preprocessing so it is fit only on training data (scikit-learn: pipelines).

An exercise

List every feature in your project. For each, write down when you would know its value in a real prediction. Flag any whose availability depends on the outcome. Remove or redefine those. For the broader learning path, see the Applied AI program previews or the data analysis program.