How to Turn a Vague AI Idea Into a Testable Project Question

Many applied AI projects start with a broad ambition: "we should use AI for this." That sentence is a starting point, not a plan. To make progress on a first supervised-learning project — where you have labelled examples and want to predict a label for new cases — you need to turn that ambition into a single, measurable question. The same sharpening instinct helps with other approaches, but the steps below assume you are working toward a prediction with a target you can measure.

Start with a decision, not the technology

A useful first question is not "what can AI do here?" but "what decision would a better prediction inform?" A model earns its place when its output could support a choice someone makes. Write down the decision in plain language: who makes it, when, and what they currently rely on. For example, "the support team lead decides which tickets to escalate first, currently based on a quick scan of the subject line" describes a decision. "Make support better with AI" does not. Even if your project is primarily a learning exercise, naming a plausible decision helps you choose a meaningful target instead of an arbitrary one.

Turn the decision into a prediction with a label

Once the decision is clear, restate it as a prediction problem. If the decision is "which tickets to escalate first," the prediction becomes "how urgent is each incoming ticket?" Now ask: could a human label that? One option is to use whether a ticket breached its target response window as a proxy for urgency. Be honest about what that proxy measures: a response-time breach is affected by staffing levels, existing queue priorities, and ticket complexity — not only by intrinsic urgency. It is a useful, labelable signal, but it is not an objective measure of urgency. That still gives you a concrete target variable: a column you can fill with yes/no or a category. A prediction you cannot label is a prediction you cannot train or evaluate, so this step narrows the scope more than any other.

A worked example: from broad idea to question

Suppose a small music label wants to "use AI on our catalogue." Ask what decision it could inform: "the release scheduler decides which tracks to promote each week." Restate as prediction: "which tracks will get the most streams in their first 14 days?" Can a human label that? Yes — past releases already have 14-day stream counts, so the label exists in historical data. The testable question becomes: "Given a track's audio features and the artist's prior performance, can we predict whether it will exceed 10,000 streams in its first 14 days?" That is one sentence with an input, an output, a threshold, and a time window — and it can be wrong. A caveat worth naming: 14-day stream counts depend on how the label promoted each past release and on which data was actually available at the time of release. Treat this as an illustrative example of how to frame a question, not as a model you can deploy without revisiting those confounders.

Common mistakes that keep the question vague

One mistake is falling in love with a technique before defining the problem — "we should use a neural network" is a solution hunting for a problem. Another is skipping the label: "predict customer satisfaction" sounds concrete until you realise nobody has a consistent satisfaction score to train on. A third is an unbounded time window: "predict churn" means little without "within the next 30 days." Each of these lets a project drift before anyone notices it was never answerable. A good question names the input, the output, the threshold, and the window. For a thorough treatment of how to frame a prediction problem and evaluate it honestly, the scikit-learn user guide on model evaluation is a reliable reference (scikit-learn: metrics and scoring).

An exercise to sharpen your own question

Take your current idea and fill in four blanks: "Given [inputs you actually have], can we predict whether [a specific, labelable outcome] will happen within [a defined time window] well enough to inform [a named decision]?" If any blank is empty or fuzzy, that is your next task — not modelling. Show the completed sentence to the person who makes the decision and ask whether a correct prediction would actually change what they do. If they say "not really," you have saved yourself effort on a model that would not have been used.

Your next step

Once the question is sharp, the next move is an inexpensive first check: a simple baseline. Before training anything, predict the outcome with a simple rule and measure it. That baseline tells you whether your eventual model is adding value — and it is the subject of the next piece in this series. If you want to see how this fits into a structured learning path, explore the Applied AI program previews or the data analysis program overview.