The Five Questions Every AI Program Should Be Able to Answer

We've sat in enough rooms to know the signature. The fix isn't more pilots. It's asking five questions early — questions that decide whether an AI program scales or stalls — and being honest about the answers.

These five questions map directly to the five phases of the Enterprise AI Operating Model. If you can't answer them crisply, the gap will tell you where to start.

1. Are you using AI because it's the best-fit technique — or because it's the technique getting headlines?

Generative AI is the loudest technology in the room. It is not always the right one. Pricing, scheduling, routing, allocation, anomaly detection — these are optimization or classical-ML problems, and forcing a generative model onto them produces slower, more expensive, less reliable systems that look modern in a screenshot.

Take your top three AI investments. For each one, ask: if we built this without a large language model, what would the system look like? If the answer is “actually, a rules engine plus a small classifier would handle it,” you didn't have an AI strategy. You had a vendor preference.

Best-fit technique discipline is what separates programs that ship from programs that demo.

2. What percentage of your AI investments are running in production with real users today?

Not deployed-to-an-internal-staging-environment. Not running-as-a-pilot-for-the-AI-team. Production. With users who have alternatives and noticed when it broke.

The math most mid-market leaders avoid is the ratio of announced AI initiatives to AI systems actually running in production. A 10-to-1 ratio is normal. A 20-to-1 ratio is common. A 50-to-1 ratio means you have a roadmap, not a program.

The next question — for every initiative that isn't in production yet — is why not, specifically. The honest answers are usually the same three: the data work was bigger than expected, integration into the rest of the business was deferred, or governance review never got scheduled. None of those are technology problems.

3. When you've brought in AI consulting help before, what did you actually get?

Did senior engineers embed with your team, or did junior associates run pre-built decks? Did the engagement leave you with working systems in production, or with a 60-page strategy document and a recommended next phase?

The mid-market has been getting the second answer from the Big Four for a decade. It's expensive theatre. The OpenAI Deployment Company just launched a $4 billion Forward-Deployed-Engineering operation for the F500 because the F500 figured out the same thing and is now buying the alternative. The middle market deserves the alternative at middle-market scale.

If the help you bought didn't ship code, you bought the wrong help.

4. Which phase of the Enterprise AI Operating Model is your organization actually in — and what's blocking the next one?

Orienteer AI's Operating Model names five phases: Vision (you have a strategy), Foundation (you have the data, talent, and platform), Scale (production systems are running and growing), Govern (you have observability, risk controls, and lifecycle discipline), and Adapt (the operating model itself learns).

Most mid-market organizations live somewhere between Foundation and Scale. Most enterprise leaders we meet think they're one phase ahead of where they actually are.

The diagnostic question isn't which phase do you wish you were in? It's what specifically is blocking the next one — and is that block a technology problem, an organization problem, or a leadership problem? Naming the block correctly is half the fix.

5. How many of your AI systems can you observe in production right now?

Token usage by model, by team. Output drift over the last 30 days. Error rates by endpoint. Latency tail. Cost per outcome. Security events touching the AI stack. The same telemetry you'd demand of any other production system.

If the answer is “the AI team has a notebook,” you're going to find out about your incidents from a customer.

Observability is the discipline that converts AI from an experiment into infrastructure. It's also the discipline most pilot programs skip — because the team isn't planning to run the system long enough to need it. The day a leadership team realizes they need to see what their AI is doing is the day the program transitions from project to system. The cheapest version of that transition starts before the system reaches production, not after.

What the answers mean

If your team can answer all five questions crisply, your AI program is in good shape and probably doesn't need us.

If two or more of them produce hedging, your program is in pilot purgatory — and a clearer strategy slide isn't going to fix it.

We embed senior engineers inside your operation to ship working AI systems in 30 to 90 days, with the operating-model and observability scaffolding that keep them running after we leave. Sized for the middle market — not the F500.