What are exit criteria for an AI pilot?

Measurable conditions agreed before the pilot starts that decide whether it scales: value, quality on a test set, safe failure, cost, speed, permissions, a path to production and adoption.

How big should an AI test set be?

A common starting point is 100 or more real cases, including hard and unusual ones, with good answers agreed by the people who do the work. Keep it fixed so results are comparable sprint to sprint.

Is stopping a pilot a failure?

No. Stopping a pilot that doesn't meet its criteria saves the cost of scaling something that won't deliver, and what you learned (and the test set) carries into the next one.

AI pilot exit criteria: a go/no-go checklist (free template)

By Rafael · Updated October 22, 2026 · 3 min read

The short answer

Exit criteria are the measurable conditions, agreed before the first sprint, that decide whether an AI pilot goes to production: value against a baseline, quality on a fixed test set of real cases, safe failure, cost per task, speed, permissions and data, a named path to production, and adoption. If any isn't met at the gate, you adjust and re-test, or stop.

Why exit criteria matter

Without criteria agreed up front, a pilot ends with a demo and a feeling. The demo uses good examples, everyone is impressed, and then nothing scales because no one can say whether it's good enough. Criteria turn the end of the pilot into a decision.

The eight criteria

Value: the metric the owner cares about moves from a measured baseline to an agreed target.
Quality on a fixed test set: 100+ real cases, including hard ones, rated by the people who do the work. The same set every sprint.
Safe failure: when it's unsure or out of scope, it says so and hands off to a person.
Cost per task: from real usage, not list prices, and below today's cost.
Speed: fast enough for the workflow it sits in.
Permissions and data: people only see what they're allowed to, and sensitive data is handled per policy, signed off by security or privacy.
Path to production: a named system it runs in and a team that supports it.
Adoption: the people it's for choose to use it.

Leave model choice and screen polish off the list. They matter less than the test set, which is the asset you keep whatever you build on.

Free go/no-go checklist

Copy this into your plan and fill in the brackets with the business owner before you start.

AI pilot exit criteria checklist

AI pilot exit criteria (go/no-go)
Agree these with the business owner BEFORE the first sprint. The pilot scales only when every one is met.

Use case: [what the AI does, for whom]
Business owner: [name]          Gate review date: [date]

1. Value
   Metric: [e.g. time to first reply]  Baseline: [measured]  Target: [number]
   Met? [ ]

2. Quality on a fixed test set
   Test set: [100+ real cases, including hard and unusual ones]
   Target: [e.g. ≥ 90% rated acceptable by the people who do the work]
   Met? [ ]

3. Safe failure
   When unsure or out of scope, it says so and hands off to a person.
   Tested with tricky, out-of-scope and adversarial cases.
   Met? [ ]

4. Cost per task
   Target: [below today's cost of doing it by hand], measured from real usage.
   Met? [ ]

5. Speed
   Target: [fast enough for the workflow, e.g. under 10 seconds]
   Met? [ ]

6. Permissions and data
   People only see what they're allowed to; sensitive data handled per policy.
   Security/privacy review signed off by: [name]
   Met? [ ]

7. Path to production
   Runs in: [system people already use]   Supported by: [team]
   Met? [ ]

8. Adoption
   Target: [e.g. 4 of 5 pilot users choose to use it weekly]
   Met? [ ]

Decision: [ ] Go   [ ] Adjust and re-test by [date]   [ ] Stop
Signed: [owner], [sponsor]

Want it done for you? Pilotwise drafts exit criteria for your use case, tracks each one through the pilot, and prints a one-page brief for your sponsor.

Running the gate review

Share the numbers firstTest-set results, cost per task, usage and the value metric against baseline, before anyone sees a demo.
Decide against the criteriaGo if all are met. Adjust and re-test by a date if one or two are close. Stop if the core value isn't there.
Write the decision downWho decided, on what evidence, and what happens next.

Common questions

What are exit criteria for an AI pilot?: Measurable conditions agreed before the pilot starts that decide whether it scales: value, quality on a test set, safe failure, cost, speed, permissions, a path to production and adoption.
How big should an AI test set be?: A common starting point is 100 or more real cases, including hard and unusual ones, with good answers agreed by the people who do the work. Keep it fixed so results are comparable sprint to sprint.
Is stopping a pilot a failure?: No. Stopping a pilot that doesn't meet its criteria saves the cost of scaling something that won't deliver, and what you learned (and the test set) carries into the next one.