A pilot answers whether something can work. Production readiness answers whether it will keep working on the data, users and exceptions it will actually meet, and whether the team will know when it stops. The difference between the two lies in data and evidence as well as in the model.
Questions a production readiness check should answer:
- was the task evaluated on data drawn from production, including the exceptions
- were the acceptance criteria agreed with the business before the evaluation
- are the data version, preparation and evaluation conditions behind the result recorded
- who rechecks the result when the data, the model or the process changes
How is AI production readiness different from a successful pilot?
- A successful pilot shows a good result on the data and conditions of the pilot. Pilot purgatory is what happens when that result cannot be shown to hold beyond them.
- Production readiness requires the result to hold on production data and cases, against criteria the business set in advance.
- It also requires a way to recheck the result later, because data and processes keep changing after launch.
Where does AI production readiness break in AI work?
A support-routing agent passes a pilot on English tickets from one region and is approved for rollout. In production, the queue also holds other languages and product lines the pilot never saw, and the team cannot say which data the pilot result was based on, so it cannot tell what changed. Business rules the pilot sample never exercised are another gap; why AI misses business rules and exceptions covers how to find them before rollout.
In Syntitan, Ready for Model compares results on the data before and after preparation under the same evaluation conditions, and Ready for Agent builds Tasks and Cases for agent workflows. Execution Trace, which will tie each run to the data Version and conditions it used, is in development. Why AI fails after deployment describes what happens when these checks are skipped.