Teams migrate models when a provider retires a version, when a newer model is cheaper or better, or when an application moves from a hosted model to one run on their own infrastructure. The steps are the same in each case: fix the evaluation set and the pass criteria, run the current model and the candidate on the same inputs, compare the results, adjust prompts, thresholds, or post-processing where the new model behaves differently, and roll out once the candidate meets the criteria. Skipping the side-by-side run is how teams discover a regression from users instead of from a test.
The data side is what makes a migration readable afterwards. If the data changed between the two runs, the comparison mixes two causes and nobody can say whether the new model or the new data moved the result. Freezing the data version and the evaluation conditions before the switch keeps the model as the only variable, which is the same discipline used to tell a migration apart from model drift. A worked example is in Model migration: freeze the data version.
How is a model migration different from a data migration?
A model migration changes which model answers. A data migration changes where the data lives and often what shape it takes. Teams run them together because a provider change is a convenient moment to move data too, and that is exactly when a comparison stops being readable.
- Model migration: same inputs, same evaluation set, new model. A changed result belongs to the model.
- Data migration: same model, new source or schema. Field meanings, encodings, and defaults can shift without a single error.
- Both at once: two causes behind every changed result, and no way to say which one moved it.
Run them in sequence, with a frozen data version between the two, and each change can be judged on its own.
Where does model migration break in AI work?
A team moves a classification workload from a retired model to its successor. The side-by-side run shows a small gain, so the switch goes live. In the same week, the reference table the prompts draw on is refreshed by another team. A month later a category that used to be caught is being missed, and nobody can tell whether the new model or the new reference data caused it.
The migration itself was done correctly; the data changed underneath it. Freeze the data version and the evaluation conditions before the comparison, record what each run read, and rerun the comparison when the data changes. Syntitan keeps each run tied to its Version and evaluation conditions as an Execution Trace, which is what makes the second comparison possible.