When an AI system produces a different result after a data update, the first challenge is finding out what changed. A team may be able to trace the update and restore an earlier data state, but that does not necessarily explain why the result changed or whether the earlier outcome can be reproduced.
lakeFS data versioning addresses the first part of that investigation. It gives data in object storage a repository history, allowing teams to isolate changes, create reproducible checkpoints, compare states, and reverse a bad commit.
Explaining the AI result requires a wider record. The team may also need the code version, preprocessing logic, model artifact, parameters, evaluation data, metric definition, and execution conditions associated with the run.
This is the practical distinction between lakeFS and Syntitan. lakeFS controls and recovers versioned data states. Syntitan addresses whether a particular state is ready for the intended AI task and how that state is connected to the evidence behind the run. A production workflow may need both layers.
Start by identifying the missing evidence
An investigation becomes easier when the team separates three questions that are often compressed into the word reproducibility.
- Can we recover the data? The team needs an immutable reference to the repository state and a history of the changes that produced it.
- Can we reconstruct the run? The team needs the model or agent configuration, code, preprocessing, parameters, evaluation setup, and other execution records.
- Can we defend the result? The team needs a bounded comparison that shows which change mattered for the intended task and what the evidence does not prove.
These questions depend on one another, but they are not interchangeable. A model run cannot be traced to its input if the data state is unknown. A known data state cannot explain the result if the rest of the run is missing.
What lakeFS records and controls
lakeFS applies Git-like versioning semantics to data in object storage. Its official documentation defines a branch as an isolated version of the repository, a commit as an immutable checkpoint containing a complete snapshot, and a merge as an atomic update from one branch to another.
The architecture matters. lakeFS stores versioning metadata separately while the underlying data remains in object storage. Creating a branch is a metadata operation rather than a full copy of every object. This gives teams an efficient way to test and compare data changes without duplicating the repository for each branch. The lakeFS architecture guide also explains that native clients retrieve version metadata from lakeFS while data operations continue against the underlying object store.
That model provides concrete operational evidence:
- the commit that captured a data state;
- the branch on which a change was isolated;
- the differences reviewed before a merge;
- the history of how the production state was created; and
- the revert that recorded the reversal of a committed change.
The distinction between reset and revert is useful during an incident. In the lakeFS CLI, reset removes uncommitted changes from a branch, while revert creates a new commit that reverses the effect of an earlier commit. Revert therefore preserves an auditable recovery path instead of erasing the history of the committed change.
lakeFS can also enforce checks around those transitions. Its Actions and Hooks documentation describes pre-commit and pre-merge validation, including file-format and schema checks, as well as hooks that notify or trigger downstream systems. These capabilities make lakeFS more than a passive archive. They let a team place controls before a changed data state reaches an important branch.
This is substantial production infrastructure. It should not be reduced to a simple rollback button.
Why the data commit is only one part of an AI run
lakeFS describes reproducible data states as useful for debugging and for rerunning machine learning work against earlier data. The limit is not that lakeFS lacks reproducibility. The limit is the object being reproduced.
A lakeFS commit identifies the data state. A complete run record may also need the code and preprocessing versions, model or agent configuration, parameters, evaluation setup, runtime conditions, and output artifacts.
This broader record is standard experiment-tracking territory. MLflow Tracking separately records code versions, parameters, metrics, datasets, model weights, and other run artifacts. The point is not that every team must use MLflow. Its documented run model provides a neutral illustration of why a data reference alone is not the complete execution record.
The result is a layered evidence problem. Data version control can answer, “Which data state did we recover?” It cannot answer, without records from the surrounding workflow, “Did we run the same transformation and evaluation against that state?”
Five records needed to investigate a changed AI result
| Evidence layer | Question it answers | Example record | Failure if missing |
|---|---|---|---|
| Data state | Which exact data reached the run? | Repository, branch, commit or released-state reference | The input cannot be recovered or compared reliably |
| Data change | What changed from the prior state? | Commit diff, merge history, change manifest | The investigation cannot isolate the candidate change |
| Execution | How was the result produced? | Code, preprocessing, model, parameters, seed, environment | The team may restore the data and still run a different experiment |
| Evaluation | What task and test defined success? | Evaluation set, split, metric definition, threshold | Two scores may look comparable while measuring different conditions |
| Result | What output is being reviewed? | Metrics, predictions, logs, report artifacts | The team cannot connect the observed change to the recorded run |
Existing data, orchestration, experiment-tracking, and governance systems may already hold parts of the evidence envelope. The requirement is that the references remain connected closely enough for a reviewer to move from the result back to the exact data state and execution conditions.
A rollback test should narrow the cause, not declare it
Consider a hypothetical churn model whose recall drops after an April data update. The team uses lakeFS to identify the relevant commit and revert it, then reruns the workflow against the March data state.
If recall returns to the earlier level under the same model, preprocessing, split, seed, and metric definition, the rollback provides evidence that the data change contributed to the deterioration. The team can then inspect the data diff and determine which fields, distributions, or class balance changed.
If recall does not return, the rollback still creates useful evidence. It rules out a simple explanation in which the data commit was the only material change. The team should then inspect the remaining execution and evaluation records. A preprocessing change, for example, can create preprocessing drift even when the raw data and model version appear unchanged.
In both cases, lakeFS improves the investigation because it makes the data intervention controlled and reversible. The conclusion still depends on whether the rerun held the other conditions constant and preserved the resulting evidence.
Where Syntitan fits in the evidence path
Syntitan approaches the problem from the AI-use side of the boundary. Its current product flow begins with a six-axis readiness diagnosis and Get AI-Ready. The evidence path that follows is organized around four operating steps:
- Release State fixes the data state approved for the AI task.
- Run Binding connects the AI run to that Release State.
- Diff compares released states to identify what changed.
- Reproduce restores a prior state and reruns the investigation under controlled conditions.
The product page also describes a Change Manifest and Portable Proof Kit as supporting artifacts for Reproduce. The Change Manifest records the relevant data changes, while the Portable Proof Kit packages the before-and-after data, manifest, evaluation configuration, and comparison harness used to review or rerun the bounded comparison. They are supporting artifacts, not separate stages in the operating sequence.
These terms describe Syntitan’s current product model. They do not independently prove that every AI result will reproduce or improve.
This does not make Syntitan a replacement for lakeFS. lakeFS governs how data changes move through a versioned repository and can run validation around commit and merge events. Syntitan addresses whether a selected state is ready for a defined AI use and how that state is tied to the resulting proof. A team should evaluate overlap and integration requirements in its own architecture rather than assume the products connect automatically.
Match the control to the missing evidence
If the team cannot identify or reverse the data change that reached production, it needs stronger data version control. The relevant evidence is the branch, commit, comparison, merge, and revert history surrounding that change.
If the data state is known but the result cannot be reconstructed, the missing evidence sits at the run level. The team must connect that state to the code, preprocessing, model, parameters, evaluation conditions, and output artifacts used in the execution.
If those records exist but the team still cannot determine whether the data was suitable for the intended AI use, it needs task-specific readiness evidence and a controlled comparison.
A defensible investigation connects all three layers: the data state, the execution record, and the task-level test. Rollback then becomes a controlled step in the investigation, not the final explanation.
Connect every AI result to the data state and evidence behind it.
