Table-level lineage says that a report reads from a table that reads from another table. Column-level lineage goes one step down: revenue_usd in the report is amount from the orders table multiplied by a rate from the currency table and rounded in the transform step. That is the level at which most questions about a wrong number get answered.
Data catalogs and transformation tools now generate it by parsing SQL and pipeline code. Coverage is usually partial. Notebooks, spreadsheets, and manual steps leave gaps, and the lineage says where a value came from, not whether the value was right for the task that used it.
How is column-level lineage different from data provenance?
- Lineage maps the path between columns across tables and jobs, and it is generated from code. Data provenance records the origin and history of a specific dataset or record: who produced it, when, and under which process.
- Lineage answers “what feeds this column”. Provenance answers “where did this data come from, and can its source be trusted”.
- AI work needs both, plus a third record that neither keeps on its own: which version of the data, under which conditions, produced a given result.
Where does column-level lineage break in AI work?
A model’s strongest feature is days_since_last_contact. Lineage shows it derives from last_contact_date in the CRM. What lineage does not show is that the CRM team changed that field in June to log automated emails as contacts, so the feature’s meaning shifted while its lineage stayed identical. The model’s results drift, and the lineage graph looks healthy.
Lineage tells you where to look; it does not tell you what changed. The check that catches this records the meaning and profile of each feature at the data version the model was evaluated on, and compares them on the next extract.
In Syntitan the diagnosis and preparation results for that data sit with the project Version. Execution Trace, which is in development, will tie each run to the Version and evaluation conditions it used, so a changed result can be traced to the data and conditions behind it rather than to the graph alone. Data lineage covers the broader term, and reproducibility describes what the third record makes possible.