What is Open Table Format?

Data lakes store data as files, often in Parquet, in object storage. Files alone do not say which ones belong to a table, how its schema changed, or what the table looked like last week. An open table format adds a metadata layer that tracks this. The best known are Apache Iceberg, Delta Lake, and Apache Hudi. They support ACID transactions, so concurrent writes do not corrupt a table, schema changes without rewriting all data, and snapshots that let users query a table as of an earlier point in time.

Because the specifications are open, several engines, such as Spark, Trino, and cloud data warehouses, can read and write the same tables without copying them. Snapshots also help with data versioning: a team can reproduce the exact table state that fed a report or a model. A snapshot records what the table contained at that moment, and teams still record separately which snapshot a model was evaluated on and whether it passed.

Frequently asked questions

What are examples of open table formats?

The most widely used are Apache Iceberg, Delta Lake, and Apache Hudi. Each defines how table metadata, transactions, and snapshots are stored alongside data files.

Is Parquet an open table format?

No. Parquet is a columnar file format for storing data. Open table formats such as Iceberg manage collections of Parquet or similar files as tables.

What is time travel in an open table format?

Time travel lets users query a table as it existed at an earlier snapshot or timestamp, which helps with audits, debugging, and reproducing past results.