What is Model Collapse?

Model collapse is the gradual quality loss that happens when generative models are trained on data produced by earlier models, so diversity, accuracy, and rare cases erode with each generation.

The effect compounds. A model trained mostly on another model’s output learns a slightly narrowed version of reality, and the next model trained on that output narrows it again. Over several rounds the tails of the distribution, the unusual events that matter most in fields like fraud or rare disease, thin out and can disappear.

Model collapse is a data-quality problem, not a model problem. The safeguard is not to avoid synthetic data but to validate it: to confirm it preserves the original statistical distribution and keeps rare-case coverage before it enters a training set.

Frequently asked questions

What causes model collapse?

Training a generative model largely on outputs from earlier models. Each generation drifts toward the average and drops the rare cases the previous one already underrepresented.

How is model collapse avoided?

Keep real data in the mix and validate that synthetic data preserves the original distribution, including rare patterns, before it is used for training.

Is all synthetic data a risk for model collapse?

No. The risk comes from unvalidated, low-fidelity synthetic data. Synthetic data that holds statistical fidelity and rare-case coverage can be used safely.