Picture an inspection model on a bottling line that misses a cracked cap, and a first proposal to generate a thousand more crack images. Before that order goes out, someone should check what the model calls a crack.
AI defect detection uses a computer vision model to flag damaged or out-of-spec parts in images from a production line. Its training data decides what the model treats as a defect, so the first question for any data budget is which gap you are filling: known variation, an unseen condition or an unsettled label.
Adapted from A Thousand More Examples of the Wrong Thing by Minchan Chung, Co-Founder and Co-CEO of CUBIG, on the CUBIG Tech Blog (October 2026).
Key takeaways
- Before generating more defect images, check whether the model’s errors come from too few examples, an unseen condition or a label the inspectors have not agreed on.
- Synthetic images are easier to brief and check when inspectors can describe the defect and the camera and lighting conditions, and when the brief also covers intact parts that look misleading.
- A material or failure mode nobody has photographed needs a small real capture first; the capture shows which variations are worth generating.
- Test on real images held out before any generation or tuning, at the ordinary operating mix, and count missed defects and rejected good parts separately.
What does an AI defect detection model learn from its training data?
An AI defect detection model learns the boundary between acceptable and defective from the images and labels it is given. If the cracked caps in the training set all come with a bright streak, the model can learn the streak instead of the crack. Geirhos and colleagues call this shortcut learning: decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios.
The example in this article is a fictional cap inspection line. Suppose the model misses some cracks and also rejects intact caps whenever light catches their surface. A folder of dramatic cracks adds more examples of damage, yet it does little about the distinction that is causing the false alarms.
Real defects are often scarce, and one established research setting trains inspection models only on good parts. In the MVTec AD benchmark (Bergmann et al., 2019), each category has a training set of defect-free images and a test set that mixes defective and defect-free images. That scarcity is what makes “generate more defects” attractive, and it is also why the order needs a clear brief.
Which data gaps look the same in an AI defect detection project?
Three different gaps can produce the same symptom: the model misses defects or rejects good parts. Each one calls for a different purchase, so a single price per thousand images hides most of the work.
| Gap | What it looks like | What to buy first |
|---|---|---|
| Known variation | Inspectors can describe the defect, but the training set lacks its angles, lighting or positions | Targeted generation within documented camera and lighting conditions |
| Unseen condition | A new material finish or failure mode that nobody has photographed | A small capture on the installed camera, or a controlled capture run |
| Unsettled label | Inspectors disagree on whether a mark makes a part unacceptable | An agreed acceptance rule, another measurement or an uncertain category |
Label problems also appear in tabular maintenance data. In CUBIG’s 2026 analysis of the public, synthetic UCI AI4I 2020 maintenance dataset, 27 of 10,000 records had a failure label that did not match their failure-mode flags, although none had a missing value.
Access restrictions create a fourth kind of shortage. If existing images cannot be used, their permission and handling requirements need attention before they become generator inputs, because producing synthetic outputs does not by itself establish permission to use the source material.
When do synthetic images help AI defect detection?
Synthetic images can help when the team can state what the defect looks like and under which conditions the camera sees it. An inspector may know that a crack follows the rim, that its appearance changes with viewing angle and that certain surface marks are acceptable. Those constraints give the generation work something firmer than a request for images that look damaged.
A useful brief for the cap example covers the confirmed rim crack under the camera views and lighting range used on the line. The brief also includes intact caps under troublesome reflections, so a bright streak stops working as a proxy for damage.
The brief should also list what to leave out. A crack cannot float above the cap, and a reflection should not acquire the physical edges of a break. Combining every defect, material, angle and lighting condition at random can produce many scenes the actual process could never create.
Shorten and Khoshgoftaar describe the safety of an augmentation method as its likelihood of preserving the label after transformation. A rule that keeps the label in one task can change it in another, which is why the exclusions belong in the brief.
Generated examples are best treated as candidates to inspect and test, with their assumptions attached. For a proposed way to test whether generated images help an image classifier, see CUBIG’s article on synthetic data augmentation for classifier training.
CUBIG Shorts
100 Times Larger. Still Passed.
A short take on the same point: making a dataset larger does not by itself make it fit the task it is meant for.
When should a team photograph real parts instead of generating them?
A team should photograph real parts first when the material finish or failure mode has never been observed. A generator can fill that gap with plausible images, but it cannot tell whether the invented appearance matches what the new material will do under the installed camera.
For that gap, the useful spend is time with the physical setup: photograph the material, examine actual failures or run a controlled capture. The resulting collection might be small. Its value is that it shows which variations are worth generating next.

What should happen when inspectors disagree on a defect label?
When two inspectors disagree about whether a mark makes a part unacceptable, generation can wait. The mark may be cosmetic, its significance may depend on depth the current camera cannot show, or the acceptance rule may never have been written down.
Giving thousands of generated variations a confident “defect” label would bury that unresolved decision inside the training set. Someone needs to settle the rule, obtain another measurement or keep an uncertain category, and the right expense may be an expert’s inspection time or a different sensor.
Label errors distort evaluation as well as training. Northcutt, Athalye and Mueller estimated an average of at least 3.3% label errors in the test sets of 10 commonly used datasets, and showed that such errors can change which model looks best on some benchmarks.
How do you test an AI defect detection model without fooling yourself?
An honest test keeps real images outside every step that shapes the model or the generator. Development examples such as missed defects, false alarms and ambiguous images can guide the generation brief, but the final test set stays out of that discussion.
The scikit-learn guidance on data leakage says to split the data before preprocessing and never to include test data when fitting. The same boundary applies to generation: reserve the real test material before fitting or tuning the generator and the classifier.
Related captures of the same physical cap belong on the same side of the split. Nearly identical images on both sides would give a misleading impression of how the system handles an unseen part.
The imbalanced-learn documentation adds that resampling before the split can leak information and leave the test set with a class mix unlike actual use. A balanced test of generated cracks and generated intact caps cannot tell a plant how many good caps its system will reject in an ordinary shift.
A credible comparison keeps real captures at the ordinary operating mix, alongside a separately reported set of confirmed rare defects. The existing model and the augmented version run under the same evaluation conditions, and missed defects and rejected good parts are counted separately, because an average can hide an expensive trade.
If too few real defects exist to make that comparison credible, the conclusion stays limited. Synthetic stress cases can help explore behaviour, but they do not supply the missing evidence of field performance.
What should a first data order for AI defect detection include?
A first order can be smaller than “a thousand more defects.” For the cap example, it would look like this:
- Generated variations of the confirmed rim crack, within the documented camera and lighting conditions
- Intact caps with glare, so reflections stop working as a shortcut
- No unsupported materials and no images whose labels the inspectors dispute
- A parallel capture of the unfamiliar finish on the actual setup
- A real test set reserved before any generation, with rare defects reported separately
The next generation batch is decided after the team has seen those captures and the test results.
Where does CUBIG fit in an AI defect detection data decision?
CUBIG treats data as AI-ready when it has been prepared for a specific AI task, its before-and-after results have been checked under the same evaluation conditions, and the relevant parts can be rechecked when the data or conditions change. The cap example follows that logic: the data order starts from the model’s actual errors, and the result would be judged on real images the preparation never touched.
CUBIG’s DTS (Data Enablement for AI) fills missing cases and adjusts skewed distributions so data fits a specific AI task. CUBIG’s article on rare defect synthesis for manufacturing describes that approach, starting with a diagnosis of the data, for inspection data with scarce defect classes.
Syntitan’s Ready for Model compares results before and after preparation under the same evaluation conditions. As of November 2026, Ready for Model runs that comparison for yes-or-no predictions on tabular datasets of 100 to 10,000 rows, so an image inspection team would apply the same principle in its own evaluation setup.
The cap line in this article is fictional, and the right mix of generation, capture and labeling work depends on each line’s defects, cameras and acceptance rules.
If your inspection results also live in tables with a pass-or-fail outcome, such as measurements, sensor readings or inspection logs, try a before-and-after comparison on your data in Syntitan.
