The Diagnose step is the first stage of DTS, where the engine measures why a dataset cannot be used to train or run AI, scoring what is restricted, what is scarce, and what is structurally broken before DTS synthesizes anything.
When AI efforts stall, problems with data quality and availability are among the root causes teams name most often. Most teams read that as a modeling problem. It is closer to a diagnosis problem: they never measured what was wrong with the data before they tried to train on it. The Diagnose step exists to catch that at the start, so the synthesis that follows is aimed at a real defect rather than a guess.
What data readiness diagnosis actually measures
DTS is CUBIG’s data preparation engine: it turns locked, scarce or regulated data into AI-ready data for training, testing, evaluation and sharing. It diagnoses the data first, then synthesizes the cases and conditions the task needs. The Diagnose step comes first because you cannot fix what you have not measured. Its job is narrow and concrete: read the dataset as it stands and report, field by field, why a model cannot learn from it in its current form.
Three failure modes dominate. Data can be restricted, meaning regulation or contract forbids the raw values from reaching a training pipeline. Data can be scarce, meaning the events you most need to model, fraud, rare defects, unusual cohorts, appear too few times for a model to generalize. And data can be structurally broken, meaning schema drift, inconsistent encodings, or mismatched joins have quietly corrupted the relationships a model depends on. Diagnose produces a profile that separates these, because each one calls for a different fix.
The output is not a pass or fail stamp. It is a readiness profile with a defect map attached: which columns are blocked, which patterns are underrepresented, and where the structure has broken down. That map is what the Transform step reads next.
Why teams skip diagnosis, and what it costs
The usual sequence in a stalled AI project is to blame the model, then the features, then the labels, and only much later the data itself. By then the team has spent weeks tuning around a defect that a proper data readiness diagnosis would have surfaced on day one.
Skipping diagnosis also hides which problem you have. A team that assumes its data is merely scarce will reach for augmentation and get nowhere, because the real blocker was a restricted field they were never allowed to use. A team that assumes a compliance wall will over-anonymize data that was structurally fine, destroying the signal in the process. Naming the defect correctly is half the fix.
The three checks inside the Diagnose step
Diagnose runs against the raw data through DTS’s non-access architecture, so the data profiling reads statistical shape and structure without exporting the sensitive values themselves. Three checks run in parallel, and each one feeds a different part of the plan that follows.
| Check | Question it answers | What it feeds into the next step |
|---|---|---|
| Restriction check | Which fields are blocked by regulation or contract from reaching a model? | Structure-Preserving Synthesis of the blocked fields |
| Scarcity check | Which important patterns are too rare for a model to generalize? | Rare-pattern augmentation of the underrepresented cases |
| Structure check | Where has schema drift or bad encoding broken the data’s relationships? | Repair of joins, types, and consistency before synthesis |
The restriction check maps every field against what is allowed to leave the boundary. The scarcity check measures class balance and the density of the rare events you actually care about, not overall row count, because a million rows can still hold only forty fraud cases. The structure check compares the data’s current shape against what the schema promises and flags where they diverge.
How diagnosis hands off to the rest of DTS
Diagnosis is only useful if it changes what happens next. The readiness profile is a set of instructions for the rest of DTS. Restricted fields route to Structure-Preserving Synthesis, which synthesizes values a model can learn on while keeping the distributions and relationships intact. Scarce patterns route to rare-pattern augmentation, which increases the density of the underrepresented cases without inventing signal that was never there. Broken structure routes to repair first, because synthesizing on top of corrupted joins would only launder the corruption.
This is why the Diagnose step is a measurement stage and not a cleaning stage. It does not fix anything on its own. It tells the next steps exactly where to act, and it records the before-state so the improvement can be checked afterward. With that before-state, the synthesized data can be compared against the data it started from, and its effect checked under the same task, model and evaluation conditions, rather than trusted on faith.
Run this quick diagnostic on your own dataset
Before you commit to synthesis, you can run a rough version of the Diagnose logic by hand. If you answer no to any of these, your data is not yet ready for the model you have in mind:
- Can you name every field that regulation or contract forbids from reaching a training pipeline?
- Do you know the raw count, not the percentage, of the rarest event your model must predict?
- Does the data’s current schema match what your pipeline expects, with no silent type or encoding drift?
- Have you measured class balance on the specific outcome you care about, rather than assuming it from row volume?
- Can you tell whether your blocker is restriction, scarcity, or broken structure, rather than lumping them together as “bad data”?
Most teams stall on the second and fifth questions. They know the data is a problem but cannot say which problem, and that is exactly the gap a data readiness diagnosis closes. Regulation now assumes this measurement exists: Article 10 of the EU AI Act holds the training, validation, and testing data behind high-risk systems to documented data-governance practices and to explicit quality criteria, meaning data that is relevant, sufficiently representative, and as close to complete and error-free as possible.
Where the Diagnose step fits in CUBIG’s operating layer
DTS is CUBIG’s data preparation engine, and Diagnose is its opening move: measure the defect, then synthesize the cases and conditions the task needs. The synthesis uses Structure-Preserving Synthesis and rare-pattern augmentation, and the whole engine runs through a non-access architecture so restricted values are not exported to be profiled. Because the before-state is recorded, a dataset that Diagnose flagged can be compared with its earlier state after synthesis instead of being taken on trust.
The practical takeaway is that diagnosis is not overhead you add to a data project; it is the step that decides whether the rest of the project is aimed at the right target. Run it first, and the transformation that follows has something specific to fix.
Try it on your data for free. Run a sample proof and see it on your own workflow.
