In a hypothetical case, a maintenance team removes extreme sensor readings and fills the gaps before retraining its failure prediction model. The data looks more complete, yet the model catches fewer failures because some removed readings were warning signals.
Data cleaning for machine learning is the process of identifying and addressing data issues before or during model development, while preserving information the intended task may need. A useful cleaning decision considers both measurable data quality and how the change affects performance on a defined evaluation set.
Adapted from When Cleaner Data Makes AI Worse by Bogum Yoo, COO of CUBIG, on the CUBIG Tech Blog (October 2026).
Key takeaways
- Cleaning can improve completeness and consistency while removing signals a model needs.
- Data quality checks and task-level evaluation answer different questions, and both are needed.
- Compare a data change under fixed evaluation conditions using metrics that reflect the task.
- Record the data version, preparation rules and evaluation conditions so the result can be checked later.
What does data cleaning do for machine learning?
Data cleaning identifies and addresses issues that can make training data difficult to use or interpret. Depending on the task, a team may correct inconsistent units, investigate duplicate records, handle missing values or review observations outside an expected range. The goal is to make the data suitable for a particular use while retaining meaningful variation.
A data cleansing rule encodes a judgment about the data. Filling a missing reading, for example, assumes that the replacement is appropriate for the record and the prediction being made. A value that looks implausible in a general report may still be a legitimate observation from an unusual operating condition.
The distinction matters for labels as well as input features. In CUBIG’s 2026 analysis of the public, synthetic UCI AI4I 2020 predictive maintenance dataset, none of its 10,000 records had a missing value, yet 27 records had a failure label that did not match the failure-mode flags. Completeness alone did not show whether the labels agreed with the failure-mode flags.
A team can use that kind of finding to investigate the source and meaning of a mismatch before choosing a correction. The observation does not establish which field is right or what a model would gain from changing it. Those questions need domain review and, where appropriate, a task-level comparison.
How can cleaning remove useful signals from training data?
Cleaning can remove useful signals when a rule treats task-relevant exceptions as errors. In the hypothetical sensor example, spikes and dropouts are unusual compared with normal operation. Some may also occur before equipment fails, making them relevant to a model trained to recognize that pattern.
An outlier detection method can flag readings for review, but the flag does not establish whether a reading is a sensor fault or a meaningful event. A maintenance specialist may need to examine the surrounding operating conditions, the failure record and the way the measurement was collected. A blanket rule that deletes or replaces all flagged values can erase distinctions the model could have used.
The reverse risk also matters: keeping every extreme reading can leave genuine measurement errors in the training data. Teams can compare a proposed treatment with the existing approach and inspect which records changed. The decision should follow the evidence for the task, and a smoother distribution is not that evidence on its own.
The issue can extend beyond a single transformation. Sambasivan and colleagues’ research on data cascades describes how data problems can compound into downstream effects, often without clear indicators. Recording why a rule exists and what it changes helps a team investigate a later result without assuming that an individual step caused it.
CUBIG Shorts
Clean Doesn’t Mean Trustworthy
A short take on the same point: data that passes cleaning rules can still fail the task it is used for.
How do data quality checks differ from model evaluation?
Data quality checks describe the condition of data, while task-level evaluation measures performance on a specified task. Both views help a team decide whether a dataset is fit for use. A higher completeness rate can identify progress in preparation, yet it cannot tell a maintenance team how many failures its model will detect.
| Check | What it tells you | What it does not answer on its own |
|---|---|---|
| Completeness | Whether expected values are present | Whether filled values preserve a useful failure signal |
| Consistency and validity | Whether values meet defined rules or agree across fields | Whether the rules retain information needed for the task |
| Data observability | Whether monitored data has changed, become stale or drifted | Whether a specific preparation change improved performance on a defined task |
| Task-level evaluation | How a model performed on a defined evaluation set under recorded conditions | Whether the same result will hold for different equipment, data or business use |
Data quality work remains the starting point. Checks can reveal broken relationships, missing fields and incompatible units before a team spends time training a model. Observability also helps teams notice changes in data as systems operate.
Task-level evaluation adds a question that those checks do not answer: did the proposed data change help the model do the intended job under the conditions tested? The distinction also helps reviewers avoid treating a general quality score as a certificate of readiness for every model or use case. See data quality for AI for more context on connecting quality work to an AI task.
What do the 2026 AI data surveys show?
The surveys connect organizational data and AI practices with reported outcomes, but they do not measure the effect of an individual cleaning rule. The 2026 Data and AI Impact Report from SAS with IDC found that, globally, organizations applying trustworthy AI practices were 15 times more likely to report strong or high ROI from AI projects than those without: 62%, compared with 4%. The finding supports attention to data and governance foundations without establishing that a specific preparation step improved a specific model.
Market results also show that improvements in governance and reported impact can move differently. The SAS UK release from October 2026 reports that Spain’s Impact Index fell 7.9 points and France’s fell 6.1 points over the year, even as both strengthened governance. The SAS Oceania release from September 2026 reports a 13-point decline in Oceania’s Impact Index.
The Dun & Bradstreet AI Momentum Survey release from July 2026 says more than three-quarters of enterprises report some measurable AI return, while 6% say their enterprise data is fully ready to support AI at scale. Survey responses describe reported conditions and outcomes across organizations. A team deciding whether to retain a sensor cleaning rule still needs evidence from its own task and evaluation conditions.
How can you test whether a data cleaning step helped?
Test a cleaning step by comparing model results before and after the change under the same evaluation conditions. Define the prediction task and the proposed rule before running the comparison. Keep the evaluation set and its ground truth fixed so the comparison is not confounded by a different set of cases.
Split the data before preprocessing. The scikit-learn guidance on common pitfalls recommends separating training and test subsets before preprocessing; transformations that learn from data should be fitted on the training subset and then applied appropriately to the held-out subset. Otherwise, information from the evaluation data can leak into model development and make the reported result misleading.
Keep the model configuration, training procedure and evaluation code fixed while changing the training data treatment. If training is stochastic, compare repeated runs using matched random seeds and report the variation. The comparison can then be interpreted as the effect observed under those controlled conditions, with the remaining variation visible to reviewers.
Choose metrics that reflect the decision someone must make. For failure detection, examine missed failures and false alarms together: a model that flags more failures may also create more unnecessary inspections. If the use case requires advance warning, measure how early failures are detected with data that supports that question.
The evaluation set should represent the intended use as well as the available evidence allows. If equipment, incoming data or the task changes, revisit whether the evaluation still answers the operational question and rerun the relevant comparison. A better score on the held-out cases does not by itself establish a better business outcome; the result also depends on how predictions are used.
How can you trace model results back to the data?
Trace a result by recording the data and preparation choices alongside the evaluation conditions and output. A reviewer should be able to identify the records used, the rule applied, the model and evaluation settings, and the metrics produced. A record of the comparison makes it easier to investigate why a result changed and which part of the process needs another look.
Data lineage connects a result to the data and transformations behind it. Data versioning helps distinguish the dataset used before a change from the prepared version used afterward. Those records give reviewers a route back to the evidence; they do not, by themselves, explain every individual model decision.
A useful record also includes the boundaries of the test. Describe the evaluation population, the ground truth, the metrics and any known limitations, such as a dataset that cannot support an early-warning measurement. When a later result differs, the team can compare those conditions before attributing the difference to cleaning.
What should you check before a cleaning step goes into the training pipeline?
Check the proposed rule, the information it changes and its effect on the task before adding it to the training pipeline. The checklist below gives a reviewer a practical starting point. Document unanswered questions and investigate them before relying on the comparison.
- What data issue does the rule address, and how was that issue identified?
- Could the affected values, missingness or exceptions carry information relevant to the prediction?
- Were training and evaluation data separated before learned preprocessing, with the evaluation set and ground truth held fixed?
- Were model settings and evaluation procedures held constant, and was variation reported where training is stochastic?
- Do the metrics include the errors people acting on predictions care about, including false alarms where relevant?
- Are the data version, rule, evaluation conditions and limits of the result recorded for later review?
Where does CUBIG fit in data preparation and model evaluation?
CUBIG prepares and checks enterprise data for a specific AI task, connecting preparation with a task-level comparison. Data quality and observability tools remain needed to assess and monitor data. Syntitan, CUBIG’s AI-ready data platform, adds a way to compare results before and after preparation under the same evaluation conditions.
Syntitan keeps the original Dataset unchanged and works on a copy; preparation and validation results are kept as Versions. As of November 2026, Syntitan’s Ready for Model compares built-in models before and after its own preparation on the same evaluation records, for yes-or-no prediction on datasets of 100 to 10,000 rows. A team can use that comparison to assess the preparation for the evaluated task and inspect the relevant Versions.
For CUBIG, AI-ready data means data prepared for a specific AI task, with results checked and relevant parts available for rechecking when data or conditions change. Readiness is an ongoing judgment supported by the task and its evidence; CUBIG does not treat it as a one-time certification or score.
A favorable result holds for the data, task and conditions tested, and the same comparison does not promise a result for a different dataset or operating environment. Data quality work starts the investigation, and the task-level test shows whether a particular change helped.
Try it on your data in Syntitan: compare model results before and after preparation on the same evaluation records.
