Automation bias appears when a person has the final say but the system’s suggestion does much of the deciding. A reviewer accepts a suggested label without reading the record closely; a clinician goes along with an AI diagnosis that is off; an engineer approves a permission request without checking what it does.
Expertise and training do not reliably remove it: a 2010 review in Human Factors found automation bias in both novice and expert participants. In a randomized trial published in NEJM AI in 2026, forty-four physicians who had completed a twenty-hour AI literacy course worked through six case vignettes with ChatGPT suggestions. The half who received suggestions with deliberate errors in three of the six cases scored 14 percentage points lower on diagnostic accuracy than the half whose suggestions were correct. The trial had no untrained group, so it shows that trained physicians remained susceptible, not how much training helps.
Two common forms, as distinguished by Skitka, Mosier and Burdick (1999):
- errors of commission: following a suggestion that is wrong
- errors of omission: missing a problem because the system did not flag it
How is automation bias different from confirmation bias?
- Confirmation bias is favoring information that supports what a person already believes.
- Automation bias is favoring what an automated system says, whether or not it matches the person’s prior view.
- The two can combine: a suggestion that matches a reviewer’s first impression is especially easy to accept. Both matter in human in the loop workflows.
Where does automation bias break in AI work?
Automation bias turns into a data problem when reviewed outputs are stored as labels. If reviewers mostly keep a model’s suggestions, the resulting labels lean toward that model, and an evaluation set built from them can make the model look better than it is, as described in CUBIG’s article on human approvals and ground truth.
A practical check is to set aside a small random slice and have it labeled by people shown no suggestions, then compare. Recording whether each label was made with or without a suggestion, part of data provenance, tells a team which labels to check first.