Glossary
Automation bias
Automation bias is the tendency to favour a machine's recommendation over your own judgement, including when the machine is wrong and the evidence to notice is available.
In more detail
Automation bias was documented in aviation and clinical decision support decades before generative AI, and it appears in two forms. Errors of commission are following an automated recommendation that contradicts other available evidence. Errors of omission are failing to notice a problem because the system did not flag it.
Both matter more with language models than with earlier automation, because a model's confidence is uncorrelated with its accuracy. A cockpit alarm is either triggered or not; a model produces the same assured prose whether it is right or fabricating, so the surface cue people habitually rely on carries no information.
Automation bias is distinct from metacognitive laziness but compounds with it. Bias is about trusting the output; laziness is about losing the capacity to check. Together they describe a state where you defer to a system you can no longer evaluate.