Error analysis
Definition
Reading real failures and sorting them into named categories, rather than just counting them. '82% accurate' tells you nothing you can act on. 'Fails on scanned documents older than 2015' tells you exactly what to build next.
Error analysis is the discipline of reading the failures your AI produced, tagging each one with a specific failure mode, and clustering the results into named categories with counts. It's how a scattered pile of bad outputs becomes a prioritized list of things to fix.
Why it matters
Aggregate accuracy is a scoreboard, not a plan. Error analysis is what converts a scoreboard into a roadmap — and the only reliable way to write good deterministic checks and LLM-as-judge criteria, because both are grounded in the failure patterns error analysis produces.
Related terms