Pre-ship validation
Definition
Testing an AI feature against your golden dataset before it reaches users, so you know how it behaves at scale rather than how it behaved in a demo. The step almost nobody does — and the only one that prevents releasing an AI that disappoints your users.
Pre-ship validation is the practice of running your candidate model, prompt, and configuration against a golden dataset before deployment. It's the difference between 'we tried it on five examples' and 'we know how this behaves across two hundred real cases.' Done well, it produces a written record of the failure modes you know about, so you can decide which are acceptable and which need another round before ship.
Why it matters
With AI you can't roll back user trust the way you roll back a deploy. The user who got a useless answer on day one isn't coming back for v2. Post-ship fixes cost 10–15x more than the pre-ship testing they replace, and the silent failures that happen in between are invisible until churn shows up in the dashboard.