A client showed us a credit-scoring demo that was, frankly, dazzling. Sub-second decisions, clean UI, confident outputs. The room was sold. We asked one question: show us the same case from three months ago. Silence. There was no baseline, no log, no way to know if today’s answer was better or worse than the one it replaced.
The demo proved the system could produce an answer. It proved nothing about whether the answer was right, or whether anyone would know if it stopped being right. Those are different claims, and only the second one matters in production.
A demo proves a system can answer. Only evaluation proves the answer is right — and stays right.
We have started asking for the boring artifact instead of the demo: the evaluation harness. If it exists, the system is probably real. If the only thing on offer is the demo, the impressive part is also the warning.
When the demo is the most impressive thing in the room, ask to see the evaluation harness. Its absence is the story.