What belongs in an honest AI efficiency calculation?
The summit's benefit and risk discussion suggests a fuller comparison than timing the first answer. Take a task from the first request to the point someone accepts it, count the preparation, review, corrections and follow-up the output causes, and include work another department ends up doing.
A tool can produce an answer fast and leave a lot of work for someone else. If another department fixes the result, that effort belongs in the project.
Correcting a draft before release differs from correcting information after a customer has acted on it, so look at errors separately by consequence. A serious failure shouldn't disappear inside an average time saving.
Use similar tasks and the same completion standard for the current process and the proposed one, and keep the sample's limits visible.
Then decide whether the workflow is worth changing. It might still offer a meaningful benefit, or it might suit only a narrower set of tasks. Measurement is there to show where the tool belongs, and a number carried over from the demonstration won't do that.
What to try this week
Time one sample task from the request until someone accepts the result, including review, corrections and any fix another department makes. Run the same task through your current process to the same standard, and write down the sample's limits before you quote an average.