The demo everybody loved
August 22, 2026
Nearly half of AI proofs-of-concept are scrapped before they reach production. Very few of them failed. Most of them worked. Under conditions that were never going to exist again.
March, and then eighteen months
The demo goes well. It always goes well. That is what a demo is. The steering committee is genuinely impressed, agrees this should be productionised, and asks for a plan.
Then the plan meets the building. The pilot ran on an extract somebody had hand-cleaned over two weekends. Production data has the awkward eight per cent nobody mentioned. Security review has a queue and takes four months. The two engineers who built it are good, which is why they have already been moved to something else. The system it needs to write into is owned by a team that was never in the room. And somewhere around month eleven the sponsor changes roles, and the new one asks, reasonably, why we are still spending on this.
Nothing dramatic happens. There is no decision to cancel, no post-mortem, no lesson recorded. This is the elephant we call Pilot: sandbox success that never reaches production, or the P&L.
The evidence
Nearly half never make the crossing
46%
of AI proofs-of-concept are scrapped before they ever reach production
S&P Global Market Intelligence, 2025 — survey of 1,000+ organizations in North America and Europe. Cited through CIO Dive, a named secondary: the primary report is not publicly retrievable, so these figures carry a one-hop rate-down.
Isn't a 46% scrap rate just what experimentation looks like?
It is the right objection and it deserves a straight answer: yes, experiments are supposed to die. An organization whose proofs-of-concept all reach production is not succeeding, it is being timid. It is only attempting things it already knew would work. A POC that establishes cheaply that an idea does not hold is a good outcome, and 46% would be unremarkable if that were what it described.
But it mostly is not, and this is the distinction the headline number hides. There are two entirely different populations inside that figure. One is experiments that failed informatively: the model was not accurate enough, the data did not support the claim, the economics did not work. Those are wins, cheaply bought.
The other is pilots that worked and still did not ship. Those taught the organization nothing except that it cannot get something across the border between demonstration and operation. That lesson is never written down, so it is learned again next year with a different technology. The number worth putting in front of a board is not how many proofs-of-concept were scrapped. It is how many of the scrapped ones had already proven their case.
Why is the border so hard to cross?
Because a pilot is, by design, exempted from almost everything production requires. The exemptions are what made it fast enough to approve. It gets the best available people rather than the ones who will operate it. It gets data that was hand-selected and quietly repaired. It is allowed to skip the integration, because a screen is enough to demonstrate the point. It runs outside the compliance and security processes on the reasonable grounds that nothing real is at stake yet. And it is permitted to ignore the edge cases, which in production are not edges at all but a standing fraction of the volume.
Every one of those exemptions is sensible in isolation. Together they mean the pilot did not test the thing that was uncertain. What was uncertain was never whether the technology could produce the output. A vendor could have shown you that. What was uncertain is whether this organization, with these people, this data and these controls, can operate it on an ordinary Tuesday.
So the exemptions expire all at once, and the work that was deferred becomes the actual project: larger than the pilot, unbudgeted, and now owned by whoever is unlucky enough to be holding it.
How do you know the Pilot is in your room?
The signature is a project that never fails and never finishes.
- A proof-of-concept was declared a success, and nobody can say what it is now.
- The pilot's data was prepared by hand, and nobody has costed doing that continuously.
- The people who built it are not the people who would run it, and were never expected to be.
- Security, compliance and the owning system team are scheduled after the demo rather than before it.
- The organization has run several pilots on the same theme in successive years, each starting from scratch.
So should we stop running pilots?
No. Change what the pilot is for. Stop using it to answer whether the technology works, which is the question you are least uncertain about and the one a vendor demo already settles. Use it to answer whether you can run the thing.
In practice that means deliberately removing the exemptions rather than enjoying them. Ordinary people rather than the strongest available team. Real data with its awkward fraction included. The actual integration, even in a crude form. The compliance conversation at the start, when it is cheap, rather than at month four when it is a wall. A pilot built this way is less impressive in March and enormously more informative, because when it works you have already demonstrated the only thing that was ever in doubt.
This is why we define the middle step of an engagement the way we do: the pilot is the training. If the people who will operate something are not the people proving it, then whatever was proven does not transfer, and you have bought a demonstration rather than a capability.
How do you get the Pilot out of the room?
The same three moves we bring to any elephant, pointed at this one.
Map
Audit the graveyard before commissioning anything new. For each proof-of-concept of the last two years: did it work, and did it ship? The four boxes of that grid tell you whether you have a technology problem, a selection problem or a crossing problem. Organizations are usually confident about which one they have, and usually wrong.
Prove
Run the next one under production conditions from the first week: the people who will operate it, data as it actually arrives, the real integration, and the compliance conversation up front. Define done as running for a month in the hands of its owners. It will look slower than the demo you are used to. It is the only version that ever finishes.
Scale
Extend what survived contact with an ordinary Tuesday, and record what the crossing cost, because that number is what makes the next business case honest. An organization that knows its own cost of crossing stops mistaking a demonstration for a decision.
Find out what happened to your proofs-of-concept
The Elephant Safari names your herd in ten questions and ranks them by what they cost you. Or start from the business problem instead: having paid for AI that nobody uses. Get the Pilot out of the room. Prove you can run it.
Every figure in this article traces to a primary source. See it in Knowledge