A successful pilot has a baseline, a learning goal, and a decision date. Banks should prove the value, stop what doesn’t work, and industrialize what does.
I’ve sat through more “pilot” reviews than I can count where the word had quietly stopped meaning anything. Eighteen months in, the pilot was still running, still unscaled, still being presented as promising, and nobody in the room could tell you the baseline it was supposed to move or the date anyone had agreed to decide by. It wasn’t a pilot anymore. It was a half-build with a friendlier name: permanent enough to consume a team’s attention, temporary enough that nobody owned the outcome.
That’s the failure mode innovation theater produces. Activity that looks like progress because something shipped, without ever answering the only question that matters, which is whether it worked.
A real pilot is a short experiment with a clock, not a half-build. That distinction does all the work. Before anything gets built, you baseline the outcome you’re actually trying to move: cycle time, abandonment, cost per transaction, whatever the honest number is. You set a learning goal and a decision date at kickoff, not after the fact when someone finally asks how it’s going. You ship through the bank’s own repos and pipelines even for the earliest tries, and you use canary releases where they lower the blast radius, because a pilot that can’t scale on the bank’s own stack was never really testing what you needed to know. And on the date (not near it, on it) you stop if the benefit didn’t move off the baseline, or you write the industrialize plan if it did.
That last part is where most organizations flinch. Stopping a pilot feels like admitting failure, so the date quietly slides, and the pilot becomes exactly the kind of half-build that generates finger-pointing six months later when someone finally asks what it cost. The discipline isn’t cruelty. It’s respect for everyone’s time, including the team that built the thing. A clear no, on schedule, is kinder than a slow maybe that drags on for a year.
The best external model for this discipline is X, Alphabet’s moonshot factory. Under Astro Teller, X actively rewards teams that kill their own projects. Teams that identify a fatal flaw and shut a pilot down early receive bonuses and public recognition, not because failure is celebrated for its own sake, but because intellectual honesty that saves six months of wasted engineering time is worth more than a launch that never should have happened. Projects like Project Foghorn and Calcifer were killed on purpose, on schedule, by the people who built them. At X, the kill is considered a feature of good innovation process, not a bug in it.
When a pilot does clear the bar, industrializing it is its own discipline, and it’s where good ideas most often die of neglect:
- Move the code to standard repos and run it through the same continuous integration and continuous delivery (CI/CD) controls that protect the high-risk rails.
- Wire monitoring and alerting into the bank’s own patterns so operations can see its health from day one, not three incidents later when someone realizes nobody was watching it.
- Bring in platform teams for integration and testing so squads aren’t rebuilding plumbing that already exists elsewhere in the bank.
- Flip the funding from experiment money to run budget, with a clean owner and a clear benefit target, so the thing that worked doesn’t quietly starve when the pilot budget line disappears.
Daniele Tonella, CTO of ING, put the whole loop in one sentence during our conversation for this book:
“Explore first and prove the benefits, then industrialize on bank platforms.”
— Daniele Tonella, CTO, ING
Explore is cheap and fast on purpose. Industrialize is disciplined and slower on purpose. The failure mode is letting exploration masquerade as industrialization: moving fast without ever proving the thing was worth building at scale.
The teams that actually ship innovation aren’t the ones running the most pilots. They’re the ones willing to kill the ones that didn’t earn the next stage, on the date they said they would, not the date it stopped being awkward to ask.
— Rick Mavrovich
