Take the Marketing Capability Maturity Assessment. Fifteen minutes. Begin →
All articles

July 15, 2026Artificial Intelligence

The AI pilot that wowed the room but lost the work

By Mark Mountan

Share

4 minute read

Here's something that's become familiar in scaled marketing organizations.

Someone vibe codes an agentic AI solution over the weekend. On Monday, they show it off to the team. It drafts campaign briefs or spits out a decent first pass at email subject lines for A/B testing. If it's more data-centric, it scores leads and identifies new targeting segments.

It works. The team gets excited. "This changes everything," someone declares. There's a real buzz in the room.

Fast-forward six months later.

The team is doing its job the same way it always did. And that cool tool is now a browser tab that two or three people open out of habit, mostly because they feel bad letting it go.

The demo won the room, but it never won the work.

The tech's always to blame

Unfortunately, this is now the norm, not the exception. A 2025 study out of MIT's NANDA initiative put a number on it: roughly 95 percent of enterprise generative AI pilots produce no measurable bump to the bottom line.

When an AI pilot fails, the finger is almost always pointed at something technical, at least initially. The data was a mess. The wrong model was used. It couldn't integrate with this or that channel.

Here's the thing, though. The problem is rarely the tool. The root cause of failure is operational.

Fast-spun, agentic solutions rarely account for their surrounding operations. Too often, the organization isn't set up to absorb the tool into how work actually gets done.

A pilot is a few people walking the one path that works.

The real operation is fifty people, a hundred weird edge cases, and a process that has to keep humming even when the person who built the tool is out on vacation.

Production lives or dies on the boring questions

Going from prototype to production is the same kind of work any marketing team has always done when it adopts something new and unfamiliar. Rolling out AI into production is no different.

It starts by asking the boring questions:

  • Who owns it? Leaving this question unanswered means that when the output goes sideways at volume, there's no one whose job it is to notice.
  • How do we know when it's broken? Skip this question and quality will drift silently until a customer is the first to feel it.
  • Who's allowed to overrule it, and on what grounds? Without tackling this, the first time the machine's answer clashes with a manager's gut, everything grinds to a halt while people argue over who wins.
  • Where does it sit operationally? Ignore this question and you'll sit by as the work flows around the tool instead of through it.

Notice something. None of those questions are about the algorithm itself. They're about the operational governance and capabilities that surround and ultimately enable it.

Maturity first, tooling second

Shifting AI from a "wow" factor to a value driver means approaching the solution it provides from a capability maturity perspective first and a tooling perspective second.

If you want to know where your AI efforts are likely to stall, you need a clear-eyed view into your operational maturity and the capabilities that underpin your marketing operations. This requires resisting the urge to test drive a new model or kick the tires on another vendor's product.

At the heart of AI is the promise of acceleration, of doing more with less without compromising quality. So, if there's a single move you could make at an operational capability level that would unlock that potential, you want to make that move and keep going, not stall out trying to mature 10 things at once.

A maturity view turns a vague "our operations aren't ready" into a ranked list of what to shore up, in what order, to get value out the door as fast as possible.

The afterblink.co Marketing Capabilities Maturity Assessment (MCMA) devotes an entire section to the roles, the structure, the decision rights, and the organizational change management to drive adoption.

There's a second reason this maturity view matters, and it's getting more important by the month. New AI solutions are landing faster than any scaled marketing operation can change to absorb them.

A maturity-based view means you don't have to blow up your operations every single time the ground moves. When you understand your capabilities and where you've deliberately invested, you can adapt to a new model by tuning the few things that need tuning, not by rebuilding everything from scratch. The maturity lens turns frequent model change from a recurring fire drill into a series of manageable adjustments. It's the difference between an organization that gets whiplash from every release and one that absorbs each one and keeps moving.

When it really is the model

One honest caveat across all of this. Some AI pilots really are limited by the model, where the tech just can't do the job no matter how well tuned and efficient the surrounding operations are. When that's the case, no amount of ownership or monitoring saves it.

In those rarer situations, the path forward is more straightforward. Test driving other tools to find one that works, reverting to a non-AI solution until the market catches up to the requirements, or abandoning the use case altogether (especially if the maximum benefits are small).

An illustration of a woman reading on a park bench in the city.

Stay close to our work.

Join the Don't Blink Briefing - your 5-minute Friday email that cuts through the noise so you can achieve your transformation objectives with confidence.