AI pilots struggle to go beyond testing

Companies are green-lighting AI pilots that work well in testing, only to watch them stumble when scaled. The problem isn’t always the technology—it’s the planning, processes, and expectations behind the projects.

At a recent roundtable, business leaders pointed to a common pattern: pilots succeed in controlled environments, but fail to deliver real business results when rolled out broadly. The disconnect often starts with unclear goals.

Pilots bloom, but governance withers

Sean Bruich, chief technology officer at Amgen, said the ease of launching pilots encourages experimentation—but that can backfire. “It’s so easy with a pilot to let a thousand flowers bloom,” he said. “The key to making pilots scale successfully is actually having a wide number of ideas, but a very tight governance on which pilots are actually greenlit.”

Not every AI project deserves to scale. Some are better left as experiments. The challenge is distinguishing between the two before investing heavily in deployment.

Bells and whistles vs. real outcomes

Lashonda Anderson-Williams, chief customer and commercial officer at Salesforce, said companies often fixate on the technical success of AI features rather than the business impact. “The AI features work great, but the new technology isn’t driving meaningful business results,” she said.

For AI to work at scale, companies need a detailed understanding of workflows—the people, groups, and touchpoints involved in completing tasks. Many organizations lack this documentation, or it’s poorly maintained.

Data silos and stakeholder buy-in

Data access is another major hurdle. Information is often scattered across silos, governed by different access rules, privacy policies, and security protocols.

Bruich emphasized that transformational AI projects require broad buy-in. A project that only makes work more efficient for a small team isn’t enough. It needs to deliver “an outcome that matters to the enterprise,” he said, involving leaders from finance, HR, and other departments.

Without that alignment, even the most promising AI pilots can falter when they move beyond the lab.