The failure rate is not a model problem
MIT's 2025 State of AI in Business report measured what actually happens after companies launch AI initiatives: 95% of enterprise GenAI pilots produce no measurable P&L impact — a finding built on 52 executive interviews, 153 leader surveys, and 300 public AI deployments. McKinsey's global survey says more than 80% of organizations see no tangible enterprise-level EBIT impact from gen AI — while 88% of them are using it. BCG found only 26% of companies get past proof-of-concept to tangible value.
Read those numbers together and the obvious explanation dies first: it isn't the models. MIT's own conclusion is that the failures are not about model quality — the same tools sit inside both the 5% and the 95%. The difference is the approach, and the most common approach starts the same way — somebody names a target. Automate the claims queue. Fix the sales reporting. Put a copilot in support. The project is scoped, budgeted, and aimed before anyone has measured how the work actually happens.
Targets get chosen at the moment of maximum ignorance
Committing to a named target before you've measured the work means making your highest-stakes decision at the point where you know the least. In every company we've measured, leadership's mental model of where time goes has been wrong in the specifics — not because leadership is careless, but because nobody can see work they don't do. When Microsoft measured actual activity across its own platform, employees were spending 57% of their time communicating and 43% creating, 68% said they lacked enough uninterrupted focus time, and the heaviest email users were spending 8.8 hours a week on email alone — a split few org charts would have predicted.
Choosing a target does a second kind of damage: it narrows what you're able to see afterward. The management literature on this is twenty years old and blunt — narrow goals produce tunnel vision, neglect of everything outside the goal, and distorted risk decisions (Ordóñez, Schweitzer, Galinsky & Bazerman, "Goals Gone Wild," Harvard Business School). Point the organization at the claims queue and the organization stops noticing everything that isn't the claims queue — including whatever would have returned more.
Too big for the quarter, too small for the decade
Every pre-measurement target carries the same asymmetry. In the short run it's too big: a named transformation is an integration project, and it inherits the politics, the data cleanup, and the edge cases on day one. That's the 95% pattern in the MIT data — generic tools bolted onto existing processes, impressive in the demo, brittle in the workflow. McKinsey's survey makes the point from the other direction: fundamental workflow redesign correlates with EBIT impact more than any other factor they measured — and only 21% of organizations have done it. You cannot redesign a workflow you have never mapped.
In the long run the same target is too small, and the precedent is a century old. Electric power was technically available in the 1880s; factory productivity didn't accelerate until the 1920s — a lag of roughly forty years. The factories that swapped their steam engine for one big electric motor — the nameable target — captured a fraction of the gains. The gains arrived when factories were rebuilt around what electricity actually made possible: a small motor on every machine, and a floor plan organized around the flow of work instead of the drive shaft (Paul David, "The Dynamo and the Computer," American Economic Review, 1990). Economists have since formalized the pattern: general-purpose technologies pay off after the organizational and intangible co-investment, not after the purchase — and that hidden co-investment is big enough that one estimate puts officially unmeasured productivity at nearly 16% of the total by 2017 (Brynjolfsson, Rock & Syverson, "The Productivity J-Curve," NBER). Roy Amara's old line — we overestimate technology in the short run and underestimate it in the long run — is usually quoted about technologies. It applies with more force to targets.
So: whatever target you pick today is too big for the next quarter and too small for the decade. That is the whole case against picking one first.
What a million workers make obvious — and what they don't
Across the observation data behind this piece — over a million workers, hundreds of companies, years of hands-on operations work — the patterns are not subtle. The same problems repeat in nearly every company we've looked at: institutional knowledge and customer relationships living in personal phones, personal spreadsheets, and personal inboxes; manual glue work bridging systems that were never connected; workarounds management has never seen; and, in what we've observed, somewhere between a third and half of desk time going to repetitive motion the person doing it could describe in their sleep.
Here's the tension that resolves the whole argument: if the trends are that obvious, why measure — why not just hand every company the same target? Because the categories are universal and the instances are local. We know what kind of thing the map will show before we start. And in years of doing this, no one — not us, not the owner, not the person doing the work — has correctly guessed where it lives in a specific company, which instance is eating the margin, or which fix compounds, before it was measured. Experience makes the pattern obvious. Measurement makes it actionable. A target substitutes for neither.
Instrument first, commit second
The method that works is closer to fieldwork than to project management. Instrument reality — with the worker's genuine opt-in, on one volunteer's machine, for a couple of weeks. Let the person see and correct their own data before anyone else does. Then let the findings rank the roadmap, and commit capital only after the evidence is on the table.
The opt-in is not a courtesy. Work observed against people's will gets performed, hidden, or routed around — and then you're automating a fiction. Work people volunteer to show you is the real workflow, edge cases included, and the edge cases are where the value is. It's also where the resistance tax disappears: the first volunteer comes back with their own job made lighter, and the second person asks when it's their turn. BCG's data on what separates AI leaders points the same way — leaders put 10% of their AI resources into algorithms and 70% into people and process. The winners are overweighting exactly the layer the target-first crowd skips.
Own the system. Keep the data clean.
The quiet part of the MIT finding is what the failing 95% had in common: generic tools dropped into existing processes. The alternative isn't hiring a services army or handing your transformation to whatever copilot came bundled with your office suite. But it isn't do-it-alone either — in MIT's data, builds that paired internal people with outside specialists succeeded 67% of the time, against 22% for internal-only IT builds. Own the system; borrow the expertise. A mid-sized company — down to a four-person shop — can own its own system now: its own clean data, collected with consent, living in its own environment, queried by whatever model is currently best. Nobody on staff needs to code and nobody needs to become a data scientist; the collection is the hard part, and the collection is now cheap.
Owning it changes the long-run math. Model capability improves every quarter. An organization with clean, owned, well-mapped operating data rides every one of those improvements without new integration work — ask a better model better questions about the same real data, automate the next 1–2%, compound. An organization that bought a target gets to buy another target.
One boundary, so this argument stays honest: if a pain is already glaring and quantified — fifteen people hand-keying claims, a contact list walking out the door — act on it. Measurement re-ranks and de-risks known problems; it is not an excuse to delay them. The claim here is about sequencing commitment, not pretending you have no ideas.
Most companies right now are saying "we need to do this with AI." The ones that end up ahead let the AI show them what can be done — because the opportunity is reliably bigger than the target they would have picked.
The data above comes from someone who does this for a living. If you want the measured version of your own operation, there's a free two-week pilot and a book-a-call page — Scott Moran runs the calls himself. Example findings packages are at smbpilot.go2.io.