Why 80% of AI projects fail, and what we do differently.
Three studies, one year apart, put the failure rate of enterprise AI between 80% and 95%. The reasons are not mysterious. They are organisational, and they are fixable, if the engineers are in the building and the output is something the operation can rely on.
The numbers
In August 2024 RAND interviewed 65 data scientists and engineers with at least five years of experience building models in industry or academia. Its conclusion: more than 80% of AI projects fail, double the failure rate of IT projects that do not involve AI1. A year later MIT's NANDA initiative analysed 300 public AI deployments, interviewed 150 leaders and surveyed 350 employees. About 95% of generative AI pilots, it found, deliver no measurable impact on the P&L2. Between the two, Gartner predicted that at least 30% of GenAI projects would be abandoned after proof of concept by the end of 20253.
These are not the same measurement, and they are not the same population. They agree on the shape: the pilot is easy, the P&L is hard.
Five root causes, in RAND's words
- Stakeholders misunderstand or miscommunicate the problem the AI should solve and what the technology can do.
- The organisation lacks the data needed to train an effective model.
- Teams focus on the latest technology instead of whether it solves a real user problem.
- The organisation lacks the infrastructure to manage data and deploy finished models.
- AI is applied to problems that are too difficult for it.
MIT's version is shorter. The failure is a learning gap: generic tools do not learn the workflow, and the organisation does not learn the tool. Purchased or partnered deployments succeed about 67% of the time; internal builds about a third as often2. Gartner's four reasons are the same list in a different order: weak data quality, insufficient risk controls, rising costs, unclear business value3.
The pilot is easy. The P&L is hard. The difference is who sits in the operation while the system learns it.
What we do differently
We do not think the models are the problem. We think four things are, and we build an engagement around each.
1. Forward-deployed engineers
Our engineers work in the plant, on the ward, at the claims desk. They own one P&L line for four quarters, build against the operation's own data, run the evaluations with its people, and stay until the output is trusted and the number is signed by the CFO. This is the "partnered" model MIT finds succeeding twice as often, with the partner inside the building.
2. The data foundation first
RAND's second cause and Gartner's first reason are the same: the data. Before a model, the Master Data Management Layer: one record for every material, customer, supplier, asset and lot across SAP, Oracle, Microsoft, MES and legacy, with governance rules, conflict resolution and lineage. A model trained on a golden record is a different project from a model trained on four disagreeing ones.
3. Sovereign, on-premise systems
Most of the enterprises we work with cannot send plant telemetry, patient records or claims data to a provider outside their jurisdiction, and should not. Our AI systems run in the client's data centre or a sovereign cloud, air-gapped where the regulator or the plant requires it, with data, models and logs under the client's keys. Residency is a setting the auditor can read, not a clause in a contract.
4. Deterministic, replayable output
Language models are probabilistic. The systems around them do not have to be. Where a decision depends on the output, we make the output reproducible:
- Fixed decoding. Temperature at zero, fixed seeds, pinned model builds. The same input returns the same output.
- Schema-constrained responses. The model fills a structure, never free text: a fault class, a lot number, a decision with reason codes.
- Validators before writes. Business rules check every output against the master data and the policy. The model proposes; the validator decides what is written.
- Grounding with citations. Answers that rely on documents carry their passage. No citation, no answer.
- Confidence floors. Below a threshold the run goes to a named person with the evidence attached.
- Evaluations before every change. A fixed test set per task. A change that lowers agreement with the reference set does not ship.
Every run stores its inputs, model version, decoding parameters, validators and an output hash. A reviewer can replay it a year later and get the same answer. That is what "reliable" means in an operation, and it is engineered, not hoped for. The evidence files into ShieldOS, where the auditor will look for it.
What this looks like in a plant
| Task | Model proposes | Validator checks | Person decides when |
|---|---|---|---|
| 8D fault classification from a photo | Fault class, affected lots | Lot exists, genealogy matches | Confidence below floor |
| Claim validation | Decision with reason codes | Eligibility, policy limits | Any denial above a value |
| Rate recommendation | Rate within bounds | Bounds, parity rules, replay | Out-of-bounds proposals |
| Discharge letter draft | Structured draft with citations | Medication reconciliation | Always: a clinician signs |
The 5%
MIT's 5% are not the companies with the best models. They are the ones that integrated AI into a workflow that already had clean data, a governance model and somebody responsible for the number. That is an engineering problem and an organisational one. It is the work we do, and it is why we start with one P&L line and four quarters rather than a platform and a roadmap.