What is actually breaking organisational AI programmes, and what fixes them
The headline problem
The investment is unprecedented. The failure rate is worse.
RAND Corporation found that over 80% of AI projects fail, at twice the rate of non-AI IT projects, with 74% of companies showing no tangible value from $252.3 billion in collective AI spending in 2024. S&P Global reported that 42% of companies scrapped most of their AI initiatives in 2025, up sharply from 17% the year before, and 88% of AI pilots never make it to production.
MIT’s NANDA programme, which examined more than 300 enterprise AI deployments, landed on a number that has now become impossible to ignore: 95% of organisations saw zero measurable return from generative AI investments. Gartner’s most recent infrastructure and operations survey, covering 782 leaders, found that only 28% of AI use cases in I&O fully succeed and meet ROI expectations, while 20% fail outright.
These are not the numbers of a technology problem. They are the numbers of an engineering problem dressed up as a technology problem.
What is actually failing
Listen carefully to where the breakage occurs and a pattern emerges.
Nearly 54% of AI projects stall at the proof-of-concept stage due to prolonged data acquisition challenges. Indicium’s 2025 AI Readiness Report, surveying over 670 IT professionals, found that 92% said their data infrastructure was not ready for AI, and 46% of financial services leaders named poor data quality and governance as the primary blocker. Gartner had already warned that 60% of AI projects unsupported by AI-ready data will be abandoned.
The diagnosis is consistent across every credible study. The model is rarely the problem. MIT’s research finds that the failure is almost never the model: it is data readiness, workflow integration, and the absence of a defined outcome.
Said plainly: enterprises are spending hundreds of billions of dollars on the visible 5% of an AI system and ignoring the 95% that determines whether it ever reaches production. Tomasz Tunguz, writing about the engineering reality inside production AI, put it directly: what appeared to be a simple AI magic box turns out to be an iceberg, with most of the engineering work hidden beneath the surface.
The three engineering layers that determine outcomes
At Teraflow we think about the layer beneath the waterline in three disciplines.
Each one is necessary. None of them is sufficient on its own.
Data Engineering. Pipelines, lineage, governance, quality, observability, feature stores.
Without this layer, models train on stale or fragmented inputs, drift silently in production, and fail audits. Data readiness is the largest gap in enterprise AI programmes, and models cannot recover from missing fields, broken pipelines, or undefined ownership. The 92% of enterprises that say their data is not AI-ready are describing a data engineering deficit, not a strategy gap.
ML Engineering. Training runs, evaluation harnesses, model registries, retraining schedules, drift monitoring, prompt and tool orchestration for generative systems.
Building ML models is no longer the hard part. Running them reliably in production is. Enterprises are deploying hundreds of models simultaneously across regulated industries, making manual operations infeasible and ungoverned AI a compliance liability. The MLOps market is reflecting this reality: the global MLOps market is projected to surpass $13 billion by 2027, with demand for MLOps engineers up over 35% year-on-year.
Software Engineering. Integrations, APIs, observability, identity, security, approval flows, rate limiting, runtime, deployment, rollback.
This is the layer that turns a working model into a working system. The new wave of agentic AI is making the software engineering bill of materials even larger. Port’s recent analysis of agentic systems in production identified seven distinct infrastructure blocks surrounding the agent, with the agent code itself being the smallest part of the system. Approval orchestration, agent registries, observability, runtime guardrails: none of these are model problems. All of them are software engineering problems.
The 2015 Google paper that named this dynamic, Hidden Technical Debt in Machine Learning Systems, showed a tiny box labelled “ML Code” surrounded by enormous infrastructure boxes. A decade later, the same pattern holds for agents: agents are a small part of the picture, and the infrastructure around them is where the actual work lives.
Why this is now a strategic problem, not a technical one
Three forces are converging to make engineering the binding constraint on enterprise AI value.
First, adoption is no longer the bottleneck. Stanford’s 2026 AI Index reports organisational adoption at 88%, with generative AI reaching 53% population adoption within three years, faster than the PC or the internet. Boards have approved the budget. Pilots are everywhere. What is missing is the path to production.
Second, the success patterns are now well evidenced. Among the 77% of I&O leaders who deliver at least one successful AI use case, success is attributed primarily to integrating AI into existing workflows and systems and securing full support from business executives. Integration is a software engineering output. Workflow embedding is a software engineering output. Models do neither on their own.
Third, technical debt in AI compounds faster than in traditional software. Generative AI introduces new sources of technical debt that accumulate quickly if not managed: tool sprawl, prompt stuffing, opaque pipelines, and inadequate feedback systems. Poor data quality alone is estimated to cost organisations an average of $15 million per year, and the problem compounds as AI systems consume low-quality data to produce unreliable outputs.
The combined effect is a widening gap between organisations that have built the engineering layer and those that have not. The first group is compounding advantage every quarter. The second is paying down a debt they did not know they were taking on.
What this means for the engineering services market
The implication for buyers is straightforward. An AI strategy that is not also an engineering blueprint is not a strategy. It is a wishlist with a budget attached.
The services market has shifted accordingly.
By 2026, most organisations have realised that AI success depends far more on data engineering than on model selection. Data engineering teams are now central to AI initiatives and responsible for feature engineering, data quality automation, lineage tracking, and model data readiness.
Forrester, McKinsey, and Gartner have all reframed their data and analytics guidance around the same conclusion: data engineering is no longer just about building pipelines or managing data warehouses, it is about creating resilient, scalable, and intelligent data foundations that enable analytics, automation, and AI at speed and scale.
The internal capability gap is real. The 35% year-on-year jump in MLOps demand is not being met by the labour market. Most businesses do not have a feature store. Most do not have a model registry. And most do not have evaluation harnesses, drift monitoring, or production observability for AI systems. The strategy decks promise transformation. The engineering org has a backlog measured in years.
This is the gap specialist engineering services exist to close.
The Teraflow position
Teraflow operates inside this gap. We have built the layer beneath the waterline for some of the largest operators on the continent.
For Vodafone Portugal we run more than 2,900 data pipelines under management. Moving 3.6 petabytes of data, with 150+ ML models in production serving 37 million digital users. And for Comair/Kulula we deployed pricing and operational models that lifted aircraft fill rates to 98% across more than 4 million passengers.
None of these outcomes are model stories.
They are data engineering, ML engineering, and software engineering stories with a model as the final visible artefact.
This is why our delivery is anchored on three components rather than a single methodology.
Phase Zero uses design thinking to align the engineering blueprint to the business outcome before a single pipeline is built. DAPA, our Digital AI Platform Accelerator, is a pre-engineered technical architecture covering data ingestion, transformation, governance, feature engineering, model operations, and integration. FloJo is our enterprise-grade agile delivery method that gets the platform into production at the pace the business actually moves.
The strategic shift
The next 24 months will separate enterprises that built the engineering layer from those that did not.
Gartner forecasts that organisations will spend $1.5 trillion on AI in 2025 against the gap between investment velocity and value realisation in modern technology. The leaders inside the 5% to 20% of programmes that succeed are not the ones with the best models. They are the ones who funded the work below the waterline.
Build the iceberg. Then the AI on top actually shines.
