AI adoption is accelerating. But for most enterprises, the limiting factor isn’t the quality of their models, the size of their budgets, or the ambition of their leadership. It’s the infrastructure underneath.
That’s the central finding of the 2026 State of AI Infrastructure Report, produced by DDN in partnership with Google Cloud and Cognizant, and based on independent research by Vanson Bourne surveying 600 IT and business decision-makers across US enterprises with 1,000 or more employees.
The findings paint a picture of an industry at an inflection point. Where the race to deploy AI is being won or lost not in the boardroom, but in the data layer.
The Scale of the Problem
The headline statistic is almost difficult to believe: 99% of surveyed IT and business leaders report inefficiencies in their AI workloads. This isn’t a fringe problem or a maturity issue confined to early adopters. This is near-universal.
The consequences are tangible. 54% have delayed or cancelled an AI initiative in the past two years, and those with fragmented data and overly complex infrastructure are twice as likely to experience these failures. Meanwhile, the average time to realise ROI on infrastructure investments is 14 months and that stretches when systems are overly complex.
Infrastructure complexity isn’t just a technical inconvenience. It is a direct drag on business value.
Why Complexity Is Winning
The root cause, the report argues, is that most organisations are managing infrastructure like a patchwork rather than a platform. Only 38% of respondents say they access their data through a unified data platform. For those without one, managing data across multiple locations or environments remains a critical challenge. Even as they seek to take advantage of distributed resources and accelerators.
The result is infrastructure silos. With this mix of tools and access points, it’s no surprise that pipelines, storage, and compute often sit in silos. Where they rarely work together. It slows innovation before it even starts.
This problem is not evenly distributed across sectors. Complexity isn’t being felt equally, and in some industries it’s becoming an existential threat to progress. In automotive and manufacturing, massive streams of sensor and simulation data are overwhelming legacy systems. Public agencies face governance and interoperability challenges that slow every project. Financial firms are straining to meet real-time data demands for compliance and risk management.
The numbers reflect this: 76% of automotive leaders, 73% in the public sector, and 68% in finance admit their AI environments are already too complex for their teams to manage.
And the pressure is only intensifying. Over the next 12 months, AI workloads are set to more than double, growing by 110% across environments, with the sharpest increases expected in hybrid (+162%) and edge deployments (+227%).
The Data Problem Is Structural
Beyond complexity, the report identifies a deeper structural challenge in how enterprises handle their data. 76% of decision-makers face at least one fundamental data challenge, from siloed or inaccessible data sets to unchecked cloud sprawl. Legacy systems compound the problem: 65% report at least one challenge tied to legacy systems, whether from reliance on outdated infrastructure or the inability to scale for business demands.
When these structural issues are examined at the workload level, the leading source of inefficiency is energy and cooling costs, cited by 47% of respondents. This is followed by performance issues (40%), duplicated or fragmented infrastructure across teams (36%), and data movement challenges (35%).
Cloud Is Not Optional
The report makes clear that cloud infrastructure is no longer a complementary strategy, it is foundational. 97% of IT and business leaders agree that cloud platforms will play a critical role in scaling their AI initiatives over the next 12 months.
The data on cloud-first organisations reinforces why. Two-thirds (66%) of fully cloud organisations describe their AI infrastructure investments as strategic and forward-looking, compared to a 40% average, demonstrating that cloud adoption empowers teams to focus on business outcomes.
Cloud organisations also show measurable advantages in skills management. 33% of fully cloud organisations cite shortages of internal skills or expertise as a top challenge: compared to 40% on average. And just 25% of fully cloud organisations say a lack of internal skills or expertise caused their AI initiatives to fail or be paused, versus 30% on average.
The most common cloud AI use cases currently are hybrid and multi-cloud operations (70%), AI development and experimentation (69%), and centralised AI data hubs (51%), with model training and inference use cases growing rapidly.
Energy Efficiency Is the Next Frontier
One of the report’s most striking themes is the emergence of energy as a strategic constraint. As AI runs continuously across training and inference cycles, power demand is surging. A new measure of value has emerged: AI output per watt (also known as tokens per watt) capturing how much useful language, insight, or decisioning a system can produce per unit of energy consumed.
The efficiency picture is poor. 65% of infrastructure is sitting idle. 93% are chasing energy efficiency, but only 41% say recent investments are leading to results.
The long-term implications are significant. The International Energy Agency projects that electricity demand from data centres could double by 2030, a trajectory that makes infrastructure efficiency not just an operational priority, but an economic and environmental one.
The Skills Gap Is Structural, Not Cyclical
Perhaps the most sobering finding is that this is not a problem that will resolve itself through hiring cycles. 68% of IT and business leaders admit they aren’t prepared to manage AI workloads in the next six months. A year from now, 65% still won’t be ready. And 83% say they’re already struggling today.
This reflects a fundamental mismatch: traditional IT functions were designed for predictable systems (databases, applications, and networks) not the continuous, data-intensive workflows of modern AI.
As a result, the majority of enterprises are turning outward. 72% rely on third-party expertise to build and manage their AI infrastructure, while just 12% depend solely on in-house talent. The leading approaches to closing the gap include using cloud providers (64%), hiring AI-skilled employees (52%), and using specialist GPU or AI-as-a-Service providers (47%).
The Balanced Infrastructure Advantage
The report’s clearest signal of what works comes from what it calls “balanced” organisations. Those building AI infrastructure ecosystems that span both cloud and on-premises environments.
41% of respondents strongly agree that infrastructure inefficiencies limit the economic value they get from AI, but this drops to 32% for balanced organisations. Balanced organisations are 17% less likely to report underutilised GPUs, and only 35% say that improving GPU efficiency over the next 12 months will be a big obstacle, compared to 41% on average.
The report’s conclusion is direct: AI infrastructure purpose-built for scale, simplicity, and efficiency is now the single biggest driver of success. Trying to retrofit traditional, fragmented systems to handle modern AI workloads rarely works and often guarantees failure.
What Teraflow Does With This
At Teraflow.ai, this report reinforces what we see in enterprise AI engagements every day. The gap between AI ambition and AI impact is almost never a model problem. It’s a data, integration, and infrastructure problem and it requires a structured approach to diagnose and resolve before scaling.
Our Digital AI Platform Accelerator (DAPA) framework exists precisely to help enterprise clients identify these friction points in Phase Zero and build towards a unified, production-ready AI foundation.
If your organisation is among the 99% experiencing AI workload inefficiencies, the conversation worth having is not about which model to use. It’s about whether your infrastructure is ready to support it.
Source: DDN, Google Cloud, and Cognizant. 2026 State of AI Infrastructure Report. Research conducted by Vanson Bourne across 600 US enterprise IT and business leaders.
