Infrastructure Software Watch: Infrastructure Software Becomes AI's Control Plane
Infrastructure Software Is Becoming AI's Control Plane
The Next Battle in AI Infrastructure May Be Fought Over Software, Not Chips
The next battle in AI infrastructure may not be fought over chips or data centers. It may be fought over the software that controls them. As AI workloads become larger, more distributed, and more expensive to operate, infrastructure software is moving from a background engineering layer toward the center of the technology stack. Kubernetes, orchestration, observability, automation, and AI-powered operations are becoming increasingly important not simply for running applications, but for determining how efficiently the infrastructure itself is used.
The traditional infrastructure model—compute → network → storage → applications—is giving way to a longer, more complex chain: compute → network → storage → orchestration → observability → automation → AI agents. Each layer adds a control surface. And control of those surfaces is becoming strategically important as enterprises spend hundreds of billions on AI infrastructure.
AI Is Rewriting Infrastructure Requirements
AI workloads are not conventional enterprise workloads. They are GPU-intensive, distributed across many nodes, and characterized by dynamic resource requirements that change throughout the training or inference lifecycle. Distributed training requires synchronization across thousands of accelerators. Inference at scale demands consistent low-latency execution. Data movement between compute, memory, and storage creates bottlenecks that conventional infrastructure was never designed to address.
The economics make this urgent. According to Cast AI's 2026 State of Kubernetes Optimization Report, average GPU utilization across 23,000 Kubernetes clusters sits at approximately 5%—meaning enterprises are provisioning roughly 20 times the GPU capacity they actively use. In optimized environments, GPU utilization can reach around 50%. Moving from 5% to 50% can lower the cost of running AI workloads by several multiples.
The more expensive the infrastructure becomes, the more valuable software that can optimize its utilization becomes. This is the core economic logic driving investment into infrastructure software. GPU hours cost real money. Idle GPUs are a financial inefficiency that infrastructure software can address.
Kubernetes Becomes an AI Infrastructure Layer
Kubernetes has become the default orchestration layer for AI workloads. According to the CNCF 2025 Annual Cloud Native Survey, 66% of organizations hosting generative AI models use Kubernetes for some or all of their inference workloads. Yet only 7% deploy AI models daily—a gap that highlights the difference between running AI on Kubernetes and having a Kubernetes platform ready to operate AI continuously.
The ecosystem is evolving to close that gap. NVIDIA donated its Dynamic Resource Allocation (DRA) driver for GPUs to the CNCF, enabling Kubernetes to request and redistribute GPU resources more flexibly. The KAI Scheduler—NVIDIA's high-performance AI workload scheduler—was accepted as a CNCF Sandbox project, transitioning from a vendor-governed tool to a community standard. And GPU support for Kata Containers extends hardware-level workload isolation to GPU-accelerated workloads.
The broader challenge is that production AI pipelines are heterogeneous. Data preparation and preprocessing may run on CPUs while training uses GPUs or other accelerators. A single workload may depend on several resource types. Platform teams need to consider accelerator type, topology, and workload characteristics alongside CPU, memory, and traditional scheduling constraints. As the CNCF observed, "The goal isn't simply to make GPUs available. It is to make heterogeneous compute part of a consistent Kubernetes resource model."
From Observability to Autonomous Operations
Traditional infrastructure operations follow a linear pattern: monitor → detect → alert → human investigates → human fixes. The emerging model compresses that chain: observe → identify → diagnose → recommend → automatically remediate. This shift is being driven by a new class of AI-native infrastructure companies.
DataAgent, which emerged from stealth on September 1 with $10 million in pre-seed funding, has launched a remediation-first platform that acts as an AI-native autonomous SRE for Kubernetes. The platform deploys agents that capture high-fidelity signals at the source and act autonomously to remediate issues the moment a fault occurs—without shipping telemetry to an external vendor.
Empirik, a Sequoia-incubated startup that spun out with $21 million in seed funding on September 1, takes a different approach. It tracks system changes across enterprise infrastructure and infers their ripple effects—acting as an autonomous "traffic cop" that permits low-risk changes, sets guardrails on larger ones, and flags dangerous updates for human review. The company describes its ambition as doing for infrastructure engineers what Cursor and Claude Code did for software developers.
Datadog has extended its Bits AI suite to support autonomous detection, investigation, and remediation across the production lifecycle. The system scans infrastructure around the clock, surfaces issues, recommends fixes, and resolves them autonomously using pre-defined guardrails. As Datadog's CEO framed it, "AI has created new operational challenges where code development has outpaced human-scale management... The companies that win on AI won't just build better models; they'll build operational control around them."
The AI-Native Platform Engineer
Infrastructure engineering itself is being reshaped by AI agents. Pulumi has built its infrastructure-as-code platform to be driven by coding agents—Claude Code, Codex, Cursor, and GitHub Copilot can write, preview, and deploy infrastructure directly. Pulumi Neo, the company's own infrastructure agent, adds organizational context, policy guardrails, human-in-the-loop approvals, and scheduled autonomous work.
GitHub has announced Agent HQ, positioning the platform as an open agent ecosystem with Mission Control for assigning, steering, and tracking multiple agent tasks. Code agents from Anthropic, OpenAI, Google, Cognition, and xAI will be available directly within GitHub as part of Copilot subscriptions. New enterprise features include agentic code review and a dedicated control plane for managing AI access and agent behavior.
The question is no longer whether AI agents can write infrastructure code. It is whether the infrastructure engineer increasingly becomes the supervisor of AI systems managing infrastructure—reviewing plans, approving changes, and handling exceptions rather than writing configuration from scratch.
Infrastructure Software as the New Control Layer
The emerging infrastructure stack has six layers, each adding a control surface:
Infrastructure orchestration → Kubernetes, schedulers, resource management
Platform layer → Infrastructure-as-code, developer platforms
Observability → Monitoring, tracing, logging
Automation → Remediation, optimization
AI agents → Autonomous infrastructure operations
Control of this layer is strategically important because it determines how efficiently expensive infrastructure is used. The AI infrastructure management software market was valued at $8.27 billion in 2025 and is projected to reach $29.84 billion by 2033, growing at a 17.4% CAGR. AI resource orchestration solutions are the fastest-growing component segment, projected to register an 18.3% CAGR.
The companies that control orchestration, observability, and automation layers will determine whether AI infrastructure delivers on its promised performance. As TrueFoundry's CEO noted in announcing the acquisition of Seldon AI, enterprises need "one place to deploy, observe, and govern Agentic AI at every stage"—a control plane that spans models, agents, and tool endpoints.
The Economics of Infrastructure Software
Why are companies willing to spend money on infrastructure software? The answer lies in the magnitude of the numbers involved. If an organization spends hundreds of millions of dollars on computing infrastructure, even a small improvement in utilization represents significant economic value. At 5% GPU utilization, enterprises are effectively wasting 95% of their GPU capacity. Moving to 50% utilization would dramatically change the economics of AI workloads.
The costs that infrastructure software addresses are substantial: GPU utilization, cloud bills, downtime, engineering labor, and operational complexity. The companies that can reduce these costs—or improve the efficiency with which infrastructure is used—capture value proportional to the savings they generate.
This is not a commodity market. It is an efficiency market. And efficiency markets reward vendors that can demonstrate measurable improvements in utilization, reliability, and cost—not simply feature lists.
Consolidation Is Coming
The infrastructure software market is consolidating. IREN acquired Mirantis for approximately $625 million in May 2026 to strengthen AI cloud delivery capabilities. TrueFoundry acquired Seldon AI in June 2026 to unify MLOps and agentic AI operations on Kubernetes. New Relic acquired Pixie Labs for Kubernetes observability. These deals point to a common theme: enterprises increasingly prefer integrated platforms over specialized point tools.
The question is whether enterprises will buy many specialized tools or consolidate around integrated infrastructure platforms that span orchestration, observability, automation, and governance. The answer is increasingly the latter—driven by the operational complexity of managing AI workloads across hybrid and multi-cloud environments.
What to Watch Next
- Kubernetes AI adoption: Whether the 66% of organizations using Kubernetes for inference translates into continuous production deployment—not just experimentation.
- GPU scheduling technology: DRA adoption, KAI Scheduler community development, and whether utilization improves beyond the 5% baseline.
- AI-native observability: DataAgent, Empirik, and Datadog Bits AI will test whether autonomous remediation can scale in production environments.
- Autonomous remediation: The first enterprise deployments of self-healing infrastructure will reveal whether guardrails are sufficient for production confidence.
- Infrastructure agents: Pulumi Neo, GitHub Agent HQ, and similar platforms will show whether AI agents become the primary interface for infrastructure engineering.
- Platform engineering consolidation: M&A activity across Kubernetes platforms, observability, and IaC will indicate whether enterprises want integrated platforms or best-of-breed tools.
- Cloud-native AI platforms: The integration of AI orchestration into Kubernetes distributions will determine whether Kubernetes becomes the default AI control plane.
Infrastructure software is not a background function. It is becoming AI's operational control plane—the layer that determines how efficiently, reliably, and autonomously expensive infrastructure operates.
The GPU utilization gap—5% actual versus 50% achievable—is the clearest evidence that infrastructure software matters. Kubernetes is becoming the default orchestration layer for AI workloads. Observability is evolving into autonomous operations. Infrastructure engineering is shifting from writing configuration to supervising AI systems that write configuration.
The companies that win will be those that recognize the infrastructure software layer as a strategic asset—not a cost center—and build for the requirements of AI workloads at scale. The next platform shift in enterprise technology will not be won by the companies that build the fastest chips. It will be won by the companies that build the software that controls them.
Sources
- DataAgent — Emerges From Stealth With $10M Pre-Seed for Autonomous SRE Platform (Sept 1, 2026)
- Empirik — Spins Out With $21M to Predict Outages Before They Hit (Sept 1, 2026)
- Pulumi — Infrastructure AI Documentation (Sept 8, 2026)
- CNCF — Your Kubernetes Platform Is Ready for Containers. Is It Ready for AI? (Aug 28, 2026)
- Datadog — DASH 2026 Announcements (June 10, 2026)
- Cast AI — 2026 State of Kubernetes Optimization Report
- Data Bridge Market Research — Global AI Infrastructure Management Software Market
- GitHub — Agent HQ Announcement (Sept 2026)
- TrueFoundry — Acquisition of Seldon AI (June 24, 2026)
- IREN — Acquisition of Mirantis (May 5, 2026)
- Cloud Native Now — DataAgent Emerges From Stealth
The CODEW Stat
Average GPU utilization across 23,000 Kubernetes clusters sits at approximately 5%. In optimized environments, it can reach 50%—meaning infrastructure software that closes that gap could reduce AI compute costs by several multiples. The AI infrastructure management software market is projected to grow from $8.27 billion in 2025 to $29.84 billion by 2033, a 17.4% CAGR.
Reviewed by Erwin Castro
on
Thursday, September 17, 2026
Rating:
